[Engine-devel] SSH Soft Fencing
Hi, SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3 In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive. So my question are: 1) Should SSH Soft Fencing be executed on hosts without valid PM configuration? 2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway. Martin Perina
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not.
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
+1
Martin Perina
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
Martin Perina
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
Martin Perina
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ? Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something. Thoughts ?
Martin Perina
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
+1 on configuration. Configuration must reside at host-related entities (i.e - VdsStatic). Yair
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Thoughts ?
Martin Perina
On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
+1 on configuration. Configuration must reside at host-related entities (i.e - VdsStatic).
Yair
Why would a user like to avoid fencing VDSM when host becomes non-responsive? I think that adding another configuration option is cumbersome with no real value. Livnat
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Thoughts ?
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Sunday, June 30, 2013 8:06:28 AM Subject: Re: [Engine-devel] SSH Soft Fencing
On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message ----- > From: "Martin Perina" <mperina@redhat.com> > To: engine-devel@ovirt.org > Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" > <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> > Sent: Thursday, June 27, 2013 1:51:06 PM > Subject: SSH Soft Fencing > > Hi, > > SSH Soft Fencing is a new feature for 3.3 and it tries to restart > VDSM > using SSH connection on non responsive hosts prior to real fencing. > More info can be found at > > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3 > > In current SSH Soft Fencing implementation the restart VDSM using SSH > command is part of standard fencing implementation in > VdsNotRespondingTreatmentCommand. But this command is executed only > if a host has a valid PM configuration. If host doesn't have a valid > PM configuration, the execution of the command is disabled and host > state is change to Non Responsive. > > So my question are: > > 1) Should SSH Soft Fencing be executed on hosts without valid PM > configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
+1 on configuration. Configuration must reside at host-related entities (i.e - VdsStatic).
Yair
Why would a user like to avoid fencing VDSM when host becomes non-responsive?
I think that adding another configuration option is cumbersome with no real value.
I totally agree with Livnat here , restarting vdsm may resolve problems and is much less brutal that host restart, should be implemented without any configuration IMHO
Livnat
> > 2) Should VDSM restart using SSH command be reimplemented > as standalone command to be usable also in other parts of engine? > If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Thoughts ?
> > > Martin Perina >
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Sunday, June 30, 2013 8:06:28 AM Subject: Re: [Engine-devel] SSH Soft Fencing
On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message ----- > From: "Martin Perina" <mperina@redhat.com> > To: engine-devel@ovirt.org > Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" > <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> > Sent: Thursday, June 27, 2013 1:51:06 PM > Subject: SSH Soft Fencing > > Hi, > > SSH Soft Fencing is a new feature for 3.3 and it tries to restart > VDSM > using SSH connection on non responsive hosts prior to real fencing. > More info can be found at > > http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3 > > In current SSH Soft Fencing implementation the restart VDSM using SSH > command is part of standard fencing implementation in > VdsNotRespondingTreatmentCommand. But this command is executed only > if a host has a valid PM configuration. If host doesn't have a valid > PM configuration, the execution of the command is disabled and host > state is change to Non Responsive. > > So my question are: > > 1) Should SSH Soft Fencing be executed on hosts without valid PM > configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
+1 on configuration. Configuration must reside at host-related entities (i.e - VdsStatic).
Yair
Why would a user like to avoid fencing VDSM when host becomes non-responsive?
I think that adding another configuration option is cumbersome with no real value.
The problem i'm trying to solve is not about whether this is the right way to solve a case where vdsm is not responsive, But it's about changing the expected behavior in certain cases (and not be able to disable such a change). When a user does not configure a PM he does not expect any fencing to happen, and may be he wants to reach a state where VDSM is stuck ..... (even for development purposes ...) and buy doing the automatic SSH fencing we are actually depriving him from thas option. So let's enable him to reach that situation if he desires (by configuration), and put the default to automatically do the SSH + restart.
Livnat
> > 2) Should VDSM restart using SSH command be reimplemented > as standalone command to be usable also in other parts of engine? > If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Thoughts ?
> > > Martin Perina >
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 06/30/2013 07:33 PM, Barak Azulay wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Sunday, June 30, 2013 8:06:28 AM Subject: Re: [Engine-devel] SSH Soft Fencing
On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message ----- > From: "Eli Mesika" <emesika@redhat.com> > To: "Martin Perina" <mperina@redhat.com> > Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, > "Barak > Azulay" <bazulay@redhat.com> > Sent: Thursday, June 27, 2013 3:48:39 PM > Subject: Re: SSH Soft Fencing > > > > ----- Original Message ----- >> From: "Martin Perina" <mperina@redhat.com> >> To: engine-devel@ovirt.org >> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" >> <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> >> Sent: Thursday, June 27, 2013 1:51:06 PM >> Subject: SSH Soft Fencing >> >> Hi, >> >> SSH Soft Fencing is a new feature for 3.3 and it tries to restart >> VDSM >> using SSH connection on non responsive hosts prior to real fencing. >> More info can be found at >> >> http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3 >> >> In current SSH Soft Fencing implementation the restart VDSM using SSH >> command is part of standard fencing implementation in >> VdsNotRespondingTreatmentCommand. But this command is executed only >> if a host has a valid PM configuration. If host doesn't have a valid >> PM configuration, the execution of the command is disabled and host >> state is change to Non Responsive. >> >> So my question are: >> >> 1) Should SSH Soft Fencing be executed on hosts without valid PM >> configuration? > > I think that the answer should be yes. The vdsm restart will solve most > of > problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
+1 on configuration. Configuration must reside at host-related entities (i.e - VdsStatic).
Yair
Why would a user like to avoid fencing VDSM when host becomes non-responsive?
I think that adding another configuration option is cumbersome with no real value.
The problem i'm trying to solve is not about whether this is the right way to solve a case where vdsm is not responsive, But it's about changing the expected behavior in certain cases (and not be able to disable such a change).
When adding enhancements many times you change system behavior. For adding configuration option to enable the previous mode we need to see if it make sense and if the system users are going to use it, otherwise we are adding config option with no use. Ending up with hundreds of configuration options only makes the configuration cumbersome instead of useful.
When a user does not configure a PM he does not expect any fencing to happen, and may be he wants to reach a state where VDSM is stuck ..... (even for development purposes ...) and buy doing the automatic SSH fencing we are actually depriving him from thas option.
The developers use case, in this case, does not sound like a good reason for adding this configuration option but that is only my 2 cents.
So let's enable him to reach that situation if he desires (by configuration), and put the default to automatically do the SSH + restart.
Livnat
> >> >> 2) Should VDSM restart using SSH command be reimplemented >> as standalone command to be usable also in other parts of engine? >> If 1) is true, I think it will have to be done anyway.
I agree here. > > +1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Thoughts ?
> >> >> >> Martin Perina >> >
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
Martin ? Eli? Yair? Can you please refer to the issue above ?
Thoughts ?
Martin Perina
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Sunday, June 30, 2013 7:35:22 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ? - Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
Yes, I think we should limit that to noneResponsiveTreatment
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
We can keep this internal and not expose it to the user, just implement explicitly in non responding treatment
Martin ? Eli? Yair?
Can you please refer to the issue above ?
Thoughts ?
Martin Perina
On 06/30/2013 07:35 PM, Barak Azulay wrote:
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, engine-devel@ovirt.org, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 8:31:35 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:55:29 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Yair Zaslavsky" <yzaslavs@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: "Martin Perina" <mperina@redhat.com>, engine-devel@ovirt.org, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 5:43:17 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
That would be a nice and needed re-factoring
I would say yes - but would add it only with appropriate configuration (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
On one hand it makes sense, but I have several questions on the above: - Who do we think may want to use such a command ?
I believe you'll agree that right encapsulation and decoupling is part of writing a maintainable code, it is not necessarily about reusing it.
- Should (or even can) we limit the use of such command to noneResponsiveTreatment ?
At this point this command would be executed only within the noneResponsiveTreatment flow. We don't need to model this in the REST API nor in the UI, decoupling the vdsm fencing code is just an internal implementation detail.
Having general commands available to all code when there is only one specific case we are using it might be a bit riskey, Especially when we talk about restarting something.
I am not sure what is the risk?
Martin ? Eli? Yair?
Can you please refer to the issue above ?
Thoughts ?
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
So let me summarize it: We have come to agreement in those questions: 1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand 2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow These questions has not been resolved yet: 3) Should SSH Soft Fencing be executed also for hosts without PM configured? 4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)? 5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)? Personally I would suggest: ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me Martin
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
Martin _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
+1
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
+1
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
Martin _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
+1
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
+1
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
+1 on all above as well
Martin _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
+1
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
+1
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
+1 on all above as well
Martin _______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality: 1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed. 2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive. 3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it 4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database. The whole fencing process in oVirt 3.3 is decribed at http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3 Martin Perina ----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 12:03:13 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
+1
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
+1
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
Hi Martin, I have some more questions, - Do we persist the host root password for this feature? - If we do, is this feature limited for new hosts, can I provide it for already existing hosts? - Do we encrypt this value when storing in the DB? Thanks, Livnat On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
Martin Perina
----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 12:03:13 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Monday, July 1, 2013 11:23:12 AM Subject: Re: [Engine-devel] SSH Soft Fencing
So let me summarize it:
We have come to agreement in those questions:
1) SSH Soft Fencing logic should be extracted from VdsNotRespondingTreatment command to its own SshSoftFencingCommand
2) VdsNotRespondingCommand should be refactored so it's not inherited from VdsRestartCommand, but it should run SshSoftFencingCommand or VdsRestartCommand based on defined fencing flow
These questions has not been resolved yet:
3) Should SSH Soft Fencing be executed also for hosts without PM configured?
4) Should SSH Soft Fencing execution for hosts without PM configured be enabled by default and admin can turn off these feature using configuration options SshSoftFencingWithoutPmEnabled (or something like that)?
5) Should SshSoftFencingWithoutPmEnabled be a global option or a cluster wide option (can be turned off for specific cluster version) or a VDS option (it can be turned off for each host)?
Personally I would suggest:
ad 3) Yes, SSH Soft Fencing should be executed also for hosts without PM configured
+1
ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be enabled by default
+1
ad 5) I don't see any significant reason why someone would like to turn off SSH Soft Fencing for hosts without PM configured. But if someone would like to do that, I think he would like to turn it off only for specific hosts, so VDS level option makes sense for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 2:32:16 PM Subject: Re: [Engine-devel] SSH Soft Fencing
Hi Martin, I have some more questions, - Do we persist the host root password for this feature? - If we do, is this feature limited for new hosts, can I provide it for already existing hosts? - Do we encrypt this value when storing in the DB?
Thanks, Livnat
Well, SSH connection uses engine default SSH key, no password. So I think this is usable for all hosts.
On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
Martin Perina
----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 12:03:13 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote:
----- Original Message ----- > From: "Martin Perina" <mperina@redhat.com> > To: engine-devel@ovirt.org > Sent: Monday, July 1, 2013 11:23:12 AM > Subject: Re: [Engine-devel] SSH Soft Fencing > > So let me summarize it: > > We have come to agreement in those questions: > > 1) SSH Soft Fencing logic should be extracted from > VdsNotRespondingTreatment > command to its own SshSoftFencingCommand > > 2) VdsNotRespondingCommand should be refactored so it's not inherited > from > VdsRestartCommand, but it should run SshSoftFencingCommand > or VdsRestartCommand based on defined fencing flow > > > These questions has not been resolved yet: > > 3) Should SSH Soft Fencing be executed also for hosts without PM > configured? > > 4) Should SSH Soft Fencing execution for hosts without PM configured > be > enabled > by default and admin can turn off these feature using configuration > options > SshSoftFencingWithoutPmEnabled (or something like that)? > > 5) Should SshSoftFencingWithoutPmEnabled be a global option or a > cluster > wide > option (can be turned off for specific cluster version) or a VDS > option > (it can be turned off for each host)? > > > Personally I would suggest: > > ad 3) Yes, SSH Soft Fencing should be executed also for hosts without > PM > configured >
+1
> ad 4) Yes, SSH Soft Fencing for hosts without PM configured should be > enabled by default >
+1
> ad 5) I don't see any significant reason why someone would like to > turn > off > SSH Soft Fencing > for hosts without PM configured. But if someone would like to do > that, > I think > he would like to turn it off only for specific hosts, so VDS > level > option makes sense > for me
After re-thinking 5 - I agree. +1 on the other suggestions, but of course we need to get more consensus here.
I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 3:39:20 PM Subject: Re: [Engine-devel] SSH Soft Fencing
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 2:32:16 PM Subject: Re: [Engine-devel] SSH Soft Fencing
Hi Martin, I have some more questions, - Do we persist the host root password for this feature? - If we do, is this feature limited for new hosts, can I provide it for already existing hosts? - Do we encrypt this value when storing in the DB?
Thanks, Livnat
Well, SSH connection uses engine default SSH key, no password. So I think this is usable for all hosts.
correct, SSH public key is deployed as a part of bootstrap & host-deploy from day one. So no need to save the password anywhere. Marin - Where do you take the username (currently only root is supported), but we intend to move away from it, Actually Yaniv.B as added it to the DB as first stage - just making sure you'll use that. Thanks Barak
On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
Martin Perina
----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 12:03:13 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Yair Zaslavsky" <yzaslavs@redhat.com> Cc: engine-devel@ovirt.org Sent: Monday, July 1, 2013 12:57:34 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/01/2013 11:27 AM, Yair Zaslavsky wrote: > > > ----- Original Message ----- >> From: "Martin Perina" <mperina@redhat.com> >> To: engine-devel@ovirt.org >> Sent: Monday, July 1, 2013 11:23:12 AM >> Subject: Re: [Engine-devel] SSH Soft Fencing >> >> So let me summarize it: >> >> We have come to agreement in those questions: >> >> 1) SSH Soft Fencing logic should be extracted from >> VdsNotRespondingTreatment >> command to its own SshSoftFencingCommand >> >> 2) VdsNotRespondingCommand should be refactored so it's not >> inherited >> from >> VdsRestartCommand, but it should run SshSoftFencingCommand >> or VdsRestartCommand based on defined fencing flow >> >> >> These questions has not been resolved yet: >> >> 3) Should SSH Soft Fencing be executed also for hosts without PM >> configured? >> >> 4) Should SSH Soft Fencing execution for hosts without PM configured >> be >> enabled >> by default and admin can turn off these feature using >> configuration >> options >> SshSoftFencingWithoutPmEnabled (or something like that)? >> >> 5) Should SshSoftFencingWithoutPmEnabled be a global option or a >> cluster >> wide >> option (can be turned off for specific cluster version) or a VDS >> option >> (it can be turned off for each host)? >> >> >> Personally I would suggest: >> >> ad 3) Yes, SSH Soft Fencing should be executed also for hosts >> without >> PM >> configured >>
+1
>> ad 4) Yes, SSH Soft Fencing for hosts without PM configured should >> be >> enabled by default >>
+1
>> ad 5) I don't see any significant reason why someone would like to >> turn >> off >> SSH Soft Fencing >> for hosts without PM configured. But if someone would like to >> do >> that, >> I think >> he would like to turn it off only for specific hosts, so VDS >> level >> option makes sense >> for me > > After re-thinking 5 - I agree. > +1 on the other suggestions, but of course we need to get more > consensus > here. >
I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Barak Azulay" <bazulay@redhat.com> To: "Martin Perina" <mperina@redhat.com>, "Yaniv Bronheim" <ybronhei@redhat.com> Cc: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 5:24:21 PM Subject: Re: [Engine-devel] SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 3:39:20 PM Subject: Re: [Engine-devel] SSH Soft Fencing
----- Original Message -----
From: "Livnat Peer" <lpeer@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 2:32:16 PM Subject: Re: [Engine-devel] SSH Soft Fencing
Hi Martin, I have some more questions, - Do we persist the host root password for this feature? - If we do, is this feature limited for new hosts, can I provide it for already existing hosts? - Do we encrypt this value when storing in the DB?
Thanks, Livnat
Well, SSH connection uses engine default SSH key, no password. So I think this is usable for all hosts.
correct, SSH public key is deployed as a part of bootstrap & host-deploy from day one. So no need to save the password anywhere.
Marin - Where do you take the username (currently only root is supported), but we intend to move away from it, Actually Yaniv.B as added it to the DB as first stage - just making sure you'll use that.
Thanks Barak
My 2 cents - Actually, Since Martin's work is merged, and Yaniv already has in one of his patches fixes to all relevant places in code he will probably need to add a fix around ssh soft fencing area (or at least coordinate this with Martin). Both Yaniv and Martin are already aware of this. Yair
On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
Martin Perina
----- Original Message -----
From: "Michal Skrivanek" <michal.skrivanek@redhat.com> To: engine-devel@ovirt.org Sent: Wednesday, July 3, 2013 12:03:13 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Jul 2, 2013, at 10:44 , Eli Mesika <emesika@redhat.com> wrote:
----- Original Message ----- > From: "Livnat Peer" <lpeer@redhat.com> > To: "Yair Zaslavsky" <yzaslavs@redhat.com> > Cc: engine-devel@ovirt.org > Sent: Monday, July 1, 2013 12:57:34 PM > Subject: Re: [Engine-devel] SSH Soft Fencing > > On 07/01/2013 11:27 AM, Yair Zaslavsky wrote: >> >> >> ----- Original Message ----- >>> From: "Martin Perina" <mperina@redhat.com> >>> To: engine-devel@ovirt.org >>> Sent: Monday, July 1, 2013 11:23:12 AM >>> Subject: Re: [Engine-devel] SSH Soft Fencing >>> >>> So let me summarize it: >>> >>> We have come to agreement in those questions: >>> >>> 1) SSH Soft Fencing logic should be extracted from >>> VdsNotRespondingTreatment >>> command to its own SshSoftFencingCommand >>> >>> 2) VdsNotRespondingCommand should be refactored so it's not >>> inherited >>> from >>> VdsRestartCommand, but it should run SshSoftFencingCommand >>> or VdsRestartCommand based on defined fencing flow >>> >>> >>> These questions has not been resolved yet: >>> >>> 3) Should SSH Soft Fencing be executed also for hosts without PM >>> configured? >>> >>> 4) Should SSH Soft Fencing execution for hosts without PM >>> configured >>> be >>> enabled >>> by default and admin can turn off these feature using >>> configuration >>> options >>> SshSoftFencingWithoutPmEnabled (or something like that)? >>> >>> 5) Should SshSoftFencingWithoutPmEnabled be a global option or a >>> cluster >>> wide >>> option (can be turned off for specific cluster version) or a VDS >>> option >>> (it can be turned off for each host)? >>> >>> >>> Personally I would suggest: >>> >>> ad 3) Yes, SSH Soft Fencing should be executed also for hosts >>> without >>> PM >>> configured >>> > > > +1 > >>> ad 4) Yes, SSH Soft Fencing for hosts without PM configured should >>> be >>> enabled by default >>> > > +1 > >>> ad 5) I don't see any significant reason why someone would like to >>> turn >>> off >>> SSH Soft Fencing >>> for hosts without PM configured. But if someone would like >>> to >>> do >>> that, >>> I think >>> he would like to turn it off only for specific hosts, so VDS >>> level >>> option makes sense >>> for me >> >> After re-thinking 5 - I agree. >> +1 on the other suggestions, but of course we need to get more >> consensus >> here. >> > > I think it does not need to be configurable.
I think a configuration option, as cumbersome and confusing as it can be, is still better than no choice. Especially if it means to restore the previous behavior. If it only can happen in a theoretical problem at customer where vdsm restart cause issues for whatever theoretical reason….it would be of great help then. And if you don't understand the parameter - just don't touch it, I hope that's a general rule:-)
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
I assume ssh soft fencing isn't considered "fencing"? i.e., i assume it does not release resource occupied by the host (spm, running vms)? Thanks, Itamar
----- Original Message -----
From: "Itamar Heim" <iheim@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org Sent: Tuesday, July 16, 2013 3:27:18 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On 07/03/2013 02:55 PM, Martin Perina wrote:
Let's summarize again, SSH Soft Fencing patches has been merged yesterday with following functionality:
1) For hosts with power management configured, SSH Soft Fencing is the 1st fencing stage. If it doesn't help, real fencing will be executed.
2) For hosts without power management configured, SSH Soft Fencing is the only fencing stage. If it doesn't help, host will become non responsive.
3) SSH Soft Fencing is enabled by default, there's no configuration option to disable it
4) SshSoftFencingCommand option is used to define what command is executed during SSH Soft Fencing. It can only be changed manually in database.
The whole fencing process in oVirt 3.3 is decribed at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
I assume ssh soft fencing isn't considered "fencing"? i.e., i assume it does not release resource occupied by the host (spm, running vms)?
it should. the correct way to release resources when killing vdsm, as it implemented now, is to send SIGTERM to vdsm process, as sanlock does. but it might take time (more than 40 sec) currently we just kill and start over the process with the service tool. SIGTERM calls to prepareForShutdown, it might be necessary if after the restart the engine initiates failover. "stop" verb does send SIGTERM before SIGKILL to vdsm, we just need verify that it waits enough time before sending the KILL and vdsm initiated the prepareForShutdown scope.
Thanks, Itamar
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 06/27/2013 05:43 PM, Yair Zaslavsky wrote:
----- Original Message -----
From: "Eli Mesika" <emesika@redhat.com> To: "Martin Perina" <mperina@redhat.com> Cc: engine-devel@ovirt.org, "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com> Sent: Thursday, June 27, 2013 3:48:39 PM Subject: Re: SSH Soft Fencing
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems , so why not using it whether a PM agent is defined or not. I agree. I would like to say that I also don't like the fact that VdsNotRespondingTreatment extends RestartVdsCommand. One should ask if "non responding treatment is a restart vds operation" or maybe RestartVdsCommand is just a step in the non responding treatment (inheritance vs containment/delegation). I think that VdsNotRespodingTreatment should delegate the call to RestartVdsCommand as the 2nd step after issuing the Soft Fencing command. Thoughts anyone?
I agree. The purpose of this feature is to add escalation step when handling non responsive host. Power fencing is only a step in the escalation flow. so should be called from within the main flow controller (the VdsNotRespodingTreatment). Maybe we'd like this to be fine tuned by a custom policy in future versions.
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
I agree here.
+1
+1 The VDSM restart is a step in the escalation flow, and it should not be tightly coupled with the non-responsive treatment implementation.
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems
Would you enumerate the problems that would be solved by a vdsm restart (on list, but on the feature page, too)? I am aware of two issues, both are vdsm bugs: - If libvirtd crashes, vdsm not is not restarted unless there are running VMs - Vdsm had several bugs in its soft prepareForShutdown process, getting itself stuck there in case of various background storage processes. I think that solving these two issues would be safer and cleaner than introducing `ssh host service vdsmd restart` flow. The first issue is only a matter of untangling some vdsm internal ugliness: whenever a libvirtconnection is produced, it should be wrapped so that it cathces libvirt crashes. Unlike now, where only VM-related libvirtconnection undergo this treatment. The second issue can be avoiding by vdsm resorting to kill-9-ing itself. After all, this is what `service vdsmd restart` ends up doing after a VERY short timeout (2-3 seconds, iicr). I suppose that there are other reasoning for a remote restart, but in general, I think that it's better to have Vdsm "do the right thing" than expecting Engine to control that remotely. Regards, Dan.
, so why not using it whether a PM agent is defined or not.
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
+1
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: engine-devel@ovirt.org Sent: Sunday, June 30, 2013 5:40:49 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems
Would you enumerate the problems that would be solved by a vdsm restart (on list, but on the feature page, too)? I am aware of two issues, both are vdsm bugs: - If libvirtd crashes, vdsm not is not restarted unless there are running VMs - Vdsm had several bugs in its soft prepareForShutdown process, getting itself stuck there in case of various background storage processes.
I think that solving these two issues would be safer and cleaner than introducing `ssh host service vdsmd restart` flow.
The first issue is only a matter of untangling some vdsm internal ugliness: whenever a libvirtconnection is produced, it should be wrapped so that it cathces libvirt crashes. Unlike now, where only VM-related libvirtconnection undergo this treatment.
The second issue can be avoiding by vdsm resorting to kill-9-ing itself. After all, this is what `service vdsmd restart` ends up doing after a VERY short timeout (2-3 seconds, iicr).
I suppose that there are other reasoning for a remote restart, but in general, I think that it's better to have Vdsm "do the right thing" than expecting Engine to control that remotely.
theoretically you are absolutely right, but this is much more challenging when the platform you are using keeps changing and might introduce unfamiliar behaviors or bugs. You have enumerated several issues that we have encountered in the past and were fixed by us or by different components. - libvirt related - prepareForShutdown - ... I even remember some from SuperVDSM All the above eventually were handled brutally by the engine and caused the host to be entirely fenced and all running VMs were killed (and the service they gave went down). This is about trying to handle an unexpected situation in a more somewhat delicate manner that in most cases will save killing the VMs, in a scenario where the host is going to be fenced anyway Now the question Martin had raised is whether this functionality should be applied also when a host has no physical Power-Management device, Hopes this provides the info you refereed to. Thanks Barak Azulay
Regards, Dan.
, so why not using it whether a PM agent is defined or not.
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
+1
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
On 06/30/2013 07:26 PM, Barak Azulay wrote:
----- Original Message -----
From: "Dan Kenigsberg" <danken@redhat.com> To: "Eli Mesika" <emesika@redhat.com> Cc: engine-devel@ovirt.org Sent: Sunday, June 30, 2013 5:40:49 PM Subject: Re: [Engine-devel] SSH Soft Fencing
On Thu, Jun 27, 2013 at 08:48:39AM -0400, Eli Mesika wrote:
----- Original Message -----
From: "Martin Perina" <mperina@redhat.com> To: engine-devel@ovirt.org Cc: "Yair Zaslavsky" <yzaslavs@redhat.com>, "Barak Azulay" <bazulay@redhat.com>, "Eli Mesika" <emesika@redhat.com> Sent: Thursday, June 27, 2013 1:51:06 PM Subject: SSH Soft Fencing
Hi,
SSH Soft Fencing is a new feature for 3.3 and it tries to restart VDSM using SSH connection on non responsive hosts prior to real fencing. More info can be found at
http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
In current SSH Soft Fencing implementation the restart VDSM using SSH command is part of standard fencing implementation in VdsNotRespondingTreatmentCommand. But this command is executed only if a host has a valid PM configuration. If host doesn't have a valid PM configuration, the execution of the command is disabled and host state is change to Non Responsive.
So my question are:
1) Should SSH Soft Fencing be executed on hosts without valid PM configuration?
I think that the answer should be yes. The vdsm restart will solve most of problems
Would you enumerate the problems that would be solved by a vdsm restart (on list, but on the feature page, too)? I am aware of two issues, both are vdsm bugs: - If libvirtd crashes, vdsm not is not restarted unless there are running VMs - Vdsm had several bugs in its soft prepareForShutdown process, getting itself stuck there in case of various background storage processes.
I think that solving these two issues would be safer and cleaner than introducing `ssh host service vdsmd restart` flow.
The first issue is only a matter of untangling some vdsm internal ugliness: whenever a libvirtconnection is produced, it should be wrapped so that it cathces libvirt crashes. Unlike now, where only VM-related libvirtconnection undergo this treatment.
The second issue can be avoiding by vdsm resorting to kill-9-ing itself. After all, this is what `service vdsmd restart` ends up doing after a VERY short timeout (2-3 seconds, iicr).
I suppose that there are other reasoning for a remote restart, but in general, I think that it's better to have Vdsm "do the right thing" than expecting Engine to control that remotely.
theoretically you are absolutely right, but this is much more challenging when the platform you are using keeps changing and might introduce unfamiliar behaviors or bugs. You have enumerated several issues that we have encountered in the past and were fixed by us or by different components. - libvirt related - prepareForShutdown - ... I even remember some from SuperVDSM
All the above eventually were handled brutally by the engine and caused the host to be entirely fenced and all running VMs were killed (and the service they gave went down).
This is about trying to handle an unexpected situation in a more somewhat delicate manner that in most cases will save killing the VMs, in a scenario where the host is going to be fenced anyway
+1 We can not anticipates our own bugs ;)
Now the question Martin had raised is whether this functionality should be applied also when a host has no physical Power-Management device,
Hopes this provides the info you refereed to.
Thanks Barak Azulay
Regards, Dan.
, so why not using it whether a PM agent is defined or not.
2) Should VDSM restart using SSH command be reimplemented as standalone command to be usable also in other parts of engine? If 1) is true, I think it will have to be done anyway.
+1
Martin Perina
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
_______________________________________________ Engine-devel mailing list Engine-devel@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-devel
participants (9)
-
Barak Azulay -
Dan Kenigsberg -
Eli Mesika -
Itamar Heim -
Livnat Peer -
Martin Perina -
Michal Skrivanek -
Yair Zaslavsky -
Yaniv Bronheim