[Engine-devel] SSH Soft Fencing

Sun Jun 30 17:16:19 UTC 2013

On 06/30/2013 07:33 PM, Barak Azulay wrote:
> 
> 
> ----- Original Message -----
>> From: "Livnat Peer" <lpeer at redhat.com>
>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
>> Cc: engine-devel at ovirt.org
>> Sent: Sunday, June 30, 2013 8:06:28 AM
>> Subject: Re: [Engine-devel] SSH Soft Fencing
>>
>> On 06/30/2013 05:46 AM, Yair Zaslavsky wrote:
>>>
>>>
>>> ----- Original Message -----
>>>> From: "Barak Azulay" <bazulay at redhat.com>
>>>> To: "Martin Perina" <mperina at redhat.com>
>>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, engine-devel at ovirt.org, "Eli
>>>> Mesika" <emesika at redhat.com>
>>>> Sent: Thursday, June 27, 2013 8:31:35 PM
>>>> Subject: Re: SSH Soft Fencing
>>>>
>>>>
>>>>
>>>> ----- Original Message -----
>>>>> From: "Eli Mesika" <emesika at redhat.com>
>>>>> To: "Yair Zaslavsky" <yzaslavs at redhat.com>
>>>>> Cc: "Martin Perina" <mperina at redhat.com>, engine-devel at ovirt.org, "Barak
>>>>> Azulay" <bazulay at redhat.com>
>>>>> Sent: Thursday, June 27, 2013 5:55:29 PM
>>>>> Subject: Re: SSH Soft Fencing
>>>>>
>>>>>
>>>>>
>>>>> ----- Original Message -----
>>>>>> From: "Yair Zaslavsky" <yzaslavs at redhat.com>
>>>>>> To: "Eli Mesika" <emesika at redhat.com>
>>>>>> Cc: "Martin Perina" <mperina at redhat.com>, engine-devel at ovirt.org, "Barak
>>>>>> Azulay" <bazulay at redhat.com>
>>>>>> Sent: Thursday, June 27, 2013 5:43:17 PM
>>>>>> Subject: Re: SSH Soft Fencing
>>>>>>
>>>>>>
>>>>>>
>>>>>> ----- Original Message -----
>>>>>>> From: "Eli Mesika" <emesika at redhat.com>
>>>>>>> To: "Martin Perina" <mperina at redhat.com>
>>>>>>> Cc: engine-devel at ovirt.org, "Yair Zaslavsky" <yzaslavs at redhat.com>,
>>>>>>> "Barak
>>>>>>> Azulay" <bazulay at redhat.com>
>>>>>>> Sent: Thursday, June 27, 2013 3:48:39 PM
>>>>>>> Subject: Re: SSH Soft Fencing
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Martin Perina" <mperina at redhat.com>
>>>>>>>> To: engine-devel at ovirt.org
>>>>>>>> Cc: "Yair Zaslavsky" <yzaslavs at redhat.com>, "Barak Azulay"
>>>>>>>> <bazulay at redhat.com>, "Eli Mesika" <emesika at redhat.com>
>>>>>>>> Sent: Thursday, June 27, 2013 1:51:06 PM
>>>>>>>> Subject: SSH Soft Fencing
>>>>>>>>
>>>>>>>> Hi,
>>>>>>>>
>>>>>>>> SSH Soft Fencing is a new feature for 3.3 and it tries to restart
>>>>>>>> VDSM
>>>>>>>> using SSH connection on non responsive hosts prior to real fencing.
>>>>>>>> More info can be found at
>>>>>>>>
>>>>>>>> http://www.ovirt.org/Automatic_Fencing#Automatic_Fencing_in_oVirt_3.3
>>>>>>>>
>>>>>>>> In current SSH Soft Fencing implementation the restart VDSM using SSH
>>>>>>>> command is part of standard fencing implementation in
>>>>>>>> VdsNotRespondingTreatmentCommand. But this command is executed only
>>>>>>>> if a host has a valid PM configuration. If host doesn't have a valid
>>>>>>>> PM configuration, the execution of the command is disabled and host
>>>>>>>> state is change to Non Responsive.
>>>>>>>>
>>>>>>>> So my question are:
>>>>>>>>
>>>>>>>> 1) Should SSH Soft Fencing be executed on hosts without valid PM
>>>>>>>>    configuration?
>>>>>>>
>>>>>>> I think that the answer should be yes. The vdsm restart will solve most
>>>>>>> of
>>>>>>> problems , so why not using it whether a PM agent is defined or not.
>>>>>> I agree.
>>>>>> I would like to say that I also don't like the fact that
>>>>>> VdsNotRespondingTreatment extends RestartVdsCommand.
>>>>>> One should ask if "non responding treatment is a restart vds operation"
>>>>>> or
>>>>>> maybe RestartVdsCommand is just a step in the non responding treatment
>>>>>> (inheritance vs containment/delegation).
>>>>>> I think that VdsNotRespodingTreatment should delegate the call to
>>>>>> RestartVdsCommand as the 2nd step after issuing the Soft Fencing
>>>>>> command.
>>>>>> Thoughts anyone?
>>>>>
>>>>> That would be a nice and needed re-factoring
>>>>
>>>> I would say yes - but would add it only with appropriate configuration
>>>> (enableAutoSoftVdsmRestartWhenNoPMAvailable .... I hate the name)
>>>
>>> +1 on configuration.
>>> Configuration must reside at host-related entities (i.e - VdsStatic).
>>>
>>> Yair
>>>
>>
>> Why would a user like to avoid fencing VDSM when host becomes
>> non-responsive?
>>
>> I think that adding another configuration option is cumbersome with no
>> real value.
> 
> The problem i'm trying to solve is not about whether this is the right way to solve a case where vdsm is not responsive,
> But it's about changing the expected behavior in certain cases (and not be able to disable such a change).
> 

When adding enhancements many times you change system behavior.
For adding configuration option to enable the previous mode we need to
see if it make sense and if the system users are going to use it,
otherwise we are adding config option with no use.

Ending up with hundreds of configuration options only makes the
configuration cumbersome instead of useful.

> When a user does not configure a PM he does not expect any fencing to happen, and may be he wants to reach a state where VDSM is stuck .....
> (even for development purposes ...) and buy doing the automatic SSH fencing we are actually depriving him from thas option.
> 

The developers use case, in this case, does not sound like a good reason
for adding this configuration option but that is only my 2 cents.

> So let's enable him to reach that situation if he desires (by configuration), and put the default to automatically do the SSH + restart.   
> 
> 
>>
>> Livnat
>>
>>>>
>>>>  
>>>>
>>>>>
>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> 2) Should VDSM restart using SSH command be reimplemented
>>>>>>>>    as standalone command to be usable also in other parts of engine?
>>>>>>>>    If 1) is true, I think it will have to be done anyway.
>>>>>>
>>>>>> I agree here.
>>>>>>>
>>>>>>> +1
>>>>
>>>> On one hand it makes sense,  but I have several questions on the above:
>>>> - Who do we think may want to use such a command ?
>>>> - Should (or even can) we limit the use of such command to
>>>> noneResponsiveTreatment ?
>>>>
>>>> Having general commands available to all code when there is only one
>>>> specific
>>>> case we are using it might be a bit riskey,
>>>> Especially when we talk about restarting something.
>>>>
>>>> Thoughts ?
>>>>
>>>>
>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Martin Perina
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>> _______________________________________________
>>> Engine-devel mailing list
>>> Engine-devel at ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>>
>>
>> _______________________________________________
>> Engine-devel mailing list
>> Engine-devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/engine-devel
>>
>>
>>
> _______________________________________________
> Engine-devel mailing list
> Engine-devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/engine-devel
> 
> 