[ovirt-devel] [vdsm] vdsm hosts clock sync

Wed Apr 8 07:03:42 UTC 2015

On Apr 7, 2015, at 17:50 , Barak Azulay <bazulay at redhat.com> wrote:

> 
> 
> ----- Original Message -----
>> From: "Shahar Havivi" <shaharh at redhat.com>
>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>> Cc: "devel" <devel at linode01.ovirt.org>
>> Sent: Monday, April 6, 2015 3:56:25 PM
>> Subject: Re: [ovirt-devel] [vdsm] vdsm hosts clock sync
>> 
>> On 06.04.15 08:12, Alon Bar-Lev wrote:
>>> 
>>> 
>>> ----- Original Message -----
>>>> From: "Shahar Havivi" <shaharh at redhat.com>
>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>> Cc: vdsm-devel at lists.fedorahosted.org
>>>> Sent: Monday, April 6, 2015 3:05:20 PM
>>>> Subject: Re: [vdsm] vdsm hosts clock sync
>>>> 
>>>> On 06.04.15 08:00, Alon Bar-Lev wrote:
>>>>> 
>>>>> 
>>>>> ----- Original Message -----
>>>>>> From: "Shahar Havivi" <shaharh at redhat.com>
>>>>>> To: "Alon Bar-Lev" <alonbl at redhat.com>
>>>>>> Cc: vdsm-devel at lists.fedorahosted.org
>>>>>> Sent: Monday, April 6, 2015 2:54:07 PM
>>>>>> Subject: Re: [vdsm] vdsm hosts clock sync
>>>>>> 
>>>>>> On 06.04.15 07:50, Alon Bar-Lev wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> ----- Original Message -----
>>>>>>>> From: "Shahar Havivi" <shaharh at redhat.com>
>>>>>>>> To: vdsm-devel at lists.fedorahosted.org
>>>>>>>> Sent: Monday, April 6, 2015 2:44:06 PM
>>>>>>>> Subject: [vdsm] vdsm hosts clock sync
>>>>>>>> 
>>>>>>>> Hi,
>>>>>>>> 
>>>>>>>> I want to add a new feature that reports migration actual
>>>>>>>> downtime
>>>>>>>> (the
>>>>>>>> time
>>>>>>>> that the VM was inaccessible to the user).
>>>>>>>> 
>>>>>>>> Libvirt reports that information but the vdsm hosts need to be in
>>>>>>>> sync by
>>>>>>>> clock
>>>>>>>> time.
>>>>>>>> I can measure the ping for NTP server an report back to the user
>>>>>>>> if
>>>>>>>> the
>>>>>>>> ping
>>>>>>>> is too long (more then ~100ms or so) - a way to do that is via
>>>>>>>> ntpstat
>>>>>>>> shell
>>>>>>>> command.
>>>>>>>> The NTP delay can be report back via vdsStats and can be
>>>>>>>> performed
>>>>>>>> every
>>>>>>>> few hours or so.
>>>>>>>> 
>>>>>>>> Anyone knows of a better way that we can sync between hosts?
>>>>>>> 
>>>>>>> I am unsure how a ping to clock source is helping, can you please
>>>>>>> explain
>>>>>>> more?
>>>>>> In this case I can only report back to the user that its hosts clock
>>>>>> is
>>>>>> delayed and need to be set...
>>>>>>> 
>>>>>>> If you assume clocks are synced why anything more is needed?
>>>>>> Why do I assume that?
>>>>>>> 
>>>>>>> Or would you like to have a solution in which you do not require
>>>>>>> clock
>>>>>>> sync?
>>>>>> 
>>>>>> I do need the clock to be in sync - if not libvirts "actual downtime
>>>>>> migration" will be not accurate.
>>>>> 
>>>>> you do not need clock to sync, you need to know the delta between
>>>>> hosts.
>>>>> 
>>>>> but if you assume clock are in sync so what is the actual question?
>>>> 
>>>> As I understand from your answer is by having configured ntp the hosts
>>>> clock
>>>> are in sync.
>>> 
>>> I do  not follow.
>>> 
>>> 1. if you are using ntpd on hosts and hosts are in sync you can use
>>> distributed timestamps.
>>> 
>>> detecting if host is synced can be done using either (based on what
>>> actually installed):
>>> 
>>> ntpq -c rv | grep -q clock_sync && echo ok
>>> chronyc waitsync 1 && echo ok
>>> 
>>> 2. you can also measure the time diff between hosts by clockdiff or similar
>>> process and avoid the need of ntpd deployed.
>>> 
>>> 3. you may be able to calculate the time within in the handover is given to
>>> remote and eliminate the network time.
>>> 
>>> Not sure what your implementation is.
>> Thanks for the info I may use the clockdiff...,
>> 
>> Let me describe the problem in another way:
>> We want to report to the user how much time it took the VM to migrate (the
>> time that the VM was offline),
>> Libvirt present this feature with one constraint hosts need to be in sync or
>> the measure will be inaccurate.
> 
> When you say libvirt present this feature than it means there need to be a single libvirt that report this down time (either source or destination),
> And the accuracy relies on the time between src & dst to be in sync.

that's correct

> 
> So when you assume clocks are in sync than the reported downtime is accurate.
> We assume hosts time are in sync (and we recommend use NTP on all hypervisors + engine for that).

we assume, but we don't make sure

> 
> So I don't understand what calculation you need to do.

no calculation, but we want reporting that the hosts are indeed synchronized and if possible with what accuracy

> 
> 
>> 
>> Solutions can be:
>> 1. report back to the user that hosts are drifting
> 
> IIRC we already have an event on this 

yes, but it's a poor-man's implementation, it only sends a timestamp and the engine compares it with its own time. So you can still be way off the real clock (not that we really care, though it would cause  problem with e.g. a console ticket validation) and in case of transport delays between host and engine this is also inaccurate
So ideally I'd like to report the NTP status, but I'm open to any other suggestion

> 
>> 2. sync the hosts time automatically
> 
> this is the role of NTP (and we assume the hosts are in sync)

how can we assume when we do not check?

Thanks,
michal

> 
>> 3. or other suitable solution
>> 
>> Thanks
>>> 
>>> Alon
>> _______________________________________________
>> Devel mailing list
>> Devel at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/devel
>> 
>> 
>> 
> _______________________________________________
> Devel mailing list
> Devel at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/devel