11:25 a.m.
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
1:21 p.m.
You can use FC storage where luns come from a redundant storage array.
On Apr 15, 2017 11:25, "Konstantin Raskoshnyi" <konrasko(a)gmail.com>
wrote:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
5:31 p.m.
Well, make it not go through host1 and dedicate a storage server for
running NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other type of storage,
specially FC and iSCSI and performance is pretty much the same, so you
won`t get better results other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:42 p.m.
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <fernando.frediani(a)upx.com>
wrote:
Well, make it not go through host1 and dedicate a storage server for
running NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other type of storage,
specially FC and iSCSI and performance is pretty much the same, so you
won`t get better results other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
> Hi guys,
> I have one nfs storage,
> it's connected through host1.
> host2 also has access to it, I can easily migrate vms between them.
>
> The question is - if host1 is down - all infrastructure is down, since
> all traffic goes through host1,
> is there any way in oVirt to use redundant storage?
>
> Only glusterfs?
>
> Thanks
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
8:37 p.m.
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host.
From what I know any host talk directly to NFS Storage Array or
whatever
other Shared Storage you have.
Have you tested that host going down if that affects the other with the NFS
mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
> Well, make it not go through host1 and dedicate a storage server for
> running NFS and make both hosts connect to it.
> In my view NFS is much easier to manage than any other type of storage,
> specially FC and iSCSI and performance is pretty much the same, so you
> won`t get better results other than management going to other type.
>
> Fernando
>
> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>
>> Hi guys,
>> I have one nfs storage,
>> it's connected through host1.
>> host2 also has access to it, I can easily migrate vms between them.
>>
>> The question is - if host1 is down - all infrastructure is down, since
>> all traffic goes through host1,
>> is there any way in oVirt to use redundant storage?
>>
>> Only glusterfs?
>>
>> Thanks
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
8:40 p.m.
I think he means SPM? I've seen when node with SPM goes down it isn't
really seamless and takes a minute or two to catch up unless of course you
out that node in maintenance mode but that isn't possible if it crashes or
something
On Apr 15, 2017 1:38 PM, "FERNANDO FREDIANI" <fernando.frediani(a)upx.com>
wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host.
From what I know any host talk directly to NFS Storage Array or whatever
other Shared Storage you have.
Have you tested that host going down if that affects the other with the
NFS mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
> In ovirt you have to attach storage through specific host.
> If host goes down storage is not available.
>
> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
>> Well, make it not go through host1 and dedicate a storage server for
>> running NFS and make both hosts connect to it.
>> In my view NFS is much easier to manage than any other type of storage,
>> specially FC and iSCSI and performance is pretty much the same, so you
>> won`t get better results other than management going to other type.
>>
>> Fernando
>>
>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>>> Hi guys,
>>> I have one nfs storage,
>>> it's connected through host1.
>>> host2 also has access to it, I can easily migrate vms between them.
>>>
>>> The question is - if host1 is down - all infrastructure is down, since
>>> all traffic goes through host1,
>>> is there any way in oVirt to use redundant storage?
>>>
>>> Only glusterfs?
>>>
>>> Thanks
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
8:53 p.m.
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host to
which I connected nfs storage going down the storage becomes unavailable
and all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host.
From what I know any host talk directly to NFS Storage Array or whatever
other Shared Storage you have.
Have you tested that host going down if that affects the other with the
NFS mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
> In ovirt you have to attach storage through specific host.
> If host goes down storage is not available.
>
> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
>> Well, make it not go through host1 and dedicate a storage server for
>> running NFS and make both hosts connect to it.
>> In my view NFS is much easier to manage than any other type of storage,
>> specially FC and iSCSI and performance is pretty much the same, so you
>> won`t get better results other than management going to other type.
>>
>> Fernando
>>
>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>>> Hi guys,
>>> I have one nfs storage,
>>> it's connected through host1.
>>> host2 also has access to it, I can easily migrate vms between them.
>>>
>>> The question is - if host1 is down - all infrastructure is down, since
>>> all traffic goes through host1,
>>> is there any way in oVirt to use redundant storage?
>>>
>>> Only glusterfs?
>>>
>>> Thanks
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
4:16 a.m.
When you set up a storage domain, you need to specify a host to perform the
initial storage operations, but once the SD is defined, it's details are in
the engine database, and all the hosts get connected to it directly. If the
first host you used to define the SD goes down, all other hosts will still
remain connected and work. SPM is an HA service, and if the current SPM
host goes down, SPM gets started on another host in the DC. In short,
unless your actual NFS exporting host goes down, there is no outage.
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host to
which I connected nfs storage going down the storage becomes unavailable
and all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
> Hello Konstantin.
>
> That doesn`t make much sense make a whole cluster depend on a single
> host. From what I know any host talk directly to NFS Storage Array or
> whatever other Shared Storage you have.
> Have you tested that host going down if that affects the other with the
> NFS mounted directlly in a NFS Storage array ?
>
> Fernando
>
> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>
>> In ovirt you have to attach storage through specific host.
>> If host goes down storage is not available.
>>
>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com> wrote:
>>
>>> Well, make it not go through host1 and dedicate a storage server for
>>> running NFS and make both hosts connect to it.
>>> In my view NFS is much easier to manage than any other type of storage,
>>> specially FC and iSCSI and performance is pretty much the same, so you
>>> won`t get better results other than management going to other type.
>>>
>>> Fernando
>>>
>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>>
>>>> Hi guys,
>>>> I have one nfs storage,
>>>> it's connected through host1.
>>>> host2 also has access to it, I can easily migrate vms between them.
>>>>
>>>> The question is - if host1 is down - all infrastructure is down, since
>>>> all traffic goes through host1,
>>>> is there any way in oVirt to use redundant storage?
>>>>
>>>> Only glusterfs?
>>>>
>>>> Thanks
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2 p.m.
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
When you set up a storage domain, you need to specify a host to
perform
the initial storage operations, but once the SD is defined, it's details
are in the engine database, and all the hosts get connected to it directly.
If the first host you used to define the SD goes down, all other hosts will
still remain connected and work. SPM is an HA service, and if the current
SPM host goes down, SPM gets started on another host in the DC. In short,
unless your actual NFS exporting host goes down, there is no outage.
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
Nir
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com
> wrote:
> Hi Fernando,
> I see each host has direct connection nfs mount, but yes, if main host to
> which I connected nfs storage going down the storage becomes unavailable
> and all vms are down
>
>
> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
>> Hello Konstantin.
>>
>> That doesn`t make much sense make a whole cluster depend on a single
>> host. From what I know any host talk directly to NFS Storage Array or
>> whatever other Shared Storage you have.
>> Have you tested that host going down if that affects the other with the
>> NFS mounted directlly in a NFS Storage array ?
>>
>> Fernando
>>
>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>>> In ovirt you have to attach storage through specific host.
>>> If host goes down storage is not available.
>>>
>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>> fernando.frediani(a)upx.com> wrote:
>>>
>>>> Well, make it not go through host1 and dedicate a storage server for
>>>> running NFS and make both hosts connect to it.
>>>> In my view NFS is much easier to manage than any other type of
>>>> storage, specially FC and iSCSI and performance is pretty much the same,
so
>>>> you won`t get better results other than management going to other type.
>>>>
>>>> Fernando
>>>>
>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>
>>>>> Hi guys,
>>>>> I have one nfs storage,
>>>>> it's connected through host1.
>>>>> host2 also has access to it, I can easily migrate vms between them.
>>>>>
>>>>> The question is - if host1 is down - all infrastructure is down,
>>>>> since all traffic goes through host1,
>>>>> is there any way in oVirt to use redundant storage?
>>>>>
>>>>> Only glusterfs?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:04 p.m.
On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
When you set up a storage domain, you need to specify a host to
perform
the initial storage operations, but once the SD is defined, it's details
are in the engine database, and all the hosts get connected to it directly.
If the first host you used to define the SD goes down, all other hosts will
still remain connected and work. SPM is an HA service, and if the current
SPM host goes down, SPM gets started on another host in the DC. In short,
unless your actual NFS exporting host goes down, there is no outage.
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's the
whole point of HA services
Nir
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com
> wrote:
> Hi Fernando,
> I see each host has direct connection nfs mount, but yes, if main host to
> which I connected nfs storage going down the storage becomes unavailable
> and all vms are down
>
>
> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
>> Hello Konstantin.
>>
>> That doesn`t make much sense make a whole cluster depend on a single
>> host. From what I know any host talk directly to NFS Storage Array or
>> whatever other Shared Storage you have.
>> Have you tested that host going down if that affects the other with the
>> NFS mounted directlly in a NFS Storage array ?
>>
>> Fernando
>>
>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>>> In ovirt you have to attach storage through specific host.
>>> If host goes down storage is not available.
>>>
>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>> fernando.frediani(a)upx.com> wrote:
>>>
>>>> Well, make it not go through host1 and dedicate a storage server for
>>>> running NFS and make both hosts connect to it.
>>>> In my view NFS is much easier to manage than any other type of
>>>> storage, specially FC and iSCSI and performance is pretty much the same,
so
>>>> you won`t get better results other than management going to other type.
>>>>
>>>> Fernando
>>>>
>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>
>>>>> Hi guys,
>>>>> I have one nfs storage,
>>>>> it's connected through host1.
>>>>> host2 also has access to it, I can easily migrate vms between them.
>>>>>
>>>>> The question is - if host1 is down - all infrastructure is down,
>>>>> since all traffic goes through host1,
>>>>> is there any way in oVirt to use redundant storage?
>>>>>
>>>>> Only glusterfs?
>>>>>
>>>>> Thanks
>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2:29 p.m.
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
> When you set up a storage domain, you need to specify a host to perform
> the initial storage operations, but once the SD is defined, it's details
> are in the engine database, and all the hosts get connected to it directly.
> If the first host you used to define the SD goes down, all other hosts will
> still remain connected and work. SPM is an HA service, and if the current
> SPM host goes down, SPM gets started on another host in the DC. In short,
> unless your actual NFS exporting host goes down, there is no outage.
>
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's
the whole point of HA services
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
Nir
Nir
>
> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Hi Fernando,
>> I see each host has direct connection nfs mount, but yes, if main host
>> to which I connected nfs storage going down the storage becomes unavailable
>> and all vms are down
>>
>>
>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com> wrote:
>>
>>> Hello Konstantin.
>>>
>>> That doesn`t make much sense make a whole cluster depend on a single
>>> host. From what I know any host talk directly to NFS Storage Array or
>>> whatever other Shared Storage you have.
>>> Have you tested that host going down if that affects the other with the
>>> NFS mounted directlly in a NFS Storage array ?
>>>
>>> Fernando
>>>
>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>>
>>>> In ovirt you have to attach storage through specific host.
>>>> If host goes down storage is not available.
>>>>
>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>> fernando.frediani(a)upx.com> wrote:
>>>>
>>>>> Well, make it not go through host1 and dedicate a storage server for
>>>>> running NFS and make both hosts connect to it.
>>>>> In my view NFS is much easier to manage than any other type of
>>>>> storage, specially FC and iSCSI and performance is pretty much the
same, so
>>>>> you won`t get better results other than management going to other
type.
>>>>>
>>>>> Fernando
>>>>>
>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>>
>>>>>> Hi guys,
>>>>>> I have one nfs storage,
>>>>>> it's connected through host1.
>>>>>> host2 also has access to it, I can easily migrate vms between
them.
>>>>>>
>>>>>> The question is - if host1 is down - all infrastructure is down,
>>>>>> since all traffic goes through host1,
>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>
>>>>>> Only glusterfs?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>>
>>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
7:44 p.m.
So what's the whole HA point then?
On Sun, Apr 16, 2017 at 4:29 AM Nir Soffer <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com>
wrote:
>
>
> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
>
> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> When you set up a storage domain, you need to specify a host to perform
>> the initial storage operations, but once the SD is defined, it's details
>> are in the engine database, and all the hosts get connected to it directly.
>> If the first host you used to define the SD goes down, all other hosts will
>> still remain connected and work. SPM is an HA service, and if the current
>> SPM host goes down, SPM gets started on another host in the DC. In short,
>> unless your actual NFS exporting host goes down, there is no outage.
>>
>
> There is no storage outage, but if you shutdown the spm host, the spm host
> will not move to a new host until the spm host is online again, or you
> confirm
> manually that the spm host was rebooted.
>
>
> In a properly configured setup the SBA should take care of that. That's
> the whole point of HA services
>
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
Nir
>
>
> Nir
>
>
>>
>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> Hi Fernando,
>>> I see each host has direct connection nfs mount, but yes, if main host
>>> to which I connected nfs storage going down the storage becomes unavailable
>>> and all vms are down
>>>
>>>
>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>> fernando.frediani(a)upx.com> wrote:
>>>
>>>> Hello Konstantin.
>>>>
>>>> That doesn`t make much sense make a whole cluster depend on a single
>>>> host. From what I know any host talk directly to NFS Storage Array or
>>>> whatever other Shared Storage you have.
>>>> Have you tested that host going down if that affects the other with
>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>
>>>> Fernando
>>>>
>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>
>>>>> In ovirt you have to attach storage through specific host.
>>>>> If host goes down storage is not available.
>>>>>
>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>
>>>>>> Well, make it not go through host1 and dedicate a storage server
for
>>>>>> running NFS and make both hosts connect to it.
>>>>>> In my view NFS is much easier to manage than any other type of
>>>>>> storage, specially FC and iSCSI and performance is pretty much
the same, so
>>>>>> you won`t get better results other than management going to other
type.
>>>>>>
>>>>>> Fernando
>>>>>>
>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>
>>>>>> :
>>>>>>
>>>>>>> Hi guys,
>>>>>>> I have one nfs storage,
>>>>>>> it's connected through host1.
>>>>>>> host2 also has access to it, I can easily migrate vms between
them.
>>>>>>>
>>>>>>> The question is - if host1 is down - all infrastructure is
down,
>>>>>>> since all traffic goes through host1,
>>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>>
>>>>>>> Only glusterfs?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
6:10 a.m.
On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com>
wrote:
>
>
> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
>
> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> When you set up a storage domain, you need to specify a host to perform
>> the initial storage operations, but once the SD is defined, it's details
>> are in the engine database, and all the hosts get connected to it directly.
>> If the first host you used to define the SD goes down, all other hosts will
>> still remain connected and work. SPM is an HA service, and if the current
>> SPM host goes down, SPM gets started on another host in the DC. In short,
>> unless your actual NFS exporting host goes down, there is no outage.
>>
>
> There is no storage outage, but if you shutdown the spm host, the spm host
> will not move to a new host until the spm host is online again, or you
> confirm
> manually that the spm host was rebooted.
>
>
> In a properly configured setup the SBA should take care of that. That's
> the whole point of HA services
>
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
There are always corner cases, no doubt. But in a normal situation. where
an SPM host goes down because of a hardware failure, it gets fenced, other
hosts contend for SPM and start it. No surprises there.
Nir
>
>
> Nir
>
>
>>
>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> Hi Fernando,
>>> I see each host has direct connection nfs mount, but yes, if main host
>>> to which I connected nfs storage going down the storage becomes unavailable
>>> and all vms are down
>>>
>>>
>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>> fernando.frediani(a)upx.com> wrote:
>>>
>>>> Hello Konstantin.
>>>>
>>>> That doesn`t make much sense make a whole cluster depend on a single
>>>> host. From what I know any host talk directly to NFS Storage Array or
>>>> whatever other Shared Storage you have.
>>>> Have you tested that host going down if that affects the other with
>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>
>>>> Fernando
>>>>
>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>
>>>>> In ovirt you have to attach storage through specific host.
>>>>> If host goes down storage is not available.
>>>>>
>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>
>>>>>> Well, make it not go through host1 and dedicate a storage server
for
>>>>>> running NFS and make both hosts connect to it.
>>>>>> In my view NFS is much easier to manage than any other type of
>>>>>> storage, specially FC and iSCSI and performance is pretty much
the same, so
>>>>>> you won`t get better results other than management going to other
type.
>>>>>>
>>>>>> Fernando
>>>>>>
>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>
>>>>>> :
>>>>>>
>>>>>>> Hi guys,
>>>>>>> I have one nfs storage,
>>>>>>> it's connected through host1.
>>>>>>> host2 also has access to it, I can easily migrate vms between
them.
>>>>>>>
>>>>>>> The question is - if host1 is down - all infrastructure is
down,
>>>>>>> since all traffic goes through host1,
>>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>>
>>>>>>> Only glusterfs?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
6:13 a.m.
"Corner cases"?
I tried to simulate crash of SPM server and ovirt kept trying to
reistablished connection to the failed node.
On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com> wrote:
> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
>
>>
>>
>> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com>
wrote:
>>
>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> When you set up a storage domain, you need to specify a host to perform
>>> the initial storage operations, but once the SD is defined, it's details
>>> are in the engine database, and all the hosts get connected to it directly.
>>> If the first host you used to define the SD goes down, all other hosts will
>>> still remain connected and work. SPM is an HA service, and if the current
>>> SPM host goes down, SPM gets started on another host in the DC. In short,
>>> unless your actual NFS exporting host goes down, there is no outage.
>>>
>>
>> There is no storage outage, but if you shutdown the spm host, the spm
>> host
>> will not move to a new host until the spm host is online again, or you
>> confirm
>> manually that the spm host was rebooted.
>>
>>
>> In a properly configured setup the SBA should take care of that. That's
>> the whole point of HA services
>>
>
> In some cases like power loss or hardware failure, there is no way to
> start
> the spm host, and the system cannot recover automatically.
>
There are always corner cases, no doubt. But in a normal situation. where
an SPM host goes down because of a hardware failure, it gets fenced, other
hosts contend for SPM and start it. No surprises there.
>
> Nir
>
>
>>
>>
>> Nir
>>
>>
>>>
>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>>> konrasko(a)gmail.com> wrote:
>>>
>>>> Hi Fernando,
>>>> I see each host has direct connection nfs mount, but yes, if main host
>>>> to which I connected nfs storage going down the storage becomes
unavailable
>>>> and all vms are down
>>>>
>>>>
>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>>> fernando.frediani(a)upx.com> wrote:
>>>>
>>>>> Hello Konstantin.
>>>>>
>>>>> That doesn`t make much sense make a whole cluster depend on a single
>>>>> host. From what I know any host talk directly to NFS Storage Array
or
>>>>> whatever other Shared Storage you have.
>>>>> Have you tested that host going down if that affects the other with
>>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>>
>>>>> Fernando
>>>>>
>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>
>>>>> :
>>>>>
>>>>>> In ovirt you have to attach storage through specific host.
>>>>>> If host goes down storage is not available.
>>>>>>
>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>
>>>>>>> Well, make it not go through host1 and dedicate a storage
server
>>>>>>> for running NFS and make both hosts connect to it.
>>>>>>> In my view NFS is much easier to manage than any other type
of
>>>>>>> storage, specially FC and iSCSI and performance is pretty
much the same, so
>>>>>>> you won`t get better results other than management going to
other type.
>>>>>>>
>>>>>>> Fernando
>>>>>>>
>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com
>>>>>>> >:
>>>>>>>
>>>>>>>> Hi guys,
>>>>>>>> I have one nfs storage,
>>>>>>>> it's connected through host1.
>>>>>>>> host2 also has access to it, I can easily migrate vms
between them.
>>>>>>>>
>>>>>>>> The question is - if host1 is down - all infrastructure
is down,
>>>>>>>> since all traffic goes through host1,
>>>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>>>
>>>>>>>> Only glusterfs?
>>>>>>>>
>>>>>>>> Thanks
>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users(a)ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
6:14 a.m.
On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
"Corner cases"?
I tried to simulate crash of SPM server and ovirt kept trying to
reistablished connection to the failed node.
Did you configure fencing?
On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>
>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
>>
>>>
>>>
>>> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com>
wrote:
>>>
>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>>>
>>>> When you set up a storage domain, you need to specify a host to
>>>> perform the initial storage operations, but once the SD is defined,
it's
>>>> details are in the engine database, and all the hosts get connected to
it
>>>> directly. If the first host you used to define the SD goes down, all
other
>>>> hosts will still remain connected and work. SPM is an HA service, and if
>>>> the current SPM host goes down, SPM gets started on another host in the
DC.
>>>> In short, unless your actual NFS exporting host goes down, there is no
>>>> outage.
>>>>
>>>
>>> There is no storage outage, but if you shutdown the spm host, the spm
>>> host
>>> will not move to a new host until the spm host is online again, or you
>>> confirm
>>> manually that the spm host was rebooted.
>>>
>>>
>>> In a properly configured setup the SBA should take care of that. That's
>>> the whole point of HA services
>>>
>>
>> In some cases like power loss or hardware failure, there is no way to
>> start
>> the spm host, and the system cannot recover automatically.
>>
>
> There are always corner cases, no doubt. But in a normal situation. where
> an SPM host goes down because of a hardware failure, it gets fenced, other
> hosts contend for SPM and start it. No surprises there.
>
>
>>
>> Nir
>>
>>
>>>
>>>
>>> Nir
>>>
>>>
>>>>
>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>>>> konrasko(a)gmail.com> wrote:
>>>>
>>>>> Hi Fernando,
>>>>> I see each host has direct connection nfs mount, but yes, if main
>>>>> host to which I connected nfs storage going down the storage becomes
>>>>> unavailable and all vms are down
>>>>>
>>>>>
>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>
>>>>>> Hello Konstantin.
>>>>>>
>>>>>> That doesn`t make much sense make a whole cluster depend on a
single
>>>>>> host. From what I know any host talk directly to NFS Storage
Array or
>>>>>> whatever other Shared Storage you have.
>>>>>> Have you tested that host going down if that affects the other
with
>>>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>>>
>>>>>> Fernando
>>>>>>
>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com
>>>>>> >:
>>>>>>
>>>>>>> In ovirt you have to attach storage through specific host.
>>>>>>> If host goes down storage is not available.
>>>>>>>
>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>
>>>>>>>> Well, make it not go through host1 and dedicate a storage
server
>>>>>>>> for running NFS and make both hosts connect to it.
>>>>>>>> In my view NFS is much easier to manage than any other
type of
>>>>>>>> storage, specially FC and iSCSI and performance is pretty
much the same, so
>>>>>>>> you won`t get better results other than management going
to other type.
>>>>>>>>
>>>>>>>> Fernando
>>>>>>>>
>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <
>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>
>>>>>>>>> Hi guys,
>>>>>>>>> I have one nfs storage,
>>>>>>>>> it's connected through host1.
>>>>>>>>> host2 also has access to it, I can easily migrate vms
between
>>>>>>>>> them.
>>>>>>>>>
>>>>>>>>> The question is - if host1 is down - all
infrastructure is down,
>>>>>>>>> since all traffic goes through host1,
>>>>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>>>>
>>>>>>>>> Only glusterfs?
>>>>>>>>>
>>>>>>>>> Thanks
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users(a)ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>>>
6:15 a.m.
Fence agent under each node?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
> "Corner cases"?
> I tried to simulate crash of SPM server and ovirt kept trying to
> reistablished connection to the failed node.
>
Did you configure fencing?
>
>
> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
>>>
>>>>
>>>>
>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>
>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>
>>>>> When you set up a storage domain, you need to specify a host to
>>>>> perform the initial storage operations, but once the SD is defined,
it's
>>>>> details are in the engine database, and all the hosts get connected
to it
>>>>> directly. If the first host you used to define the SD goes down, all
other
>>>>> hosts will still remain connected and work. SPM is an HA service, and
if
>>>>> the current SPM host goes down, SPM gets started on another host in
the DC.
>>>>> In short, unless your actual NFS exporting host goes down, there is
no
>>>>> outage.
>>>>>
>>>>
>>>> There is no storage outage, but if you shutdown the spm host, the spm
>>>> host
>>>> will not move to a new host until the spm host is online again, or you
>>>> confirm
>>>> manually that the spm host was rebooted.
>>>>
>>>>
>>>> In a properly configured setup the SBA should take care of that.
>>>> That's the whole point of HA services
>>>>
>>>
>>> In some cases like power loss or hardware failure, there is no way to
>>> start
>>> the spm host, and the system cannot recover automatically.
>>>
>>
>> There are always corner cases, no doubt. But in a normal situation.
>> where an SPM host goes down because of a hardware failure, it gets fenced,
>> other hosts contend for SPM and start it. No surprises there.
>>
>>
>>>
>>> Nir
>>>
>>>
>>>>
>>>>
>>>> Nir
>>>>
>>>>
>>>>>
>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>>>>> konrasko(a)gmail.com> wrote:
>>>>>
>>>>>> Hi Fernando,
>>>>>> I see each host has direct connection nfs mount, but yes, if
main
>>>>>> host to which I connected nfs storage going down the storage
becomes
>>>>>> unavailable and all vms are down
>>>>>>
>>>>>>
>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>
>>>>>>> Hello Konstantin.
>>>>>>>
>>>>>>> That doesn`t make much sense make a whole cluster depend on
a
>>>>>>> single host. From what I know any host talk directly to NFS
Storage Array
>>>>>>> or whatever other Shared Storage you have.
>>>>>>> Have you tested that host going down if that affects the
other with
>>>>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>>>>
>>>>>>> Fernando
>>>>>>>
>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <
>>>>>>> konrasko(a)gmail.com>:
>>>>>>>
>>>>>>>> In ovirt you have to attach storage through specific
host.
>>>>>>>> If host goes down storage is not available.
>>>>>>>>
>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>
>>>>>>>>> Well, make it not go through host1 and dedicate a
storage server
>>>>>>>>> for running NFS and make both hosts connect to it.
>>>>>>>>> In my view NFS is much easier to manage than any
other type of
>>>>>>>>> storage, specially FC and iSCSI and performance is
pretty much the same, so
>>>>>>>>> you won`t get better results other than management
going to other type.
>>>>>>>>>
>>>>>>>>> Fernando
>>>>>>>>>
>>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <
>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>
>>>>>>>>>> Hi guys,
>>>>>>>>>> I have one nfs storage,
>>>>>>>>>> it's connected through host1.
>>>>>>>>>> host2 also has access to it, I can easily migrate
vms between
>>>>>>>>>> them.
>>>>>>>>>>
>>>>>>>>>> The question is - if host1 is down - all
infrastructure is down,
>>>>>>>>>> since all traffic goes through host1,
>>>>>>>>>> is there any way in oVirt to use redundant
storage?
>>>>>>>>>>
>>>>>>>>>> Only glusterfs?
>>>>>>>>>>
>>>>>>>>>> Thanks
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users(a)ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>>
6:18 a.m.
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Fence agent under each node?
When you configure a host, there's the power management tab, where you need
to enter the bmc details for the host. If you don't have fencing enabled,
how do you expect the system to make sure a host running a service is
actually down (and it is safe to start HA services elsewhere), and not, for
example, just unreachable by the engine? How do you avoid a splitbraid ->
SBA ?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> "Corner cases"?
>> I tried to simulate crash of SPM server and ovirt kept trying to
>> reistablished connection to the failed node.
>>
>
> Did you configure fencing?
>
>
>>
>>
>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com>
wrote:
>>>
>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com>
wrote:
>>>>
>>>>>
>>>>>
>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>>
>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>>
>>>>>> When you set up a storage domain, you need to specify a host to
>>>>>> perform the initial storage operations, but once the SD is
defined, it's
>>>>>> details are in the engine database, and all the hosts get
connected to it
>>>>>> directly. If the first host you used to define the SD goes down,
all other
>>>>>> hosts will still remain connected and work. SPM is an HA service,
and if
>>>>>> the current SPM host goes down, SPM gets started on another host
in the DC.
>>>>>> In short, unless your actual NFS exporting host goes down, there
is no
>>>>>> outage.
>>>>>>
>>>>>
>>>>> There is no storage outage, but if you shutdown the spm host, the
spm
>>>>> host
>>>>> will not move to a new host until the spm host is online again, or
>>>>> you confirm
>>>>> manually that the spm host was rebooted.
>>>>>
>>>>>
>>>>> In a properly configured setup the SBA should take care of that.
>>>>> That's the whole point of HA services
>>>>>
>>>>
>>>> In some cases like power loss or hardware failure, there is no way to
>>>> start
>>>> the spm host, and the system cannot recover automatically.
>>>>
>>>
>>> There are always corner cases, no doubt. But in a normal situation.
>>> where an SPM host goes down because of a hardware failure, it gets fenced,
>>> other hosts contend for SPM and start it. No surprises there.
>>>
>>>
>>>>
>>>> Nir
>>>>
>>>>
>>>>>
>>>>>
>>>>> Nir
>>>>>
>>>>>
>>>>>>
>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>
>>>>>>> Hi Fernando,
>>>>>>> I see each host has direct connection nfs mount, but yes, if
main
>>>>>>> host to which I connected nfs storage going down the storage
becomes
>>>>>>> unavailable and all vms are down
>>>>>>>
>>>>>>>
>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>
>>>>>>>> Hello Konstantin.
>>>>>>>>
>>>>>>>> That doesn`t make much sense make a whole cluster depend
on a
>>>>>>>> single host. From what I know any host talk directly to
NFS Storage Array
>>>>>>>> or whatever other Shared Storage you have.
>>>>>>>> Have you tested that host going down if that affects the
other
>>>>>>>> with the NFS mounted directlly in a NFS Storage array ?
>>>>>>>>
>>>>>>>> Fernando
>>>>>>>>
>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <
>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>
>>>>>>>>> In ovirt you have to attach storage through specific
host.
>>>>>>>>> If host goes down storage is not available.
>>>>>>>>>
>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI
<
>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>
>>>>>>>>>> Well, make it not go through host1 and dedicate a
storage server
>>>>>>>>>> for running NFS and make both hosts connect to
it.
>>>>>>>>>> In my view NFS is much easier to manage than any
other type of
>>>>>>>>>> storage, specially FC and iSCSI and performance
is pretty much the same, so
>>>>>>>>>> you won`t get better results other than
management going to other type.
>>>>>>>>>>
>>>>>>>>>> Fernando
>>>>>>>>>>
>>>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<
>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> Hi guys,
>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>> it's connected through host1.
>>>>>>>>>>> host2 also has access to it, I can easily
migrate vms between
>>>>>>>>>>> them.
>>>>>>>>>>>
>>>>>>>>>>> The question is - if host1 is down - all
infrastructure is
>>>>>>>>>>> down, since all traffic goes through host1,
>>>>>>>>>>> is there any way in oVirt to use redundant
storage?
>>>>>>>>>>>
>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>
>>>>>>>>>>> Thanks
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>
>>>>> _______________________________________________
>>>>> Users mailing list
>>>>> Users(a)ovirt.org
>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>
>>>>>
>>>>>
6:19 a.m.
Makes sense.
I was trying to set it up, but doesn't work with our staging hardware.
We have old ilo100, I'll try again.
Thanks!
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
> Fence agent under each node?
>
When you configure a host, there's the power management tab, where you
need to enter the bmc details for the host. If you don't have fencing
enabled, how do you expect the system to make sure a host running a service
is actually down (and it is safe to start HA services elsewhere), and not,
for example, just unreachable by the engine? How do you avoid a splitbraid
-> SBA ?
>
> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> "Corner cases"?
>>> I tried to simulate crash of SPM server and ovirt kept trying to
>>> reistablished connection to the failed node.
>>>
>>
>> Did you configure fencing?
>>
>>
>>>
>>>
>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>>
>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com>
>>>> wrote:
>>>>
>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com>
wrote:
>>>>>
>>>>>>
>>>>>>
>>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>>>
>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>
>>>>>>> When you set up a storage domain, you need to specify a host
to
>>>>>>> perform the initial storage operations, but once the SD is
defined, it's
>>>>>>> details are in the engine database, and all the hosts get
connected to it
>>>>>>> directly. If the first host you used to define the SD goes
down, all other
>>>>>>> hosts will still remain connected and work. SPM is an HA
service, and if
>>>>>>> the current SPM host goes down, SPM gets started on another
host in the DC.
>>>>>>> In short, unless your actual NFS exporting host goes down,
there is no
>>>>>>> outage.
>>>>>>>
>>>>>>
>>>>>> There is no storage outage, but if you shutdown the spm host,
the
>>>>>> spm host
>>>>>> will not move to a new host until the spm host is online again,
or
>>>>>> you confirm
>>>>>> manually that the spm host was rebooted.
>>>>>>
>>>>>>
>>>>>> In a properly configured setup the SBA should take care of that.
>>>>>> That's the whole point of HA services
>>>>>>
>>>>>
>>>>> In some cases like power loss or hardware failure, there is no way
to
>>>>> start
>>>>> the spm host, and the system cannot recover automatically.
>>>>>
>>>>
>>>> There are always corner cases, no doubt. But in a normal situation.
>>>> where an SPM host goes down because of a hardware failure, it gets
fenced,
>>>> other hosts contend for SPM and start it. No surprises there.
>>>>
>>>>
>>>>>
>>>>> Nir
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> Nir
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> Hi Fernando,
>>>>>>>> I see each host has direct connection nfs mount, but yes,
if main
>>>>>>>> host to which I connected nfs storage going down the
storage becomes
>>>>>>>> unavailable and all vms are down
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>
>>>>>>>>> Hello Konstantin.
>>>>>>>>>
>>>>>>>>> That doesn`t make much sense make a whole cluster
depend on a
>>>>>>>>> single host. From what I know any host talk directly
to NFS Storage Array
>>>>>>>>> or whatever other Shared Storage you have.
>>>>>>>>> Have you tested that host going down if that affects
the other
>>>>>>>>> with the NFS mounted directlly in a NFS Storage array
?
>>>>>>>>>
>>>>>>>>> Fernando
>>>>>>>>>
>>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<
>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>
>>>>>>>>>> In ovirt you have to attach storage through
specific host.
>>>>>>>>>> If host goes down storage is not available.
>>>>>>>>>>
>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI
<
>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Well, make it not go through host1 and
dedicate a storage
>>>>>>>>>>> server for running NFS and make both hosts
connect to it.
>>>>>>>>>>> In my view NFS is much easier to manage than
any other type of
>>>>>>>>>>> storage, specially FC and iSCSI and
performance is pretty much the same, so
>>>>>>>>>>> you won`t get better results other than
management going to other type.
>>>>>>>>>>>
>>>>>>>>>>> Fernando
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>>> it's connected through host1.
>>>>>>>>>>>> host2 also has access to it, I can easily
migrate vms between
>>>>>>>>>>>> them.
>>>>>>>>>>>>
>>>>>>>>>>>> The question is - if host1 is down - all
infrastructure is
>>>>>>>>>>>> down, since all traffic goes through
host1,
>>>>>>>>>>>> is there any way in oVirt to use
redundant storage?
>>>>>>>>>>>>
>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>
>>>>>>>>>>>> Thanks
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users(a)ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>>
>>>>>>
6:21 a.m.
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Makes sense.
I was trying to set it up, but doesn't work with our staging hardware.
We have old ilo100, I'll try again.
Thanks!
It is absolutely necessary for any HA to work properly. There's of course
the "confirm host has been shutdown" option, which serves as an override
for the fence command, but it's manual
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Fence agent under each node?
>>
>
> When you configure a host, there's the power management tab, where you
> need to enter the bmc details for the host. If you don't have fencing
> enabled, how do you expect the system to make sure a host running a service
> is actually down (and it is safe to start HA services elsewhere), and not,
> for example, just unreachable by the engine? How do you avoid a splitbraid
> -> SBA ?
>
>
>>
>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>> konrasko(a)gmail.com> wrote:
>>>
>>>> "Corner cases"?
>>>> I tried to simulate crash of SPM server and ovirt kept trying to
>>>> reistablished connection to the failed node.
>>>>
>>>
>>> Did you configure fencing?
>>>
>>>
>>>>
>>>>
>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>
>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>> wrote:
>>>>>
>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com> wrote:
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> When you set up a storage domain, you need to specify a
host to
>>>>>>>> perform the initial storage operations, but once the SD
is defined, it's
>>>>>>>> details are in the engine database, and all the hosts get
connected to it
>>>>>>>> directly. If the first host you used to define the SD
goes down, all other
>>>>>>>> hosts will still remain connected and work. SPM is an HA
service, and if
>>>>>>>> the current SPM host goes down, SPM gets started on
another host in the DC.
>>>>>>>> In short, unless your actual NFS exporting host goes
down, there is no
>>>>>>>> outage.
>>>>>>>>
>>>>>>>
>>>>>>> There is no storage outage, but if you shutdown the spm host,
the
>>>>>>> spm host
>>>>>>> will not move to a new host until the spm host is online
again, or
>>>>>>> you confirm
>>>>>>> manually that the spm host was rebooted.
>>>>>>>
>>>>>>>
>>>>>>> In a properly configured setup the SBA should take care of
that.
>>>>>>> That's the whole point of HA services
>>>>>>>
>>>>>>
>>>>>> In some cases like power loss or hardware failure, there is no
way
>>>>>> to start
>>>>>> the spm host, and the system cannot recover automatically.
>>>>>>
>>>>>
>>>>> There are always corner cases, no doubt. But in a normal situation.
>>>>> where an SPM host goes down because of a hardware failure, it gets
fenced,
>>>>> other hosts contend for SPM and start it. No surprises there.
>>>>>
>>>>>
>>>>>>
>>>>>> Nir
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> Nir
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi
<
>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>
>>>>>>>>> Hi Fernando,
>>>>>>>>> I see each host has direct connection nfs mount, but
yes, if main
>>>>>>>>> host to which I connected nfs storage going down the
storage becomes
>>>>>>>>> unavailable and all vms are down
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI
<
>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>
>>>>>>>>>> That doesn`t make much sense make a whole cluster
depend on a
>>>>>>>>>> single host. From what I know any host talk
directly to NFS Storage Array
>>>>>>>>>> or whatever other Shared Storage you have.
>>>>>>>>>> Have you tested that host going down if that
affects the other
>>>>>>>>>> with the NFS mounted directlly in a NFS Storage
array ?
>>>>>>>>>>
>>>>>>>>>> Fernando
>>>>>>>>>>
>>>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<
>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>
>>>>>>>>>>> In ovirt you have to attach storage through
specific host.
>>>>>>>>>>> If host goes down storage is not available.
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO
FREDIANI <
>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Well, make it not go through host1 and
dedicate a storage
>>>>>>>>>>>> server for running NFS and make both
hosts connect to it.
>>>>>>>>>>>> In my view NFS is much easier to manage
than any other type of
>>>>>>>>>>>> storage, specially FC and iSCSI and
performance is pretty much the same, so
>>>>>>>>>>>> you won`t get better results other than
management going to other type.
>>>>>>>>>>>>
>>>>>>>>>>>> Fernando
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>>>> it's connected through host1.
>>>>>>>>>>>>> host2 also has access to it, I can
easily migrate vms between
>>>>>>>>>>>>> them.
>>>>>>>>>>>>>
>>>>>>>>>>>>> The question is - if host1 is down -
all infrastructure is
>>>>>>>>>>>>> down, since all traffic goes through
host1,
>>>>>>>>>>>>> is there any way in oVirt to use
redundant storage?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users(a)ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users(a)ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>>>>
6:29 a.m.
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
> Makes sense.
> I was trying to set it up, but doesn't work with our staging hardware.
> We have old ilo100, I'll try again.
> Thanks!
>
>
It is absolutely necessary for any HA to work properly. There's of course
the "confirm host has been shutdown" option, which serves as an override
for the fence command, but it's manual
> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> Fence agent under each node?
>>>
>>
>> When you configure a host, there's the power management tab, where you
>> need to enter the bmc details for the host. If you don't have fencing
>> enabled, how do you expect the system to make sure a host running a service
>> is actually down (and it is safe to start HA services elsewhere), and not,
>> for example, just unreachable by the engine? How do you avoid a splitbraid
>> -> SBA ?
>>
>>
>>>
>>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>>
>>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>>> konrasko(a)gmail.com> wrote:
>>>>
>>>>> "Corner cases"?
>>>>> I tried to simulate crash of SPM server and ovirt kept trying to
>>>>> reistablished connection to the failed node.
>>>>>
>>>>
>>>> Did you configure fencing?
>>>>
>>>>
>>>>>
>>>>>
>>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>>
>>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>>> wrote:
>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>>>>>
>>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> When you set up a storage domain, you need to specify
a host to
>>>>>>>>> perform the initial storage operations, but once the
SD is defined, it's
>>>>>>>>> details are in the engine database, and all the hosts
get connected to it
>>>>>>>>> directly. If the first host you used to define the SD
goes down, all other
>>>>>>>>> hosts will still remain connected and work. SPM is an
HA service, and if
>>>>>>>>> the current SPM host goes down, SPM gets started on
another host in the DC.
>>>>>>>>> In short, unless your actual NFS exporting host goes
down, there is no
>>>>>>>>> outage.
>>>>>>>>>
>>>>>>>>
>>>>>>>> There is no storage outage, but if you shutdown the spm
host, the
>>>>>>>> spm host
>>>>>>>> will not move to a new host until the spm host is online
again, or
>>>>>>>> you confirm
>>>>>>>> manually that the spm host was rebooted.
>>>>>>>>
>>>>>>>>
>>>>>>>> In a properly configured setup the SBA should take care
of that.
>>>>>>>> That's the whole point of HA services
>>>>>>>>
>>>>>>>
>>>>>>> In some cases like power loss or hardware failure, there is
no way
>>>>>>> to start
>>>>>>> the spm host, and the system cannot recover automatically.
>>>>>>>
>>>>>>
>>>>>> There are always corner cases, no doubt. But in a normal
situation.
>>>>>> where an SPM host goes down because of a hardware failure, it
gets fenced,
>>>>>> other hosts contend for SPM and start it. No surprises there.
>>>>>>
>>>>>>
>>>>>>>
>>>>>>> Nir
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> Nir
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin
Raskoshnyi <
>>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>>
>>>>>>>>>> Hi Fernando,
>>>>>>>>>> I see each host has direct connection nfs mount,
but yes, if
>>>>>>>>>> main host to which I connected nfs storage going
down the storage becomes
>>>>>>>>>> unavailable and all vms are down
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO
FREDIANI <
>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>>
>>>>>>>>>>> That doesn`t make much sense make a whole
cluster depend on a
>>>>>>>>>>> single host. From what I know any host talk
directly to NFS Storage Array
>>>>>>>>>>> or whatever other Shared Storage you have.
>>>>>>>>>>> Have you tested that host going down if that
affects the other
>>>>>>>>>>> with the NFS mounted directlly in a NFS
Storage array ?
>>>>>>>>>>>
>>>>>>>>>>> Fernando
>>>>>>>>>>>
>>>>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>
>>>>>>>>>>>> In ovirt you have to attach storage
through specific host.
>>>>>>>>>>>> If host goes down storage is not
available.
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO
FREDIANI <
>>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Well, make it not go through host1
and dedicate a storage
>>>>>>>>>>>>> server for running NFS and make both
hosts connect to it.
>>>>>>>>>>>>> In my view NFS is much easier to
manage than any other type
>>>>>>>>>>>>> of storage, specially FC and iSCSI
and performance is pretty much the same,
>>>>>>>>>>>>> so you won`t get better results other
than management going to other type.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>>>>> it's connected through
host1.
>>>>>>>>>>>>>> host2 also has access to it, I
can easily migrate
>>>>>>>>>>>>>> vms between them.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The question is - if host1 is
down - all infrastructure is
>>>>>>>>>>>>>> down, since all traffic goes
through host1,
>>>>>>>>>>>>>> is there any way in oVirt to use
redundant storage?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users(a)ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>
>>>>>>>> _______________________________________________
>>>>>>>> Users mailing list
>>>>>>>> Users(a)ovirt.org
>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>
>>>>>>>>
>>>>>>>>
8:24 a.m.
But actually, it didn't work well. After main SPM host went down I see this
2017-04-17 05:23:15,554Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init:
could not find reported vds or not up - pool: 'STG' vds_spm_id: '1'
2017-04-17 05:23:15,567Z INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is
non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically switch SPM
host?
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Makes sense.
>> I was trying to set it up, but doesn't work with our staging hardware.
>> We have old ilo100, I'll try again.
>> Thanks!
>>
>>
> It is absolutely necessary for any HA to work properly. There's of course
> the "confirm host has been shutdown" option, which serves as an override
> for the fence command, but it's manual
>
>
>> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>>> konrasko(a)gmail.com> wrote:
>>>
>>>> Fence agent under each node?
>>>>
>>>
>>> When you configure a host, there's the power management tab, where you
>>> need to enter the bmc details for the host. If you don't have fencing
>>> enabled, how do you expect the system to make sure a host running a service
>>> is actually down (and it is safe to start HA services elsewhere), and not,
>>> for example, just unreachable by the engine? How do you avoid a splitbraid
>>> -> SBA ?
>>>
>>>
>>>>
>>>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>
>>>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>>>> konrasko(a)gmail.com> wrote:
>>>>>
>>>>>> "Corner cases"?
>>>>>> I tried to simulate crash of SPM server and ovirt kept trying to
>>>>>> reistablished connection to the failed node.
>>>>>>
>>>>>
>>>>> Did you configure fencing?
>>>>>
>>>>>
>>>>>>
>>>>>>
>>>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>>>> wrote:
>>>>>>>
>>>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com> wrote:
>>>>>>>>>
>>>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> When you set up a storage domain, you need to
specify a host to
>>>>>>>>>> perform the initial storage operations, but once
the SD is defined, it's
>>>>>>>>>> details are in the engine database, and all the
hosts get connected to it
>>>>>>>>>> directly. If the first host you used to define
the SD goes down, all other
>>>>>>>>>> hosts will still remain connected and work. SPM
is an HA service, and if
>>>>>>>>>> the current SPM host goes down, SPM gets started
on another host in the DC.
>>>>>>>>>> In short, unless your actual NFS exporting host
goes down, there is no
>>>>>>>>>> outage.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> There is no storage outage, but if you shutdown the
spm host, the
>>>>>>>>> spm host
>>>>>>>>> will not move to a new host until the spm host is
online again,
>>>>>>>>> or you confirm
>>>>>>>>> manually that the spm host was rebooted.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In a properly configured setup the SBA should take
care of that.
>>>>>>>>> That's the whole point of HA services
>>>>>>>>>
>>>>>>>>
>>>>>>>> In some cases like power loss or hardware failure, there
is no way
>>>>>>>> to start
>>>>>>>> the spm host, and the system cannot recover
automatically.
>>>>>>>>
>>>>>>>
>>>>>>> There are always corner cases, no doubt. But in a normal
situation.
>>>>>>> where an SPM host goes down because of a hardware failure, it
gets fenced,
>>>>>>> other hosts contend for SPM and start it. No surprises
there.
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>> Nir
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nir
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin
Raskoshnyi <
>>>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>>>
>>>>>>>>>>> Hi Fernando,
>>>>>>>>>>> I see each host has direct connection nfs
mount, but yes, if
>>>>>>>>>>> main host to which I connected nfs storage
going down the storage becomes
>>>>>>>>>>> unavailable and all vms are down
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO
FREDIANI <
>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>>>
>>>>>>>>>>>> That doesn`t make much sense make a whole
cluster depend on a
>>>>>>>>>>>> single host. From what I know any host
talk directly to NFS Storage Array
>>>>>>>>>>>> or whatever other Shared Storage you
have.
>>>>>>>>>>>> Have you tested that host going down if
that affects the other
>>>>>>>>>>>> with the NFS mounted directlly in a NFS
Storage array ?
>>>>>>>>>>>>
>>>>>>>>>>>> Fernando
>>>>>>>>>>>>
>>>>>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>
>>>>>>>>>>>>> In ovirt you have to attach storage
through specific host.
>>>>>>>>>>>>> If host goes down storage is not
available.
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM
FERNANDO FREDIANI <
>>>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Well, make it not go through
host1 and dedicate a storage
>>>>>>>>>>>>>> server for running NFS and make
both hosts connect to it.
>>>>>>>>>>>>>> In my view NFS is much easier to
manage than any other type
>>>>>>>>>>>>>> of storage, specially FC and
iSCSI and performance is pretty much the same,
>>>>>>>>>>>>>> so you won`t get better results
other than management going to other type.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>>>>>> it's connected through
host1.
>>>>>>>>>>>>>>> host2 also has access to it,
I can easily migrate
>>>>>>>>>>>>>>> vms between them.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> The question is - if host1 is
down - all infrastructure is
>>>>>>>>>>>>>>> down, since all traffic goes
through host1,
>>>>>>>>>>>>>>> is there any way in oVirt to
use redundant storage?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> _______________________________________________
>>>>>>>>> Users mailing list
>>>>>>>>> Users(a)ovirt.org
>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
4:06 p.m.
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
But actually, it didn't work well. After main SPM host went down
I see this
[image: Screen Shot 2017-04-16 at 10.22.00 PM.png]
2017-04-17 05:23:15,554Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init:
could not find reported vds or not up - pool: 'STG' vds_spm_id: '1'
2017-04-17 05:23:15,567Z INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is
non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically switch SPM
host?
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com
> wrote:
> Oh, fence agent works fine if I select ilo4,
> Thank you for your help!
>
> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> Makes sense.
>>> I was trying to set it up, but doesn't work with our staging hardware.
>>> We have old ilo100, I'll try again.
>>> Thanks!
>>>
>>>
>> It is absolutely necessary for any HA to work properly. There's of
>> course the "confirm host has been shutdown" option, which serves as an
>> override for the fence command, but it's manual
>>
>>
>>> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>>
>>>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>>>> konrasko(a)gmail.com> wrote:
>>>>
>>>>> Fence agent under each node?
>>>>>
>>>>
>>>> When you configure a host, there's the power management tab, where
you
>>>> need to enter the bmc details for the host. If you don't have
fencing
>>>> enabled, how do you expect the system to make sure a host running a
service
>>>> is actually down (and it is safe to start HA services elsewhere), and
not,
>>>> for example, just unreachable by the engine? How do you avoid a
splitbraid
>>>> -> SBA ?
>>>>
>>>>
>>>>>
>>>>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>>
>>>>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>
>>>>>>> "Corner cases"?
>>>>>>> I tried to simulate crash of SPM server and ovirt kept trying
to
>>>>>>> reistablished connection to the failed node.
>>>>>>>
>>>>>>
>>>>>> Did you configure fencing?
>>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer(a)redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>> When you set up a storage domain, you need to
specify a host to
>>>>>>>>>>> perform the initial storage operations, but
once the SD is defined, it's
>>>>>>>>>>> details are in the engine database, and all
the hosts get connected to it
>>>>>>>>>>> directly. If the first host you used to
define the SD goes down, all other
>>>>>>>>>>> hosts will still remain connected and work.
SPM is an HA service, and if
>>>>>>>>>>> the current SPM host goes down, SPM gets
started on another host in the DC.
>>>>>>>>>>> In short, unless your actual NFS exporting
host goes down, there is no
>>>>>>>>>>> outage.
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> There is no storage outage, but if you shutdown
the spm host,
>>>>>>>>>> the spm host
>>>>>>>>>> will not move to a new host until the spm host is
online again,
>>>>>>>>>> or you confirm
>>>>>>>>>> manually that the spm host was rebooted.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> In a properly configured setup the SBA should
take care of that.
>>>>>>>>>> That's the whole point of HA services
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> In some cases like power loss or hardware failure,
there is no
>>>>>>>>> way to start
>>>>>>>>> the spm host, and the system cannot recover
automatically.
>>>>>>>>>
>>>>>>>>
>>>>>>>> There are always corner cases, no doubt. But in a normal
>>>>>>>> situation. where an SPM host goes down because of a
hardware failure, it
>>>>>>>> gets fenced, other hosts contend for SPM and start it. No
surprises there.
>>>>>>>>
>>>>>>>>
>>>>>>>>>
>>>>>>>>> Nir
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Nir
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin
Raskoshnyi <
>>>>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi Fernando,
>>>>>>>>>>>> I see each host has direct connection nfs
mount, but yes, if
>>>>>>>>>>>> main host to which I connected nfs
storage going down the storage becomes
>>>>>>>>>>>> unavailable and all vms are down
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO
FREDIANI <
>>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>>>>
>>>>>>>>>>>>> That doesn`t make much sense make a
whole cluster depend on a
>>>>>>>>>>>>> single host. From what I know any
host talk directly to NFS Storage Array
>>>>>>>>>>>>> or whatever other Shared Storage you
have.
>>>>>>>>>>>>> Have you tested that host going down
if that affects the
>>>>>>>>>>>>> other with the NFS mounted directlly
in a NFS Storage array ?
>>>>>>>>>>>>>
>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>
>>>>>>>>>>>>> 2017-04-15 12:42 GMT-03:00 Konstantin
Raskoshnyi <
>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> In ovirt you have to attach
storage through specific host.
>>>>>>>>>>>>>> If host goes down storage is not
available.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31 AM
FERNANDO FREDIANI <
>>>>>>>>>>>>>> fernando.frediani(a)upx.com>
wrote:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Well, make it not go through
host1 and dedicate a storage
>>>>>>>>>>>>>>> server for running NFS and
make both hosts connect to it.
>>>>>>>>>>>>>>> In my view NFS is much easier
to manage than any other type
>>>>>>>>>>>>>>> of storage, specially FC and
iSCSI and performance is pretty much the same,
>>>>>>>>>>>>>>> so you won`t get better
results other than management going to other type.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>>> I have one nfs storage,
>>>>>>>>>>>>>>>> it's connected
through host1.
>>>>>>>>>>>>>>>> host2 also has access to
it, I can easily migrate
>>>>>>>>>>>>>>>> vms between them.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> The question is - if
host1 is down - all infrastructure is
>>>>>>>>>>>>>>>> down, since all traffic
goes through host1,
>>>>>>>>>>>>>>>> is there any way in oVirt
to use redundant storage?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> _______________________________________________
>>>>>>>>>> Users mailing list
>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>
4:12 p.m.
This is a multi-part message in MIME format.
--------------5323FA587AD2C243D1E46CA3
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Should I understand then that fecing is not mandatory, just advised and
there are no downtimes to the rest of the cluster if a pending manual
action has to be done to confirm a host has been rebooted in order to
move the SPM to another live host. Perhaps only effect is not be able to
add new hosts and connect to storage or something ?
Fernando
On 17/04/2017 10:06, Nir Soffer wrote:
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi
<konrasko(a)gmail.com <mailto:konrasko@gmail.com>> wrote:
But actually, it didn't work well. After main SPM host went down I
see this
Screen Shot 2017-04-16 at 10.22.00 PM.png
2017-04-17 05:23:15,554Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1]
SPM Init: could not find reported vds or not up - pool: 'STG'
vds_spm_id: '1'
2017-04-17 05:23:15,567Z INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1]
SPM selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1]
spm vds is non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically
switch SPM host?
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi
<konrasko(a)gmail.com <mailto:konrasko@gmail.com>> wrote:
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com
<mailto:dyasny@gmail.com>> wrote:
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi
<konrasko(a)gmail.com <mailto:konrasko@gmail.com>> wrote:
Makes sense.
I was trying to set it up, but doesn't work with our
staging hardware.
We have old ilo100, I'll try again.
Thanks!
It is absolutely necessary for any HA to work properly.
There's of course the "confirm host has been shutdown"
option, which serves as an override for the fence command,
but it's manual
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny
<dyasny(a)gmail.com <mailto:dyasny@gmail.com>> wrote:
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin
Raskoshnyi <konrasko(a)gmail.com
<mailto:konrasko@gmail.com>> wrote:
Fence agent under each node?
When you configure a host, there's the power
management tab, where you need to enter the bmc
details for the host. If you don't have fencing
enabled, how do you expect the system to make sure
a host running a service is actually down (and it
is safe to start HA services elsewhere), and not,
for example, just unreachable by the engine? How
do you avoid a splitbraid -> SBA ?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny
<dyasny(a)gmail.com <mailto:dyasny@gmail.com>>
wrote:
On Sun, Apr 16, 2017 at 11:13 PM,
Konstantin Raskoshnyi <konrasko(a)gmail.com
<mailto:konrasko@gmail.com>> wrote:
"Corner cases"?
I tried to simulate crash of SPM
server and ovirt kept trying to
reistablished connection to the failed
node.
Did you configure fencing?
On Sun, Apr 16, 2017 at 8:10 PM Dan
Yasny <dyasny(a)gmail.com
<mailto:dyasny@gmail.com>> wrote:
On Sun, Apr 16, 2017 at 7:29 AM,
Nir Soffer <nsoffer(a)redhat.com
<mailto:nsoffer@redhat.com>> wrote:
On Sun, Apr 16, 2017 at 2:05
PM Dan Yasny
<dyasny(a)redhat.com
<mailto:dyasny@redhat.com>> wrote:
On Apr 16, 2017 7:01 AM,
"Nir Soffer"
<nsoffer(a)redhat.com
<mailto:nsoffer@redhat.com>>
wrote:
On Sun, Apr 16, 2017
at 4:17 AM Dan Yasny
<dyasny(a)gmail.com
<mailto:dyasny@gmail.com>>
wrote:
When you set up a
storage domain,
you need to
specify a host to
perform the
initial storage
operations, but
once the SD is
defined, it's
details are in the
engine database,
and all the hosts
get connected to
it directly. If
the first host you
used to define the
SD goes down, all
other hosts will
still remain
connected and
work. SPM is an HA
service, and if
the current SPM
host goes down,
SPM gets started
on another host in
the DC. In short,
unless your actual
NFS exporting host
goes down, there
is no outage.
There is no storage
outage, but if you
shutdown the spm host,
the spm host
will not move to a new
host until the spm
host is online again,
or you confirm
manually that the spm
host was rebooted.
In a properly configured
setup the SBA should take
care of that. That's the
whole point of HA services
In some cases like power loss
or hardware failure, there is
no way to start
the spm host, and the system
cannot recover automatically.
There are always corner cases, no
doubt. But in a normal situation.
where an SPM host goes down
because of a hardware failure, it
gets fenced, other hosts contend
for SPM and start it. No surprises
there.
Nir
Nir
On Sat, Apr 15,
2017 at 1:53 PM,
Konstantin
Raskoshnyi
<konrasko(a)gmail.com
<mailto:konrasko@gmail.com>>
wrote:
Hi Fernando,
I see each
host has
direct
connection nfs
mount, but
yes, if main
host to which
I connected
nfs storage
going down the
storage
becomes
unavailable
and all vms
are down
On Sat, Apr
15, 2017 at
10:37 AM
FERNANDO
FREDIANI
<fernando.frediani(a)upx.com
<mailto:fernando.frediani@upx.com>>
wrote:
Hello
Konstantin.
That
doesn`t
make much
sense make
a whole
cluster
depend on
a single
host. From
what I
know any
host talk
directly
to NFS
Storage
Array or
whatever
other
Shared
Storage
you have.
Have you
tested
that host
going down
if that
affects
the other
with the
NFS
mounted
directlly
in a NFS
Storage
array ?
Fernando
2017-04-15
12:42
GMT-03:00
Konstantin
Raskoshnyi
<konrasko(a)gmail.com
<mailto:konrasko@gmail.com>>:
In
ovirt
you
have
to
attach
storage
through
specific
host.
If
host
goes
down
storage
is not
available.
On
Sat,
Apr
15,
2017
at
7:31
AM
FERNANDO
FREDIANI
<fernando.frediani(a)upx.com
<mailto:fernando.frediani@upx.com>>
wrote:
Well,
make
it
not
go
through
host1
and
dedicate
a
storage
server
for
running
NFS
and
make
both
hosts
connect
to it.
In
my
view
NFS
is
much
easier
to
manage
than
any
other
type
of
storage,
specially
FC
and
iSCSI
and
performance
is
pretty
much
the
same,
so
you
won`t
get
better
results
other
than
management
going
to
other
type.
Fernando
2017-04-15
5:25
GMT-03:00
Konstantin
Raskoshnyi
<konrasko(a)gmail.com
<mailto:konrasko@gmail.com>>:
Hi
guys,
I
have
one
nfs storage,
it's
connected
through
host1.
host2
also
has
access
to
it,
I
can
easily
migrate
vms between
them.
The
question
is
-
if
host1
is
down
-
all
infrastructure
is
down,
since
all
traffic
goes
through
host1,
is
there
any
way
in
oVirt
to
use
redundant
storage?
Only
glusterfs?
Thanks
_______________________________________________
Users
mailing
list
Users(a)ovirt.org
<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
<mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
--------------5323FA587AD2C243D1E46CA3
Content-Type: multipart/related;
boundary="------------E141655F6BC3F6F8ABEBA096"
--------------E141655F6BC3F6F8ABEBA096
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Should I understand then that fecing is not mandatory, just
advised and there are no downtimes to the rest of the cluster if a
pending manual action has to be done to confirm a host has been
rebooted in order to move the SPM to another live host. Perhaps
only effect is not be able to add new hosts and connect to storage
or something ?</p>
<p>Fernando<br>
</p>
<div class="moz-cite-prefix">On 17/04/2017 10:06, Nir Soffer
wrote:<br>
</div>
<blockquote
cite="mid:CAMRbyyuHYKMZ_KuiS_DfYWC3na5PSUvV7iumm2c-FLFKmnnsEg@mail.gmail.com"
type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Mon, Apr 17, 2017 at 8:24 AM Konstantin
Raskoshnyi <<a moz-do-not-send="true"
href="mailto:konrasko@gmail.com">konrasko@gmail.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">But actually, it didn't work well. After
main
SPM host went down I see this
<div><img src="cid:part2.ECBD2DE4.FFC0D107@upx.com"
alt="Screen Shot 2017-04-16 at 10.22.00 PM.png"
class="" style="max-width: 100%;"></div>
<div><br>
2017-04-17 05:23:15,554Z ERROR
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init: could
not find reported vds or not up - pool: 'STG'
vds_spm_id: '1'<br>
</div>
<div>2017-04-17 05:23:15,567Z INFO
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM selection -
vds seems as spm 'tank5'</div>
<div>2017-04-17 05:23:15,567Z WARN
[org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is non
responsive, stopping spm selection.</div>
<div><br>
</div>
<div>So that means only if BMC is up it's possible to
automatically switch SPM host?</div>
</div>
</blockquote>
<div><br>
</div>
<div>BMC?</div>
<div><br>
</div>
<div>If your SPM is no responsive, the system will try to
fence it. Did you </div>
<div>configure power management for all hosts? did you check
that it </div>
<div>work? How did you simulate non-responsive host? </div>
<div><br>
</div>
<div>If power management is not configured or fail, the system
cannot</div>
<div>move the spm to another host, unless you manually confirm
that the</div>
<div>SPM host was rebooted.</div>
<div><br>
</div>
<div>Nir</div>
<div> </div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
<div>Thanks</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Apr 16, 2017 at 8:29 PM,
Konstantin Raskoshnyi <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Oh, fence agent works fine if I select ilo4,</div>
<div>Thank you for your help!</div>
<div class="m_-7049131619673457077HOEnZb">
<div class="m_-7049131619673457077h5">
<div><br>
<div class="gmail_quote">
<div>On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny
<<a moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div>
<div class="gmail_extra">
<div class="gmail_quote">On Sun, Apr 16,
2017 at 11:19 PM, Konstantin
Raskoshnyi <span><<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>Makes sense. </div>
<div>I was trying to set it up, but
doesn't work with our staging
hardware. </div>
<div>We have old ilo100, I'll try
again. </div>
<div>Thanks!
<div>
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886h5"><br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>It is absolutely necessary for
any HA to work properly. There's of
course the "confirm host has been
shutdown" option, which serves as an
override for the fence command, but
it's manual</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div>
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886h5">
<div class="gmail_quote">
<div>On Sun, Apr 16, 2017 at
8:18 PM Dan Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16, 2017 at
11:15 PM, Konstantin
Raskoshnyi <span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>Fence agent
under each node?</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div
class="gmail_quote">
<div>When you
configure a host,
there's the power
management tab,
where you need to
enter the bmc
details for the
host. If you don't
have fencing
enabled, how do
you expect the
system to make
sure a host
running a service
is actually down
(and it is safe to
start HA services
elsewhere), and
not, for example,
just unreachable
by the engine? How
do you avoid a
splitbraid ->
SBA ?</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0
0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743HOEnZb">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743h5">
<div><br>
<div
class="gmail_quote">
<div>On Sun,
Apr 16, 2017
at 8:14 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16,
2017 at 11:13
PM, Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>"Corner
cases"? </div>
<div>I tried
to simulate
crash of SPM
server and
ovirt kept
trying to
reistablished
connection to
the failed
node. </div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>Did you
configure
fencing?</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210HOEnZb">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210h5">
<div><br>
</div>
<div><br>
<div
class="gmail_quote">
<div>On Sun,
Apr 16, 2017
at 8:10 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16,
2017 at 7:29
AM, Nir Soffer
<span><<a
moz-do-not-send="true"
href="mailto:nsoffer@redhat.com"
target="_blank">nsoffer(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><span>
<div>On Sun,
Apr 16, 2017
at 2:05 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@redhat.com"
target="_blank">dyasny(a)redhat.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Apr 16, 2017
7:01 AM, "Nir
Soffer" <<a
moz-do-not-send="true" href="mailto:nsoffer@redhat.com"
target="_blank">nsoffer(a)redhat.com</a>&gt;
wrote:<br
type="attribution">
<blockquote
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quoted-text">
<div>On Sun,
Apr 16, 2017
at 4:17 AM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>When you
set up a
storage
domain, you
need to
specify a host
to perform the
initial
storage
operations,
but once the
SD is defined,
it's details
are in the
engine
database, and
all the hosts
get connected
to it
directly. If
the first host
you used to
define the SD
goes down, all
other hosts
will still
remain
connected and
work. SPM is
an HA service,
and if the
current SPM
host goes
down, SPM gets
started on
another host
in the DC. In
short, unless
your actual
NFS exporting
host goes
down, there is
no outage. </div>
</blockquote>
<div><br>
</div>
</div>
<div>There is
no storage
outage, but if
you shutdown
the spm host,
the spm host</div>
<div>will not
move to a new
host until the
spm host is
online again,
or you confirm</div>
<div>manually
that the spm
host was
rebooted.</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<div><br>
</div>
</div>
<div>
<div>In a
properly
configured
setup the SBA
should take
care of that.
That's the
whole point of
HA services </div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>In some
cases like
power loss or
hardware
failure, there
is no way to
start</div>
<div>the spm
host, and the
system cannot
recover
automatically.</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>There are
always corner
cases, no
doubt. But in
a normal
situation.
where an SPM
host goes down
because of a
hardware
failure, it
gets fenced,
other hosts
contend for
SPM and start
it. No
surprises
there. </div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><span
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191HOEnZb"><font
color="#888888">
<div><br>
</div>
<div>Nir</div>
</font></span>
<div>
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191h5">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div><br>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><font
color="#888888">
<div><br>
</div>
<div>Nir</div>
</font>
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354elided-text">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Sat, Apr 15,
2017 at 1:53
PM, Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hi
Fernando,</div>
<div>I see
each host has
direct
connection nfs
mount, but
yes, if main
host to which
I connected
nfs storage
going down the
storage
becomes
unavailable
and all vms
are down</div>
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354m_6867735599077560094m_2305649857973554660HOEnZb">
<div
class="m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354m_6867735599077560094m_2305649857973554660h5">
<div><br>
</div>
<div><br>
<div
class="gmail_quote">
<div>On Sat,
Apr 15, 2017
at 10:37 AM
FERNANDO
FREDIANI <<a
moz-do-not-send="true" href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div>
<div>Hello
Konstantin.<br>
<br>
</div>
That doesn`t
make much
sense make a
whole cluster
depend on a
single host.
From what I
know any host
talk directly
to NFS Storage
Array or
whatever other
Shared Storage
you have.<br>
</div>
Have you
tested that
host going
down if that
affects the
other with the
NFS mounted
directlly in a
NFS Storage
array ?<br>
<br>
</div>
</div>
<div>Fernando<br>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">2017-04-15
12:42
GMT-03:00
Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>In ovirt
you have to
attach storage
through
specific
host. </div>
<div>If host
goes down
storage is not
available. </div>
<div><br>
<div
class="gmail_quote">
<div>On Sat,
Apr 15, 2017
at 7:31 AM
FERNANDO
FREDIANI <<a
moz-do-not-send="true" href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div>Well,
make it not go
through host1
and dedicate a
storage server
for running
NFS and make
both hosts
connect to it.<br>
</div>
In my view NFS
is much easier
to manage than
any other type
of storage,
specially FC
and iSCSI and
performance is
pretty much
the same, so
you won`t get
better results
other than
management
going to other
type.<br>
<br>
</div>
Fernando<br>
</div>
<div
class="gmail_extra"><br>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">2017-04-15
5:25 GMT-03:00
Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
</div>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hi guys,
<div>I have
one
nfs storage,</div>
<div>it's
connected
through host1.</div>
<div>host2
also has
access to it,
I can easily
migrate
vms between
them.</div>
<div><br>
</div>
<div>The
question is -
if host1 is
down - all
infrastructure
is down, since
all traffic
goes through
host1,</div>
<div>is there
any way in
oVirt to use
redundant
storage?</div>
<div><br>
</div>
<div>Only
glusterfs?</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
</div>
<br>
</blockquote>
</div>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">_______________________________________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
<br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
<br>
</blockquote>
</div>
<br>
</div>
_______________________________________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
</blockquote>
</div>
</div>
</div>
<br>
_______________________________________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>
--------------E141655F6BC3F6F8ABEBA096
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-ID: <part2.ECBD2DE4.FFC0D107(a)upx.com>
iVBORw0KGgoAAAANSUhEUgAAA8wAAACoCAYAAAA1pWe/AAAME2lDQ1BJQ0MgUHJvZmlsZQAA
SImVVwdYU8kWnltSCAktEAEpoTdBepXeO9LBRkgChBIgIajYkUUF14KKBURFV0RUXAsga0HE
ziLY+wMVlZV1sWBD5U0K6Pra9873zZ3/njnnzH/mnjvfDACK1qy8vGxUCYAcfoEgOtCHmZiU
zCT1ARyQAAowoMliC/O8o6LCAJSx/u/y7iZAxP01S3Gsfx3/r6LM4QrZACBREKdyhOwciI8A
gKuz8wQFABA6od5gdkGeGA9BrCqABAEg4mKcLsXqYpwqxZMkNrHRvhB7AUCmsliCdAAUxLyZ
hex0GEdBzNGaz+HxIa6G2IOdweJAfB/iSTk5uRArkiE2Tf0uTvrfYqaOx2Sx0sexNBeJkP14
wrxs1tz/czn+t+Rki8bm0IeNmiEIihbnDNdtT1ZuqBhTIT7OT42IhFgF4gs8jsRejO9miILi
ZPaDbKEvXDPAAPBTc1h+oRBrQcwQZcV5y7AtSyDxhfZoBK8gOFaGUwW50bL4aCE/OyJMFmd5
Bjd4DNdwhf4xYzZpvIBgiGGloUeKMmITpDzRjkJefATEChB3C7NiQmW+D4syfCPGbASiaDFn
Q4jfpgkCoqU2mHqOcCwvzIrNkswFawHzKsiIDZL6YolcYWLYGAcO189fygHjcPlxMm4YrC6f
aJlvaV52lMweq+FmB0ZL1xk7KCyMGfO9WgALTLoO2KNMVkiUbK53eQVRsVJuOArCgC/wA0wg
gi0V5IJMwOsabB6Eb9KRAMACApAOuMBSphnzSJCM8OEzBhSBPyHiAuG4n49klAsKof7LuFb6
tARpktFCiUcWeApxDq6Je+BueBh8esFmizvjLmN+TMWxWYn+RD9iEDGAaDbOgw1ZZ8MmALx/
owuFPRdmJ+bCH8vhWzzCU0IP4RHhBqGXcAfEgyeSKDKrWbxiwQ/MmSAc9MJoAbLsUr/PDjeG
rB1wH9wd8ofccQauCSxxe5iJN+4Jc3OA2u8Zisa5fVvLH+cTs/4+H5lewVzBQcYidfzL+I5b
/RjF97s14sA+9EdLbDl2GDuPncYuYsexZsDETmEtWCd2QozHK+GJpBLGZouWcMuCcXhjNtYN
1gPWn3+YmyWbX7xewgLunALxz+CbmzdXwEvPKGB6w92Yywzms60mMW2tbZwAEO/t0q3jDUOy
ZyOMS990+W0AuJRBZfo3HcsAgGNPAaC/+6YzeA3LfQ0AJ7rZIkGhVCfejgEBUIAi/Cs0gA4w
AKYwH1vgCNyAF/AHISASxIIkMBOueAbIgZxng/lgCSgF5WAN2AC2gG1gJ9gD9oNDoBkcB6fB
OXAZdIMb4B6si37wAgyBd2AEQRASQkPoiAaiixghFogt4ox4IP5IGBKNJCEpSDrCR0TIfGQp
Uo5UIFuQHUg98ityDDmNXER6kDtIHzKAvEY+oRhKRVVRbdQYnYw6o95oKBqLzkDT0Xy0CC1B
V6Gb0Fp0H9qEnkYvozfQXvQFOowBTB5jYHqYJeaM+WKRWDKWhgmwhVgZVonVYgewVvidr2G9
2CD2ESfidJyJW8LaDMLjcDaejy/EV+Jb8D14E96BX8P78CH8K4FG0CJYEFwJwYREQjphNqGU
UEnYTThKOAv/m37COyKRyCCaEJ3gf5lEzCTOI64kbiU2EtuIPcTHxGESiaRBsiC5kyJJLFIB
qZS0mbSPdIp0ldRP+kCWJ+uSbckB5GQyn1xMriTvJZ8kXyU/I4/IKckZybnKRcpx5ObKrZbb
Jdcqd0WuX26EokwxobhTYimZlCWUTZQDlLOU+5Q38vLy+vIu8lPlefKL5TfJH5S/IN8n/5Gq
QjWn+lKnU0XUVdQ6ahv1DvUNjUYzpnnRkmkFtFW0etoZ2kPaBwW6gpVCsAJHYZFClUKTwlWF
l4pyikaK3oozFYsUKxUPK15RHFSSUzJW8lViKS1UqlI6pnRLaViZrmyjHKmco7xSea/yReXn
KiQVYxV/FY5KicpOlTMqj+kY3YDuS2fTl9J30c/S+1WJqiaqwaqZquWq+1W7VIfUVNTs1eLV
5qhVqZ1Q62VgDGNGMCObsZpxiHGT8WmC9gTvCdwJKyYcmHB1wnv1iepe6lz1MvVG9RvqnzSY
Gv4aWRprNZo1HmjimuaaUzVna9ZontUcnKg60W0ie2LZxEMT72qhWuZa0VrztHZqdWoNa+to
B2rnaW/WPqM9qMPQ8dLJ1Fmvc1JnQJeu66HL012ve0r3D6Ya05uZzdzE7GAO6WnpBemJ9Hbo
demN6Jvox+kX6zfqPzCgGDgbpBmsN2g3GDLUNQw3nG/YYHjXSM7I2SjDaKPReaP3xibGCcbL
jJuNn5uomwSbFJk0mNw3pZl6muab1ppeNyOaOZtlmW016zZHzR3MM8yrzK9YoBaOFjyLrRY9
kwiTXCbxJ9VOumVJtfS2LLRssOyzYliFWRVbNVu9nGw4OXny2snnJ3+1drDOtt5lfc9GxSbE
ptim1ea1rbkt27bK9rodzS7AbpFdi90rewt7rn2N/W0HukO4wzKHdocvjk6OAscDjgNOhk4p
TtVOt5xVnaOcVzpfcCG4+Lgscjnu8tHV0bXA9ZDrX26Wbllue92eTzGZwp2ya8pjd313lvsO
914PpkeKx3aPXk89T5ZnrecjLwMvjtdur2feZt6Z3vu8X/pY+wh8jvq893X1XeDb5of5BfqV
+XX5q/jH+W/xfxigH5Ae0BAwFOgQOC+wLYgQFBq0NuhWsHYwO7g+eCjEKWRBSEcoNTQmdEvo
ozDzMEFYazgaHhK+Lvx+hFEEP6I5EkQGR66LfBBlEpUf9dtU4tSoqVVTn0bbRM+PPh9Dj5kV
szfmXaxP7OrYe3GmcaK49njF+Onx9fHvE/wSKhJ6EycnLki8nKSZxEtqSSYlxyfvTh6e5j9t
w7T+6Q7TS6ffnGEyY86MizM1Z2bPPDFLcRZr1uEUQkpCyt6Uz6xIVi1rODU4tTp1iO3L3sh+
wfHirOcMcN25Fdxnae5pFWnP093T16UPZHhmVGYM8nx5W3ivMoMyt2W+z4rMqssazU7Ibswh
56TkHOOr8LP4Hbk6uXNye/Is8krzevNd8zfkDwlCBbuFiHCGsKVAFR5zOkWmop9EfYUehVWF
H2bHzz48R3kOf07nXPO5K+Y+Kwoo+mUePo89r32+3vwl8/sWeC/YsRBZmLqwfZHBopJF/YsD
F+9ZQlmSteT3YuviiuK3SxOWtpZolywuefxT4E8NpQqlgtJby9yWbVuOL+ct71pht2Lziq9l
nLJL5dblleWfV7JXXvrZ5udNP4+uSlvVtdpxdc0a4hr+mptrPdfuqVCuKKp4vC58XdN65vqy
9W83zNpwsdK+cttGykbRxt5NYZtaNhtuXrP585aMLTeqfKoaq7WqV1S/38rZerXGq+bANu1t
5ds+bedtv70jcEdTrXFt5U7izsKdT3fF7zr/i/Mv9bs1d5fv/lLHr+vdE72no96pvn6v1t7V
DWiDqGFg3/R93fv99rccsDywo5HRWH4QHBQd/OPXlF9vHgo91H7Y+fCBI0ZHqo/Sj5Y1IU1z
m4aaM5p7W5Jaeo6FHGtvdWs9+pvVb3XH9Y5XnVA7sfok5WTJydFTRaeG2/LaBk+nn37cPqv9
3pnEM9c7pnZ0nQ09e+FcwLkz573Pn7rgfuH4RdeLxy45X2q+7Hi5qdOh8+jvDr8f7XLsarri
dKWl26W7tWdKz8mrnldPX/O7du568PXLNyJu9NyMu3n71vRbvbc5t5/fyb7z6m7h3ZF7i+8T
7pc9UHpQ+VDrYe0/zP7R2OvYe6LPr6/zUcyje4/Zj188ET753F/ylPa08pnus/rnts+PDwQM
dP8x7Y/+F3kvRgZL/1T+s/ql6csjf3n91TmUONT/SvBq9PXKNxpv6t7av20fjhp++C7n3cj7
sg8aH/Z8dP54/lPCp2cjsz+TPm/6Yval9Wvo1/ujOaOjeSwBS3IUwGBD09IAeF0HAC0Jnh26
AaAoSO9eEkGk90UJAv8JS+9nEnEEoM4LgLjFAITBM0oNbEYQU2EvPnrHegHUzm68yUSYZmcr
jUWFNxjCh9HRN9oAkFoB+CIYHR3ZOjr6ZRckeweAtnzpnU8sRHi+324lRt39L8GP8k+I32zs
Bfxd5wAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAAZ1pVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAA
ADx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IlhNUCBDb3Jl
IDUuNC4wIj4KICAgPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5
LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4KICAgICAgPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJv
dXQ9IiIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlm
LzEuMC8iPgogICAgICAgICA8ZXhpZjpQaXhlbFhEaW1lbnNpb24+OTcyPC9leGlmOlBpeGVs
WERpbWVuc2lvbj4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjE2ODwvZXhpZjpQ
aXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+
CjwveDp4bXBtZXRhPgqcwRHqAAAAHGlET1QAAAACAAAAAAAAAFQAAAAoAAAAVAAAAFQAAEPb
KmoCuAAAQABJREFUeAHsfQl8FUW2/glb2EEQHJxxQX0JA2MQBWXIk4lIBMU8QEFcAozIIItm
UASREPkLhEE0CpFFEWHQABPJyOYCwgiDI4Iw+ieKAk/FyAhCZElYL1veV91d3dV97w25N7m5
uX1P/35J3+6u9Tt1qs45daoqpgQX8cUIMAKMACPACDACjAAjwAgwAowAI8AIMAI2BGJYYbbh
wQ+MACPACDACjAAjwAgwAowAI8AIMAKMgIYAK8zcEBgBRoARYAQYAUaAEWAEGAFGgBFgBBgB
HwiwwuwDFH7FCDACjAAjwAgwAowAI8AIMAKMACPACMTsPXSS1zBzO2AEGAFGgBFgBBgBRoAR
YAQYAUaAEWAEHAiwwuwAhB8ZAUaAEWAEGAFGgBFgBBgBRoARYAQYAYFAzNHiyN4le/9PP2qU
bPHrK5mijAAj4ECA+cMBCD8yAiFEgPkthOBWgaSZvlWACCEsAtM3hOBWgaSZvlWACBFcBFaY
I5h4XHRG4GII8ABxMYT4OyNQcQgwv1UcllUxJaZvVaRKxZWJ6VtxWFbFlJi+VZEqkVMmVpgj
h1ZcUkYgYAR4gAgYMo7ACASNAPNb0NBFRESmb0SQKehCMn2Dhi4iIjJ9I4JMVbaQrDBXWdJw
wRiB8iPAA0T5MeQUGIGyIsD8VlakIjMc0zcy6VbWUjN9y4pUZIZj+kYm3apKqVlhriqU4HIw
AiFAgAeIEIDKSTICfhBgfvMDjEteM31dQkg/1WD6+gHGJa+Zvi4hZJiqwQpzmIDnbBmBykCA
B4jKQJnzYAR0BJjf3N0SmL5MX3cj4O7aMf+6m76hrh0rzKFGmNNnBMKIAA8QYQSfs446BJjf
3E1ypi/T190IuLt2zL/upm+oa8cKc6gR5vQZgTAiwANEGMHnrKMOAeY3d5Oc6cv0dTcC7q4d
86+76Rvq2rHCHGqEOX1GIIwI8AARRvA566hDgPnN3SRn+jJ93Y2Au2vH/Otu+oa6dqwwhxph
Tp8RCCMCPECEEXzOOuoQYH5zN8mZvkxfdyPg7tox/7qbvqGuHSvMoUaY02cEwogADxBhBJ+z
jjoEmN/cTXKmL9PX3Qi4u3bMv+6mb6hrxwpzqBHm9BmBMCLAA0QYweesow4B5jd3k5zpy/R1
NwLurh3zr7vpG+rascIcaoQ5fUYgjAjwABFG8DnrqEOA+c3dJGf6Mn3djYC7a8f86276hrp2
rDCHGmFOnxEIIwI8QIQRfM466hBgfnM3yZm+TF93I+Du2jH/upu+oa4dK8yhRpjTZwTCiAAP
EGEEn7OOOgSY39xNcqYv09fdCLi7dsy/7qZvqGvHCnOoEeb0GYEwIsADRBjB56yjDgHmN3eT
nOnL9HU3Au6uHfOvu+kb6tqxwhxqhDl9RiCMCPAAEUbwOeuoQ4D5zd0kZ/oyfd2NgLtrx/zr
bvqGunasMIcaYU6fEQgjAjxAhBF8zjrqEGB+czfJmb5MX3cj4O7aMf+6m76hrh0rzKFGmNNn
BMKIAA8QYQSfs446BJjf3E1ypi/T190IuLt2zL/upm+oaxf1CnPx/t301Y5dVFjsIaou4I6l
Zi3j6abWcRRbI9Twc/qMQGgRCGiAOOehwoICOl09FoUCP9S7iq5oJn77vgp/2E2nwS9m2EtO
096CA+CjWMSuQ9dd3dx3xKr6tvggfVt4imKrX7zuWhXOFZn11auEeLGX0RUtGjlqCFx/AK4a
VkbI83XoiquaV14fE2jdjBpIGnvOE8p7VeWV14FgpDwGxG+yUiZtjBfnPVS7WRw1aygDGHev
9ibeN6Yrrna2N0c8fqwwBIKirzN30Pvz7Vto739Ef6F/bHZdAv0OMkdDXzKHs33Y0qtDtevF
UmzDRtTQR1ct+ZfAv41+dRU1rGeLrD14Dh+kvcXo97Q+H/1Xs+htT0HR9wTGjZ/FuIGREP36
ddf6Hvf0MUCEIWqOvtQnrb3J4543inwhcKrU8c9AMSj6KhQIVF8oLiyggydMNldSws9Y8G5s
LDUE73rpGl59fRWWLey1cvVT9CrMhVtpyoBu9PrX/uibQOPemEt/6hrnI4CHPn/nRXp2RwL9
PSNFjnk+wpXtlWf/Vpo56EW6bs6b1PNqH6Ne2ZLhUIyAFwIBDRAn8unu1km0w0xlFH1WkE7N
zGflx+GN1LJdL+tF0lzaPu40tb0jzXiXTO/uzqU2ATbnwi/y6Olxu2jcqnS6zpfwaOVY4b++
XdKPkseu1dNtnU07P0gtlbc9u3KolVlfWZxU2vhdNl2hlt2JlRH0uQ/304D4AAGS2QR4D7Ru
WvKefBoUl0TrZXk/QHlbV055A6xelQkeEL8ZpbbRRtakx0LaMztFPmn34k+zqO39mbZ3RMHx
mSMRfiwjAsHQ10waQvD6NzJo0JQc85X9h2+Zw2f7sEfUntr0mEbTXxhM10ml2MG/bZ7bTO/+
0SnPeGjFnS1opJSDytDv+cjaNa+Coa+TPo/l7KRRtzqUZs9u9KUdK7QvDedYGQzBneNlZY5/
srzB0FeLG5S+4KFc8NZYyVuyED7u/cYtp+ce7WzKG06s9ChVU7bwUR3XvopOhfnwVhrUrpvZ
eZVG3UFzvqCMu64ygxR+vYqevXMgrRZvoCTsXNjHbORmoLL+OHeQVs8eTcOyVmkxpkKA7ldJ
AnRZi8jhIhuBQAeI1WOb0LAlVp2nrd5PPWDEqVvHeid+7V2RRp3TLMEvbcEe+mOztXTj3UOM
gCmU98VC+i30q1gIcJrzhj0J+1NhPs2ckERZ74nXKVC2FwasbNsTDPzp23fSKPkJo07d5tLn
L4O3q5FX3WXKnu/yqFUXWV/5lmja6sPUA11G3br6O9+KDtHEFfvp3v+KpRrAthbyCeUVaN20
spzbTWOv7Ui5RsHM8oKmtWqEsrSRm3ag/CZqaqONWfURtOrLSXQtsK6DP3Ftyk6ih7Ly9Qfz
fwqtAp9dgzC1wWchbkZmrtH6Ixj6alid2E1ZrTvSzDIA1/qJ5fTeyM5mSN/tw/zs+DEKBrt0
3WDn4N9kKMzT+8ZR9Zrok2vJaFCYh0Nh1vpdvCtDvydjuvEeDH296ZMKnsym3zZRxj0HLWRf
Kpy5vGYXLwZsFRgrL1ZEX9+d46XEoDLGP1meYOhLQesLDt6ShfB3H5RLeyYka1+dWMkoVVG2
kGWLhntUKsw7/tqP7p5gzCRpVE6m4aNT6PJaRfTR0gz6aLdK+lT6Ozq/G9D5VYPLks1i1GMu
bcvsQ3SBqGFTIoxDAV1OK1Lmyv105+WxVA2CdiNpJQ4oRQ7MCNgRCHSAKP5iFrXtlWEmkjh6
A83ol0B1Mc1s6cxFlDuwJY3dIIOl0tJN2fTrfXnUqY9UIPV3LSGYxTYmqn8R5rBb6aEwb1tI
LZB87UugeFaScmYTfDrNhYLSh4Rc2QB1N+VLWWXc7YNaAt7oykzPzM2U0S2O6qBPqAsNZtM0
KDqznIoOUUbufuoJjag6gG1cX0k4BD8DrZtWBIeQNzlvP3WH8aR6bZS3QQgK6YIkA+U3UWUb
beLQjnbrbUXDu2UsNb5UCN4HaeZVrShLw8hqa8K4tHTTQiorn7kA4rBWIRj6Ejn7S70KiQ+O
og4Niba+mkWfOGr17Dv7aeBNkAXw3tY+tHAJlNjpMvrllwO0y2gravQuEzfTKw/Eoe+xG7y6
pG+mab3jNC2uKeQZ/XII9WXo92RMN96Doa83fYjihy6nJcM70yXSu93Rl5p9fxB9aVUYK4Oh
vX28rNzxT5Y3GPqWR1+wGaNEIeKSKfHSA+DdfPCuLJV1z1y6n/rcHEslNmO81d9XRdnCKr37
f0WhwuwcILJpfXYqWfKfh1ZntqDxyyzii86tV6tYatIYcQfCGrvB+NYpG0J1KtWKgVANoUYV
qj3FRVTs8WBdC0yINWpTw7ribqUpfjkV5jGL99N9cRUkQJ8jKsbaJ7FeRpQhNrY2ZvpQBr6i
CoGABwjHwE40itZsSafLwCCmEQcuxnfDHXuHRPLh5bRthD4jUg1tvD6mk4tP4qMxrexTYT5R
RIUnwR8Nm2tr775dMhDu0LqnhVACFm1cSPEwHPmMK/Mt693BCwReaOiDF2yCjxQcwdsNwdu+
9H1LALAGNK1IXaFsT+2jzTA3qneQsqDozPRRVlNoUhVmrPMqPlIE0xx4FvjF1sP6pouxrQdY
ij0YcMXGYi2jj8WMzrptgzGALiBOtSK6gP6hob98gJ1Y/3j4CFGJQU9/Cr7Z5/kpg1ZA+c+g
v3iMrYs2cBEDoZm26E/90E8mHc57wPyGwpq0icfDLqv0QvGZdhcML8JodARLINorSyDMYJbC
XBuGqXqyoZrtCHBVR5tocrFGZCRoxvOxLlZ8w7gm2qbPdXdmmfQfHtDYAx4XLVPyuSOI70dH
3+A7UHjeBkNfT/48apUyRilwKs15/wXq0FzSxENb3xxCw7Jl/4egvRfStmdTqDEU2z2q54vo
G2EgiZfCBta771r3Ij2UrptStEwMuaRRw900EW7A0kNEKswx6KOboE3pl1MeMgyFpfR7MqYb
78HQ1+RfByDC4NXrhlidJx3jqs++X8QvQ78Y0rHSUQftUeP705Alwcnof5s1kVYAX4Ht74oL
IYMacazxUg/jFwN7EhX6FDh9nfwRoL6gem9ImULW6Aw8TGfeR+MXW8b0RBi1ZtyLPv9oHrXR
vNcqULaQ+fI9aARYYY4bQfOnj6cEY/CqBqHw7A95NKD3Uvp118ugSDeiLiPGU+LlBTTllo70
ji+oMbh9OT2FxCTR3k/zaMrEIbTax7qFfuNysU4hGeIGhKQVGZScNstXajTl3f30wKVQSjr2
M5WS6f/EbBRmd/QLSv3YFqbrbL+XN9PUe2A5FhfWSW1aOY+mPJFpxtU/4H/rFJr+7ETq+XvL
xdz8xj9ciUDgA4T3jKgY+O+8FgYjY1ZiL9puZ6XtSkPPGfCNNcM8gpZumUQtwU+1i1bR724f
qOPbOpWe63cZTZhgCXi33Uq0/mNf8I+glbsm0a/+lUk3PyLDj6C1302y1jdDEJkCt+HXjejT
P9JnbfVH8MI7fngBaz/HvTHJtkeBTfCRg1spgqMpALTCoLZTH/TikfEu0mfXr4ES2OTsWig6
/fDWGvj0MA4Le8xuWjErk0bOUoRmo04EzObPfIFuAw1sF9w8c1/KoLHzVG8ZESKFpuakU79b
jT4Bb2x16zqN5vQmmj5ijKqfUfdRC+n5tBRqKDNxCHlyhrlG7G76y/VSEB9F727uS5smdaQp
78mI4p5M01fNpZ4JDuHKT5nbPID1l+Ow/tLMXE/rW/SnWT770wR67OWJ9Ng91rovNfdw/Q6G
30zayIYh24pieDnzqcEDrVEz29jiVJg9tGnJZHporGNsaY1xbuZTaEMKPTCL0VITyojaDJ5L
03sSTUkZYluq1B1r66Zjbd2OJZl071jJgzq6j2VvoFE9Rbu2X9+uy6GsrDSvMfC2wdk07s+p
dhr/sIpa/sF/39AOSX9hJN9mcC69m5Fsy2wHDG13m4Y2R99gC1kxD8HQ12uZy7LD1OUKqzwx
mEaOiTlIs9q1ojfEa3gZpPacSCP7dda8T375UFkq4lCYY9A/lZQUUc6DLWm6nLEy2k0s+pRp
N0k+JWKF2cLc369g6GvyrzPRuEm0PncEXYlxs5qjL/VSFsvUL0J5g9w3cokzI/E8gl4He/5p
lMH3SdNo+8LBZl++KbsflnMY40RSNr6lWt9UDyjw2B7JY4fzKXdeFo31GpOSMb5Mso0vogSr
J2E51zzxi6jfqGnU7N9jaOYG7RH/RtB7H7alHncMkS8sD6tz+TS1Y5Jp2BEB7OO4GaXcPwKn
r0NhLqu+cEUsvOM8tC5NWe4gZQqjFhrvnthKf+7czfQwkTx6/uc8ukUsb6sI2SLE3mvlJkoE
JRCFCrNQCJrARdJOpfiuo+hPD9xKba9ri919daGi6BfonyVGuPO76WkozP+wR9OfEudS/uI+
VOJzUxZHhOHLac/Tne0CrCPIhGVQmButsq2PtK9vtjNx96mbaQ5csMS1HmtQB/nsUK1Mxi3f
Q39qpwhO1if+5TIEAh8gvD0f4tPW0KIBHaguXIzrVMOyBHhZWO7Yo2glZqAvh2J85kcozPfI
ATGFVn69kFpj9vTcHrnW11Ia2wDnHQbWvX5PtPxTX8Cn0NtfLaRL1mL2+QmpSDrWN0MQGQaF
WdtTAEmofOJ7zac9nxFv7KSnujbXXtoEHzm4lUVhtiepPWkC0XXYd/b/Z1KiqezbA1pC00HK
aSPdbe1h1Kf5mw/TbS2MN4UbaRBmHNerARy/u03dQK8+oCs0Zt1MpcwR2HhsPXINvfdEB/3J
ge1fsOlXX+EBQ3bMfaekv83ZdpgSmxkhsBZsGPaOkLTyjpdMizbnUiejjoVrMujmIY6O2hGp
9TiU91GjvI5v4XgMht9M2ngVWJ9NbAVPi+0z0a8bwqg9mKowF9HfB7ekCRvsIdSnJ3N20uPG
hkSmwUcNEODvse/soUdvkmOJh9ZPauGnnDLhBJrz0Rrqbhh/rDL47htS72lDOe/sMCI7eN/m
po4gDyyknVPLvwmnLKmve8D0dShKZCiztYzEa8Go1gD0FZcHs3H7TsIbQHijyQv9z6GNadRN
7q0AhXnpFrjgo781vW+Qh2o0JNl3XdhNU2/uSHlGWlIY5xlmCa73PWD6Ign//EvUM/MLeq73
VdSoBtzjldl+q+/H8pYzZe0X7XKfvfToK5Yl00O904zXCbToiw3USTNyO72ckunvX+bSjQ0R
9Bwmgq5tZxqc+2Vj8qVnHHlgyGolDVn2jMyn1iMW0jtjJL9ZZTPH9tbg6a91QzJ1zqaPn6lN
t94p5QMYjPMOU88rCujVW9qR2rV1n7oB8izihuAKhr5B6Qsoe/U6HvrnGEVhlryvyhSYZGj5
BwsTyaNn9+bR73tb71UoApMtQr/cSy2b239HpcJM+2HV7mhYtX1SOAEWssH0QI++9Bu4NwpL
EEFhnozBZ5mv8Jhh3pR+C+W0b0Wzle+padPo8jP5NO3VHOstrI4bYHU8+XEG3aXM0lkBiMRM
ThdapczWWbNRNTC4NqpndU4inmSymFNb6albrc3M7hmeTX9ocyV59n1MYzLV2YF0WvflKLpW
dJh8uRqBYAYIwmZ0WdeqbsS6UnxlA7Q9j90dOzENa5wH6IPb+f2wimKGSr8sQb7a4Ty6UbEs
2wEfSqN6vUpZy+1v9Sd9lprWpFHfZyUPWelqrqpQ3tSNqaQgUgPlvCFRdWFNoZHpyXQGm/bN
XmbNyApjwJLBHbS1ZjbBRwqd6uDmKKIl7IsPydSl0wH6aJMuIMi135/NakLDF+gRe6I/oNVj
aIUxEyTLevSLTOr6J4U/E0fQmKRf0a71GbRik5XpSGys9vAtEKjr+VgTmZhKw+OIZi+QOOll
WvhJLiX+xuHWiXCAjRIfSqeOdfIpa540Rog4qZS3LZt+ByU31iHsy/LaZ5hFHHEl0+NP3Uw7
Xsykj/QX2n/hVvwSvF8aNbD3WeJjq3vSqXujApquljkxm/69IJWaxBbR63e2pCnGbOpv+6RT
aqcO1OT8j7R0dJotj9c2HKaklvYlMUoRKvVnMPxmb3cplPjLKvrEaCOa90b8zzT7pnY0X6tJ
AtpxX/ogM8PwDrD4Yd+GNPqfpxT6x6fSyG6X0QfZWYonQTLlbsulm0Ffe/vVYYpPHkUpVx+g
F19X0jEQ/MPgdEo4/Bm98o7FP0IB3JzVh5rBHfwnGDg6Owwcib1TYRnLMeujJ4VNkb5BG8NY
5qsMRna4jaC8xfHU58E089XwV/fQoJsbUWMY76o7hE3BH6nX68q71jfUMKNV2I+A6eswOkk3
e1Ggmqi/r6UIRwq1bVG0MsdAMf4FdLUUZhgclq2gtqg/QSA/d6SAVs/oZvfueDCXtj2JmXgo
zM9DZlmqpaTICcCFXbINUBy3gOmL+Db+daQn+sX562EEvHI3jVQMu7Iv9VKqEN9/v9iXPp7g
f4Z56abBtKJTO5KcO2jWThresTnVL1lLrTQvJ6twGq/A+6dmUR61NbxMhBfUnDUbqEPj3TQd
k0MyHT1WMvWEV9IKZewU73s9v5my7o/T9vfxWq9rZUc9X9hJT7fcaJNpJ+duJvpbR9vyx8S0
5ZAn9OVdIrou7yoJlfNnMPQNSl8QZQd/rh+tKMxx6bRy3mN06YXTVLORh44WbKSsXvBGVeqU
+tJOGtm5OdkV5uBlC3/Lp5Qs+WcACESnwgyACj+dRzffP+aiUD02dzONwgY++gWhbwAY4J/G
o1zDLB7PYz3CX1+hjd98TB9ugNKba1nRD23JpG4jpEBsrEHCZg91fsmhNsqxNNK1VSRnd2+1
FGadAezCp1SYz/6QQ7/vI4WLBBozay71ah9HtcTs3+48Gj9/OyXc0pbir0igtu3iqDYG7EtY
aRZwu/YKaoAAGs6NLrQBHrOLJ7faheLJcC/sbrgXVsO6mxu7DjGwtAR5OpRH7bvJ9+LzCFq0
bjy1pALavq8xdWjdnPYsG0h9M6XiBh4x1jCL0Hve960wazMscDv0pTBXh6tTzltv0eebcmj9
11C8N8E9vJZIDZvszEqiYQsMy7fBw7Xx7dA/FNfHgBXmETR5YmMa/2ymnomI/3JnmnNLK3pL
f0OT31xDO3GUnRREpNB0FLyZu2oVXGlX0fnec2l+ur7ZGGEGbTaMcLqiBAVXrG/Cpj01gWdb
Bc9EbDAzY7AuZJyBm22nfhbWQjDqD2+Sw+uVuqE8wyFMDbpFn1nfA0Wnb/oso5RG/wTPgKZY
A+kT21qYuUqwXD0JRsA1b42gpuhn6GQ+TeqcRCuM1GTfVOvwKkq4wzJSdhm9gaZhMzlxib6p
04NWmWe8j5n0y/NpzA1JpjDRc/RyGtm7MzUQNEQe0yfOo4atb6ar4+Op7fUdqCmMOdZGRlqy
YfkXDL/ZBO7e02jyZWtp/Ku6UqrRvHMB+Ee49YtrFM1/M54GDZB4GXxWvYBmYqbmr1oY/Euc
RutnDNb35nDQRAhlT8Krou4v0vNDj6S2o/x5/WiQUQbxtUs66NVb0MtDb8PNcNomPY6czaxd
o4Dm3GDNVAlFYUbem5SoLSPyXqMrZt4mYuYtFq6H9p3mnX0D0XzVEC3W9qZjVgsKekGe6o6N
ZRBbsrWZVyrFyGWUOuhbwPR1GJ0kP4gyOvc9kYU6cwriBNy05e7otvYhA5Vy14wK7WE44Bnm
UlDy/Slg+iIZG30enEuL2u2ih0ZLeQ8BHlxO/37qKpp2YzvT7Vj2/dULV1E7nLoir4v2izAM
Hljhe78PkcbWeRjbXjXGNsNwUowxu8sw2b/rOUmvsZ9WD6Se441xNw59xuLBdAaG2m6jrfCJ
Q3PphcHJ2h49Z/ZupNG9e5kuxGIsX7l1EsU399AH6npdZJOauYFGdo2nPV9vodhrO9OlB9Ul
W3o51P+tHs6l+SP0fMR74QnR+JKK3fk/GPqKsgStLzgwEWn5v2CwWCcMFlgWapthDl62YIXZ
P9rBfIlahVmAdf6Xg7T2vaW0NMe5M7YdyueW7aEBNwrLNRTVYVCY3ze+SxcLGdwYqI//THT6
5EH66YedtOuLj+mDNdgFc5cMZCkSNYtV656lFIuQwSjMMUc30k1d1Vk1Pc/45FS6u1My3dy+
M13bQrfAy9LUxUwDZGO+XIpAsAMEqWsLgU38ULhlD06g959qQc9uMMASa7QWj9CEcjETUu+I
Kvxa7TwGCt5NioI3CAL7cFhR1cufUizC+PumbXLkT2FGoxY7T8OYS0eOEh07WkB7vs2n/G1r
aTVmNHfKzCUPo/xHMJNjHisVhMK8KC+ZZveRwgQ29lnQmaY/PMSY3YPSvr4/rbjNstybQpNR
1lNF0AVPeejQvgL6HmXdumkpzVcs+lLYPvJ5FiUPMRRzYwYjAQqjdsV4aPe2LVTSvBVd3rS5
6e7500dp1HOMoaordBNxYn4CfXo6FDAopnUcmwaZ5XUozKqhT6S35pkmlG5MQsoy2415MCgu
3kmJaAJnzsdSrVPbaVivXiZN9JkRopybWtF0kaB6xafQ4B7J1LHjrdTmmqtsm7HVAgYNYIgM
5xUMv9kEbhhMlnY7QH2HZujV6JpNc7oV0DApgPfOpfUDTtNtvaWQbfAZdmgf2ymJ1hmV7zkR
wuqtV9GZMx6qVesUrbytHb0sgZGb9NkMWcbskpi5xOXkOXWjqR8+GEh9MgwhW/JJ0UYacHsv
c3l16gtf0MjbrtITw3+xRnc26PmGfGPwXW2H94nvvgHGtGeN/Iw1vL9tUESLrm9p7Bou+6cO
WurV4NEcKkNwwPR1KszGRm52t2gJiu+7rX34DmK+TYTHzAwsn9GuEswwd+AZZhOcMvwImL5I
00YftOttU5Ppbawpn2Z4iYhsM15dTgeG9qK5RhlkX3r+Pzl0Sy85yVGWfrE5HYSL/l2qi76x
S75I+szueTA+jjFyGUFrtk2iH2Y0oUffMl7JG5TjTYv708rUFjTVGAyl98NWGMuGmcYya8mV
jHr4s0y6Y7hlEJiMTXHvbEX08ThlNlVsFrotnYzuRIvqlGllevpdL6sML+SJxk0qVlkW+QRD
X1nOoPSFABTmka/upNT2ulx0FmPy780xGUbEcsoWsg58Lx8CUa0wC+hOFUNI9aCjOVlEP323
nbZuybO7UCOMcK98JTWBLhG7ZKsMIIUFaS0W5yrPe4Vm/GWWKfx5k8dSJJwzb7ITFce21IE1
TrW8m9804dpeDimUxtTw0MfPo9Na4p2r9SaFXng7m26Pa0Q10CnFQqkI9RmwVt78q7IRCH6A
sLvEEqXT0vc704y7utG/jErIAVY8Ctep2rbZIqudOxVmuXmUiBcLJUes2PsaO8H2esZQ6MQ6
PXlUDuw7P7+rKLLKt4sqzKfyaeHMLPp/f5XCtsjRcSk8bBNElPcX3yVbpIkZro3j6ZPOPpQ8
8bkrZsYyE2gSZgHl7KvK02e/X0VzXsmiBR8aswMijuOSfP4DZgX6yFkBBQ8hZDSCkIEbnTkO
IwFmquRVgI2D7h1n4CuNBPhYDYqxfaZRwb4BlqHEWzPJZnkdCrN8L/Kq29RDHz5mCU6+yyxL
5fsuZ9L3woDRW3Ux9hF8+IubafDtOFsWla6JvrEO6hPOKxh+swvcC7EPRSMaCsOnaWNVKiRc
GzPabKT2d9kNHPYj3ZQIvn4aXhXCUGIJZdYMrbcbsNUmhKJn81aQfPKfVdS+l1TiiVQjinQ9
Xo9Ngcx12Ea8mP+gDHBLlJevvuH0wY30wK3WrvzD4TUx6NefUfs75Ky7voxJHHkmrlC5Y4u0
A6avB+v9sXZ1tYiMSy7V8Kkwix2SPbWxC7FeDz2GQyGTL9U7Ngnr0iaZuvf+E3WBt468qmHd
7PM3WPwredGet12WkB4D4uQPf/2eTN+N94DpCxBs/Cv5Aft5tDf38/BGSvaZP64dSPc8U8r4
pEQ1+0V4QvV+2vdYWd9xlNjkNzfQvgFJtqWCepIpNGfxI7T0wV7m8pZp8Ozpgubz0UtNwL9G
xnE4CWYxToLBo+gX6mN8qeVY0qjXBfGebUFj3jfiKWNMTE2MS/BkLP4SHmj/Y/G6EdK8CcVb
29sA3hXBHNNqJlTKj2DoqyZXLn1BTcj4HY9jplolJtN9fftjlt7i+5LCPOpgrvcun2wR6iMr
fVTLta+iT2HGZjlTJrxBe09g7c+GfJqDjXTugFnrxAm4QZzDjpOC1Ee30qSu3UzBVg4idWpj
Ef/TljAo3+uDSxEtwZq7CV872krnFOq6cZVp+VfPzXQqErITFW4UdQ4EozBjbRIs758smkeT
4BrqS+DSS6ds+uAoLj+6C4HyDBA2QcALFvuMVH14KpDt7EBFyHbMMMt2LpQ1eU6lPS8rrlCK
968KQmE+vZWe/G9rPb8s/h/uS6V/vi2FDbyVAg4ExEJY7u+UlnvlvT/B0b7+EmXGZjzHXk2i
QdLdW2aKu3BBzejmoTHtO5oCisThJGYFOpuzAlak5L4ptHapJUxJgdeuMFuKTiyEkvrWmEvH
YQwU2nNNCCx7VaODrBs+l0a32lCYJwWgMOvnM9sFcLPMK6HkT7TqYtXS+5c0xIj2cSg/h2ZN
TKPlymyNM8ZYbGL4aBXZxDAYfrO1fWOGan77lj6EXCiGYgkEtnFqb24Io/PKr/fmwBXfmqly
YmR7NuhPjk36pJGqFoyoBz7wzXPCFfpnlR9lWgVwt7zXEoalwiyE5SaIIy5fCjP8qm3xJE+o
fYPm2aUYquPhuvn0ddj0zlxGkI7ZrFH6bBbaeyhd8wOmLzZVysKmStaxcvrM26UwPFjriAU6
4BvjyMo2Sal03z39qVePDtQQ4WztAx4li7AmVhy5Z16os/PSZujgITIO62ZzjY+Sr0pVmKWi
wwqzE1K/zzb6yL4V+O1ZnER9s3wbQGU7//E9KMwTAusXC7Cnx73pcgyzj5XiWDkbn6mlfmgu
zbh6I/05U8ZVP8JbbJvwFvPQmiktKP0d45uiMNeBnFxXHAzuR2Fem96CnlljxJM44FF6MdrH
SyOcejM8ny6DLAEROCRXwPxbkfqCcHl/y1gmI2vng3eFh0xdKMzWhFnwsgW7ZEugK+YedQqz
8+zj257bQPP/mGCiefYMZmdOF9GcPi1pltQ4DeaPrY4Z3PGKwqwMLrGHcuh6ZT3ygIzl9FBy
Z2oqBraDsL7fJa3vVgdXmsLcGOvL1N3zpOVdZwD77J8USs2BEOc/f/dtEZ33/Ezbv/yM8jeu
pZx1ykYtKFIGznzu0xbnaUI44su9CAQ8QKhQOM9bVr8p6/dlu7MPiP7bubQkqxt62IQOZdZU
KMyH31ePsbIURG2G+Zh9EzLfgkgCTXhtLnW7QV/PfwjrebuNNoQGObBXkMJ86bYsum1EpoqU
9nsydgTtfsVuGoPNVD4yvuplJfrwiRY07mPjZasR9Grm49T+NzD1YzD9KBPW/mX6N8nnh5HH
HUOtPGaswU7UEGY0PGoKhbUbdj29lXreHE9tfp9CneIb2YVupc7CGHDBj6EjYIUZUk7j+r4V
5uNfz6KkAYabMaozAycBiDKju9WuY8fhj14rFi7kteFCrGv9Wrtq4KED3xVR0blT9P2OrfTl
F9i0bbFdwEzE+tpX7kugRmgrQp4L5xUMv9naPmgjzsi2rbM3K2S0faxva99HKqcGn3lgILqt
G200wg566Qsanvgrgke2dnng738GvhwN6gBfcbA32tYZm7Jq8WttrBvcv9K3wuxlwJJtCTPF
7ZWZYumSrRtRRBEcO/UaY6dTYfbVN4jYxaWcQCEVQRFO7VPEc0VfwdB307QknMphKU7C7XzE
bc3NY/q0MjqWwBBNou0FI7Sjf2ztQ+kbfdUtBv1YDZC3oVii4ZjdJqPPjq2hKOsOl3EZhmeY
r/QFr893NvpIfhAGB2yelQ6Dhdx0TY0sx6mTu2fBWBpYv/jD6jR4GEmlV+Fb9H9CYfbHK73g
nTLmyg99Gtak54Mw3KyFl+IzZqEtl+wGUGRhw6TCj3HEXariko1NartfjXEMCjMOLdAvRTaW
Bme7fIBgndJp8m1HaXzmLCMS/LTAG091a06NQiSXBsq/IdEXzNraf4jj5WphDK0PncGOFWgM
Y3zgsgVOtNDGZHs+/BQ8AlGnMJNj10oB3aDMNTSoZwf6tRhkiuFWvWQ0DZtiCWVmZ4L1gRvg
dvLUewbgUmm4ALfS43l0vbnbIKzAm2AFFr0LLvuaEKuD81KYocT2jEMjR7zG9ewb7vTE2qeM
u+B6CAY4++UsbFhmdbJSkN63NoP+5xnZ+WAdzZZcukZasA6upYfu6mfOOssO2zkrpZeY/7sF
gUAHCHu9rVkP+3t9YBO7OYpLzEg1QLv06uQNt2p7O7fav9qZ24QOCIXapl8Q/GpDqaqHmTB5
XqzITwrVQnjfPkc5XxLfZLv+cW0aXN0MoUJbrzVYG+zF5nzz+2M3ezljqQg4FTHD3BIKfHuv
fQQMRQdbU0+GwrxcVAKXqTArgoa66zjOk6GHulmuuZLPY7Ab+U3mbuRICOte14/FTsVQfs9j
A622pisXUXec0T7nnriwK8wxh1ZhHbs0GhLZNrdRNiqLj0uh4c/PpUScEmDNohJNfHs/9bgm
VvcAAg1nYzO1+RqK1mZooVy7amR10Vsw/GZr+4ageQzrBLsp6wS1jPFt29Q+jv0tDH5ybvqF
tc/rp6bqm36dgadAp46ax5RwAbx96DQa1Bnrm/0pzOCr0rw6bN8k/1wooNk3y528RWn1Tb86
t4qlxjB6bHptCD2kjKlSofZXBrVv0OruVOy0l+Kf3dNFzmaZnyv4RzD0dc7IiSKlTlxDjz/U
gUQPuvfrPJrgOP9aW8v9h+ZUHcrtPrjg9njSW0HSxm4oSBA/tKual7WoiN50eL11GbqQxj50
B/3XlTjyrljssD2QRs6zlHmz/xEKH/oTJB9VVzD0tfGv5AfgJzZ1O+3H0CPHqerF2PSrSwD9
Is73dSrMtrFSEOxEPg1rbW2YKAmoeadclk9/xl4Hn8iXxl1OyIjH09sz6L8fkXIk+ldj06+m
zeCBuX8jjezYy1xioG36tWUSjpXEUhxlHJNjlboBn10+EOMfjpW61nGGuFjatCmbrv+1vlTL
UcxyPwZM3wrVF7ARKIyhQi2oD7lG5S0n79qx0hXmwGULVpjL3WAcCUSfwgwAnDsAOzDxehz5
Jo6raI3FlFCYP32xBT2+xBEE5zBvm3Aaa6oUl7hOo7D5xq30w4fP0svvWAOS6pJdG1v6/+72
IY7EiCYs309/bI2NXpS1TyJQl4cn0Q01P6OX5lrKvPYeu+dOw+65x+HamWRz7Uyh8a+k0rUl
B2jl22m07F8yK2s3PmsGQH7ju5sQCHiAcFS+cB2syV7nCMMgtNFwCzQEAzEIeHXyF1GYa0DJ
lpbkvSvUWWRZCAwUOxbS9T/lUCuVtyAkDxqdSjV2jKG578uw+l0KIk5Xt8SH52JDDWxI9coQ
Wm5scqLFUASccivMWn0P0pvY1TdbLZa0tuNoukwozMaEsaUwqzPMiDdoYi51qL6bZqRn2PZC
kEJITI0iWjuypXIWtsgsmXrfR7TsbdWTBEeabMilG9B1/YhZCXONuFLnypph1sqMjXDGfm0B
E987nf54A9FbEzLNzaJEPRZvzaWbqm+lR2/sRhus4DQ0fS7dAqX5q/dzKOvvVj3lrsDS00GJ
Uuk/g+E3nwI3vDsG3GFtoiUq0jMTRlOc2GDfPMcyQO1Z2Y/6TrRwobhUenZgAn2zaAwtVXAf
nbuHBmMTy3NQmNv42NXeaxZZmdX0+qa0pcJPsKQhTSp2OvSJ90Jp/yaHViv5W0K2/1lutW+Q
RPw8O4nudbq4yvwRqDLoHwx9RfnLcia8rKfYCX0NzrbXdp3HywMfQ2GWS0UctBAziqVdhetw
lrmi/JQWVnwzTz1Q+vWLxXHT92Do65N/TYMDjM7qRrEGWHKcql6niNb0DaBfFN4f8LhKSrMU
Wj1Jfaxsr83MIs+xzr1ssJu1odiuGQmXa1MWFLGtJQ3Clb+JmKxRzozW00+me+8n+vvflP4F
H3pic78MbXM/zEwrLtlyrCpdYcYEEc5jP7Mby0mUo+OE0SYbG4w2wbhV0Vcw9C2PvmCfYDMU
ZrNt+K+db1kqUNmCFWb/CAf3JSoVZuEe9uaAVjThnxcHTZ0JEZ3JCQxenR1CAcWn0yerUmgB
Opl5F08SGyrsofuw6ZbT5VFGHTEfbim3N6d/zUii/i+pyrYMYb/Lzknb9Aud1sj37N+9nuQu
qfjgZcn3CswvIhmBYAYIW31hrR4Ea/V69aVxtIt4FQNNWQ5szk5e7qzrb4ZZCN+mwOc4U1XP
LoHmbdpAtzc/SK9c14pe0l+W+l8KIuexAdktd3sbo7wjG7PZ9cq/hlnWd/sbSfTIHItvpaIj
znJ/BgqzFDlkWX+Ei/g90kXcu4DWG3i0rM9O1dY1Njm3kXq0tytUVkD9l+kZg0fbrIRUMoyB
uzSX7KewhlluWCTLW70W6oFjpbzea+5fvl2yNWUGZX4YZd7gLKjybGKFd3vXYXObsXYFTAlq
/ISwtwXrV9E3V4bC5J2//U0w/OZb4D5Ii7Cr9MtK8nIWyDkrK9td7QYHafGjrWjKBiWS8yd4
dxOOZQLb0jls+tXR3InVOE4MH3SlGEfXPCENs85viru2rS156H2xx8ffnJmqzwlwx19DiZgp
E9dZ26ZflvJv6xtkdHgiqJ4m4rW2o7pxPJr0dJHBQ3EPhr56OTz0Lxjr+r8oMfVXugSalreG
uhgbmGnr+DHDbO7eL7xvYJgT3ms+MfJK1pfy5BVIe5E4Gjts9+ug/dYUpya+w7n5bTD09c2/
ygw91sE6+2qzLxV95qnA+sWaWK7XtvsQBxmMsbIFPA7gaXDIaeiWZ3MjlrNfNb0K8E0uafDs
whraO5x52LMUewksMI+B8tg2/bJkUsv93y4fSA8rodR5aPPkFjTMnIhCH/E+xv04RT6wZx30
UzD0DV5fcCzhtPWV9hlmZ4XsWFk8H6hswfK9E9nyPUepwixAw9mQy5bSqyPTzDWFKpTxnUZQ
/wGDqXv7q8zX1Wpjk6JaBfT6kHYOoUR3I2l5fCu9PqkbvfaxGQU/Uugvix6jg7O70cuGH4w8
01IMeIc3Yw1LquVeLWJKobF6rSL65+whNNLc4l98TaAnps+lDueX0oOj9HUk0l1bFxg99MXy
tyjrz2O83G5EWTJmp2NtI3oi46oKR7HIsvC94hEIboCwl2P9JGxkpbjtibN9U6/Xzb+xuNWv
pYe3d/I4RgkW7ZZQZGoXrYInhXQ7M3gFcewCn7fbpkhVKgmxJfk0+8kkmqPyFmbQsjMfozPL
h9BTi3QFVYYXA8XZ7/PoKbg6ynWdWimTJ9G8gVfS4tSBJt+L9fw942PJNkMGN+dN4jzkUqzB
/up7fGcWJaVm6qCIOmC9bnehIOBc1Odvtta0yfeirD99mEl3P2GtCxORuwxfSA+3+pH6p8n+
AQMnzqdu1dAQQk7spr9mZtBzi6QKbmSJo5cmPzHR1nf9tBHHSkm3TmfdbOsnFfo4Nv0ysYXC
rJ7DbL7XFGai1dgNeZhhOZTrS01lFmV+LX0ITV1mGRS0Uvsos3A5PfXVKsp+cSDlbDLqptwe
Gb+cUlNwNjPamLgi1iV7BRQiaYSVxii0u68X96MBWZK2Fj/ZlUzrvc5PRfSP2S/S4OedM1AJ
NHziC5R6Vwd9aQLwOrcPCrO5ay3obpxjLNI5+RE8S4bI9mjPQ8xwJcsZLuk9YfKJh/694i2a
PXMMfSSXPWjUgWvnw9k08uFUaqlsWFUHLqltTJdUpe2hDKYxzYhP5HQx1vlB2wAL+fs719iM
XgE/ytuf7vl0Lc3P7uezPaeOXkj9e6dQU6M/FcWt2xR9g9hkTbYPnHtr9qs+MfJVSQ99/rd5
9NLTGT5kAoRHPzr5yVG2/kIqTr5Sc/O7YOhr845y9q0GWDalGu+cfSYF1C96KP+v9iUOIhuZ
ptiYr/6prTToemvTS/W4tmqOo9zkEieRhrqkwbMvnxbMzaLnFziNPMk4Jmsa9VRkY7G532ev
lNLvi8Rt4wzRNGxg2OUKY9IGRgPVqCCMvdkPYpa5go02wdBXFD04fcFDGzOVmX5prDT7Sj1l
X/8rUrbgXbJ9IRzcuyhWmHXAjv4ijpY6qB3BUqsO3EOwC02DBo2w8YwdUGFxVc+FO7IXR0Cc
wBmX1RtRg8a6tVzEqIvOquiHg1R8Hms7a+FbA+ubPUV0FFDAG4t109ikq2BPEVR4nEmKvBsY
eUsB8NjPRbT/KHZvqY5NW0TZDAHRmZ4plOJDEep1sriIjsldX1DOpko5tbiwRF6CAdlr6ZMz
YX6OWASCHyAcVQZfHMa+TCXqa3T8jbFOy9kcz57AVgAnrYDaAI4B1fnerjDr4c8XFtF3h7zb
ujZ7BFY68AM2gNLOlr0IbxnKm8jzp70Htc2latVBnLr++VEq/yePoE/AjvnaVQZB3FkvWd9j
h+ByKhcYIjFzx2CkfRh5SCylFfgk6v0f1F+s3vLVB+kFEglBIALuoibncXTU0YNFdAjnN4vL
2R+Jd+psUWl1c9bDpI+f8pK/9yJTXKIPOmdUUu2bRJkL9xdRsSgzGo+vftLECuloZUZfJuuo
4dPYux8UykWdMHdm5eE3f7Q5UmitUxW4SmXGSS/Z7sSi1iMYMwqP6PiKcaMpxg0boypCm990
JPaSF/Bs5uH85uATbfzBMUnHZLtsgDPBHWOqPG4KSZWpb6BiuxJAyqyZfVdtkWJorvLQV5bo
2GGi40cNbETn6WdcV/ERcW3tA88qLcT3i12SJh70n2cgnwg+UuUNGd8p68j30XAvD31t9HHw
g8DuAvq9I8ftKMq+X7wNtF8sdazE+Ccu7PVHJzF2mxf6x8boJ0Wzs5UXz+o4IcNrxxMe99Cx
Y6fRZvRxuSn6XtuFNNVjoPz1+zKOs79RMSjGmHFWDoyIUBOeXw0VA5tMI9h7eegr8gxWX1Ax
EeN3WY17TqwkzwcqWwSLF8ezIxD1CrOAQ2zQKhRlhU8tlNC4a2CgF+fI2a6z6PwQ74IjkhAy
a0PAKIag7vwmdq+8gHzke1WIxDHQdErt2JCZKTRCACqCgH1OEb5FWYSQUBP5e1AWcanpiefj
R1EvfHMUUXzSNhZzylDaB/7nKgTKO0CoYBRDATyrtEFp0FHDiN/OTl6ei3oePHFUUaRNhcyR
gBAodYHO+iCVhAunwQsQOiQPyRDiDF5YnMzyyfDi+6lj+EM8Gx+Ar4VR7CziyPfSgHUSfHPK
4Cl1DZbMy3n3V195ZqMMb5bJoWia7xHwOPjcoygoIq4QZISRzKP0D1K5F99Fn1KMOp5XaCPe
i0v0OQ3Qd0k9srS6+aWPv/L6e69nTWp7cfZNwoJxtBhlluAbcWSZnf2tRkNBKx/hq9WAoRKG
R7H7b7iv8vCbkzbSGOXkB8lP/tqdxMAZT74XeNW7BG3KeOFMR+ySXc/A8hR44aTkBYSXeYuo
zvI6N4nyN66K3WBrg151ZQGQlv+256EdX+7CgHWUVjzdi15X1kKPwd4i94m9RXCp/KC9CNG/
8tBXLVJp7Vnwex3gUwdGRvWy4Y0PKi3UcKX9Lk0mEPFEf+HkvdLSc9u38tDXRh+ML05+EFid
QP9+Wunf1b5fwzLAftEXj9vSRJ95GP2s7DZVw5KT733tGyDKdBaK/gmM277GF9h6qLFDNi61
30d6Tl5XDUNOo4IvJV7DKch/5aGvzNJfv6Z996MvOBVmX21Dpq/enTSSPB+MbKGmy7+DQ4AV
ZgU3MRl7HpbXahiwLkD4FDtU1nIMWkpw7acWB2GrCUbBjDGimtcZCOlCIBQ74NVExyKFVjOA
8wc6UqH8CrlXCDWxjrzP49sZ0dkaTFnzognqGcgyak+IG4tyljGqngD/j1gEKmKACEflz0Nw
0JRm0dbBCzUNAV6WRSi6woBUZt5CRA8Gfo23wKSxirAu06wyd/C4VNjL0gfJcpuY4YXoj2pG
AJ9rdBSzXShvWWip1lHUWwj4Ze0HRfhQX1WN3y5gzNAMqsBXYCyMRNUrufMXRzWe02Y0QWMf
41ppNHEe62KFlefG4g3qU1meUhVNX7X9BzquW1gE/kvQRMg40vAYKF0CzzEyYlQ0fYOttdou
LtYvmn0i+NvXWBlsGbziCfkUf1qbEX2JQ971Cl8FX1QkfYPRF6ogJFykABBghTkAsDgoIxBp
CFTkABFpdefyMgKVjQDzWwUjDjfsu7EWc4ctWfvGYdIzxBYkRA9M3xABW0WSZfpWEUKEqBhM
3xABGyXJssIcJYTmakYnAjxARCfdudbhQYD5raJxh0v2ug9pF9bpHz1emy69Io4S2iWYG72J
3Oo3g8dIRWfrJz2mrx9gXPKa6esSQvqpBtPXDzD8ukwIsMJcJpg4ECMQmQjwABGZdONSRyYC
zG8hohuWYBzCWkznpa5/dH4LxTPTNxSoVp00mb5VhxahKAnTNxSoRk+arDBHD625plGIAA8Q
UUh0rnLYEGB+Cx30YrMdsfGQ2MBI7DMSi91zK3ujN6Zv6OhbFVJm+lYFKoSuDEzf0GEbDSmz
whwNVOY6Ri0CPEBELem54mFAgPktDKBXYpZM30oEOwxZMX3DAHolZsn0rUSwXZgVK8wuJCpX
iRGQCPAAIZHgOyMQegSY30KPcThzYPqGE/3Q5830DT3G4cyB6RtO9CM/b1aYI5+GXANGwC8C
PED4hYY/MAIVjgDzW4VDWqUSZPpWKXJUeGGYvhUOaZVKkOlbpcgRcYVhhTniSMYFZgTKjgAP
EGXHikMyAuVFgPmtvAhW7fhM36pNn/KWjulbXgSrdnymb9WmT1UvHSvMVZ1CXD5GoBwI8ABR
DvA4KiMQIALMbwECFmHBmb4RRrAAi8v0DRCwCAvO9I0wglWx4rLCXMUIwsVhBCoSAR4gKhJN
TosRKB0B5rfS8Yn0r0zfSKdg6eVn+paOT6R/ZfpGOgXDW35WmMOLP+fOCIQUAR4gQgovJ84I
2BBgfrPB4boHpq/rSGqrENPXBofrHpi+riNppVaIFeZKhZszYwQqFwEeICoXb84tuhFgfnM3
/Zm+TF93I+Du2jH/upu+oa4dK8yhRpjTZwTCiAAPEGEEn7OOOgSY39xNcqYv09fdCLi7dsy/
7qZvqGvHCnOoEeb0GYEwIsADRBjB56yjDgHmN3eTnOnL9HU3Au6uHfOvu+kb6tqxwhxqhDl9
RiCMCPAAEUbwOeuoQ4D5zd0kZ/oyfd2NgLtrx/zrbvqGunasMIcaYU6fEQgjAjxAhBF8zjrq
EGB+czfJmb5MX3cj4O7aMf+6m76hrh0rzKFGmNNnBMKIAA8QYQSfs446BJjf3E1ypi/T190I
uLt2zL/upm+oa8cKc6gR5vQZgTAiwANEGMHnrKMOAeY3d5Oc6cv0dTcC7q4d86+76Rvq2rHC
HGqEOX1GIIwI8AARRvA566hDgPnN3SRn+jJ93Y2Au2vH/Otu+oa6djEluEKdCafPCDACjAAj
wAgwAowAI8AIMAKMACPACEQaAqwwRxrFuLyMACPACDACjAAjwAgwAowAI8AIMAKVggArzJUC
M2fCCDACjAAjwAgwAowAI8AIMAKMACMQaQiwwhxpFOPyMgKMACPACDACjAAjwAgwAowAI8AI
VAoCrDBXCsycCSPACDACjAAjwAgwAowAI8AIMAKMQKQhwApzpFGMy8sIMAKMACPACDACjAAj
wAgwAowAI1ApCLDCXCkwcyaMACPACDACjAAjwAgwAowAI8AIMAKRhgArzJFGMS4vI8AIMAKM
ACPACDACjAAjwAgwAoxApSDACnOlwMyZMAKMACPACDACjAAjwAgwAowAI8AIRBoCrDBHGsW4
vIwAI8AIMAKMACPACDACjAAjwAgwApWCACvMlQIzZ8IIMAKMACPACDACjAAjwAgwAowAIxBp
CLDCHGkU4/IyAowAI8AIMAKMACPACDACjAAjwAhUCgKsMFcKzJwJI8AIMAKMACMQWgSWLl0a
2gyqWOp9+/atYiUKbXGYvqHFN9ypM33DTQHOnxHwjwArzP6x4S+MACPACDACjEDEIMACd8SQ
KqiCMn2Dgi1iIjF9I4ZUXNAoRIAV5igkOleZEWAEqjYC+/fvpwsXLmiFLCkpod/85jdVu8Bc
uiqBgBS41x1Zp5XntSGvVYlyVXQhZD2jdYbZ7fVm+rrbcyLS6HvkyBES43BMTAxdcsklF+3O
jh07RmfPnqXq1atTo0aNLhrejQHOnz9PRUVFWtUEZgK7SL9YYY50CnL5GQFGwDUI/Pzzz5SW
lkZSoJAV69ixIy1ZsoSuvvpq+YrvjIAXArLdsMLsBY0rXlX9SNYAAAObSURBVEj6ssLsCnJ6
VYLp6wVJ2F+cOHGCEhIS6Pvvv9fKsn37du25tIINHjyY3njjDbr11ltp/fr1muJcWng3fsvL
yyPZT23dupXat28f+dWE1SQqr8OHD5ccOnSoBJagMtUflhIt/NGjR8sUngMxAowAIxAIAhiQ
S5o3b16CUcXnX4MGDUp27NgRSJIcNsoQePvtt0vE35DXhmh/Za3+vn37SnJyckr69+9va4P3
339/ycaNG/0ms3nz5pLExESzvT799NMl/sbILVu2lDzzzDMl7777rt/0yvpB1rOs4d0Srjz1
hsdKCYxuNvreeOONJWvXrvULD9PXLzQh+cD0DQms5Ur01KlTJVCYzT4uLi6u5OTJk6Wm+fjj
j2vh77vvvhLMtJYa1q0fYfwxMYPC7IpqCjeDqLuOHz9ecs0112jETElJuWiDFgzTunVrLTws
RiXnzp2LOsy4wowAIxA6BEQf06FDB3OAGT16dElhYWGJUGTGjRtnvhf9VnFxcegKwilHNAJS
4A5UYRaCnT9DjXg/atQor3FSKFoyjjDmyN+ijToFyjNnzpjt+x//+Ee5MZb1LHdCEZZAsPWG
e2jJ0KFDTRpJWsn7Sy+95IUE09cLkpC/YPqGHOKAM3AqzIJnsrKySk1HKsxl0S9KTSiCP7LC
HMHEU4uuMkBZLECBhlfz4t+MACPACFwMgXXr1pnC7KBBg7yUk4yMDPP7ypUrL5Ycf49SBKTA
HajC/Mgjj2jta8KECdqM8v/+7/9qM4/SsCyExA8++MBEVcyaJCUlaXGEsiVmLw8ePGgalp1t
dMWKFVpYIUBWhMFZ1tMsUJT8CLbeCxYsMPuPKVOmlBQUFJTs3LmzRNJd0Ff1JGD6hqdBMX3D
g3tpuaryvzQwiTtcs/1GY4W5pIQVZr/NI7I+qAxQFgtQoOEjCw0uLSPACIQbATGjLAZhMVMn
XLOd1969e02BVwi5QkHhixFwIiAF7kAVZuEmLZYoOS+xdEl6V82cOdP8LLy0xHuxhEAoyvKS
ipkaVh0/K8o1T9ZT5hst92Dqrc7ujxkzxgaVUIx79Oih9S2PPfaY2a8wfW0wVdoD07fSoC5z
Rmr/pSrMpblml0VhFobDH3/8sUQYJ7/99ltt3BeeIP4u8U1+Fx48X331Vck333xjW1YqvqsG
SWEUy8/P18L98ssvXknv3r1b+y7CYP8Ur+/OF8K7TZRVprlnzx6/S3DcqDD/HwAAAP//guTa
mAAAMLtJREFU7Z0JvFVT+8dXhvJveMtMMr5es4yJEhlCevGiKN5IVEpFUUKh0iiKBlNJSFQa
36REJEqTSgPJHImXJpSK89+/1ftsz1l373POPffce/e557c/n3v3sNZee63vs87a61nrWc82
sRzctmzZEqtatWrMGBO77LLLYn/88UdCCvmNnzAxBpIACZCAIqDbl5o1a8a2b9+uQnce/vnn
n7HGjRvbNmu//faL/fzzz3ni8AIJjB49Ooa/5k81t3+ZIHL33Xfbejdo0CA/Oamzbl184okn
bNxnnnnGj4v84F3bsGHDpO9a/6YkB1LOJNFKXHA65f7iiy8sf8hg+fLleZhMmTLFhlepUiW2
adMmG0755sFUJBco3yLBnK+HyG8Bv5/OnTvH6tSp4/+eHn300cC02rRpY+OE6ReTJ0+Ooe1E
mvoP1yZMmJAnzdWrV9t4FSpUiI0bN87XX+TeOXPmxMaPH2/jHHHEEbGPPvooVrdu3bi0Effm
m2+2/YsffvghrhySTseOHWNbt27N8/zPP/88JmWSuHrftGnT2Lp16+LuGzNmjP/8+fPnx4Vl
64nJ1owXJN/6BxBWoXX6yeLrDu5vv/0WW7x4cWzp0qW20q5fv14nxWMSIAESiCOwcePG2HHH
HWdfLj169IgL0yfywkLHFvdwIwGXgHS4M6Uwb9u2LVatWjVbN4MUZrcuSidJ4v7yyy9+3V62
bJmb3bTPpZxpJ5ClN6ZT7kWLFln5uYMbguDTTz/1w9GRxiZ9HspXKBXNnvItGs75eYr8FqAg
Pv/887EVK1bY34sojEuWLMmTnLyrXf0CA9/t27ePu1/S0XsooFqvWLVqVdw90l+QexYuXBiT
tleuhe3RDgQp6xK/Q4cOceWBLiNhifa1atWKy7PODxXmOKTZdaJ/AG6FDipJovgy+o4K/t57
7wVWLIxC7dixIyhpXiMBEshxAitXrvTbDT0z52LRL6BPPvnEDeY5CdjZZXS6M6Uwi7KFjtLU
qVN9wloR1rOW8j4UhXnYsGG2bmNmA51FbLCOWLNmjf2Ta37CKR6ko1ikmHSko6VT7pEjR1oZ
uMqvFFR3xqVjS/kKnaLdU75FyzuVp+n+v7RrvXr18t/ZRx11VAwDi3oLU5iHDx/u34c2FfLG
JBt+b6+++mpcmDwL6erfKO7DTPPQoUNj999/fwzPh3Kt+weIg4FOTN5h0u65556LSxvhF198
sQ3/6aef4sJ1O4H2+YorrvDv7dmzZ+zrr7+2s9AbNmyITZo0yQ9DmtJ+IM86P/o6wrJ14wxz
AU2yW7VqFVdhUGmC/q655pq40ZdsrTDMNwmQQGYJ6Jehfkm6TymJLyC3jDwvGAHpcGdCYf7x
xx9jMO/D+wx7dOz0Jp1CdMzmzZsX051BKNHoUMn9n332mX3/derUKe79iM4ZBozyu0k583tf
tsdPp9zSbqAPErT8TCsE6GDLRvkKiaLbU75FxzrVJ+nfh7yfoeDKsk60j65ptvx29IQcBgql
PcQ9CxYsyJMFPZuLWeC1a9faOLqPgHsxoyybTMbJ7xzhUKLd9loGMxGOpV+uko+ZZYRBGYcJ
Njb93LD2A3UW9+FPK8Y6P/q65Dsb91SYC6gwyw9DKkyDBg18W36YaugfSKLZo2ysPMwzCZBA
wQnol5K8kINSLYkvoKBy8lr6BKTDXVCF+ddff/VNsfFu07PIkjuY77qmgYgrnccBAwbYTpSY
+D3wwAN+h6x3796x2rVr23MozbJ2VtJOtpdyJotX0sLTKbe0G2EdXq0QaDlTvkVfeyjfomee
7In696HfzxgklH4/9to0W/QCrTBrSzKEh23STiJNUTR1H0Fb6+g05HeO+15++WUdZI9ljTPC
33zzzYTh8lws/Zo+fXps8ODBsQ8//DDPPbigyyX34brOj76OsGzdqDBnUGGuV69enllk7XAD
yvPmzZuzta4w3yRAAoVAQL8M9QvZfVRJfAG5ZeR5wQhIh7sgCjOUZcxAoGOFv2nTpoVmCqaA
6HyNGjXKdtJkqQBmpzFDgtkKmPCh0ynK9dy5c216uCbro7W5d+jDVICUU13KicN0yo17IEd3
jaEA++abb3xZw3mQ3ihfTaPwjynfwmec3yeEKcxIRyu32jQ7SGHW729pA4PyAsVU2l60q9h0
HyFs4k2nry1F5Bk6XM9QB4WHKbgw3/74449jb7zxhlWimzdv7ucVedb36efp6/K8bNxTYc6g
whxWKbp06eJXqrA42Vh5mGcSIIGCE9AjtGEvQzxFjxDrmaCC54AplBQC0uFOV2GGoqtNDV9/
/fW00MgaP6yxwyZ1XK+Pw3XMRqOjJbPSuJbKJuVMJW5JipNOud9++23LGAMYQd71xemXNsVM
xozyTUYovXDKNz1uhXlXIoU5zDQ7mcKcSA/QyrEMoOtrYX2EZApquuFYxzx27NiEjsJEwdfl
Sva8wpRZYaVNhTlDCjNmj901AyK0iRMn+gpz0MiPxOOeBEgg9wjotU3oiIZt0kl1lY6w+Lye
ewSkw52OwgxrKL2E6N13300LIBx6oQMVtAZPmygi8ZdeesnGhcOb/GxSzvzcUxLiplNu6WzL
bL/LQZyVhinUbnzK1yWSuXPKN3MsM5VSIoUZz3BNszEDK74adHuXqgIpA1hoQ4MUZq2U6jIm
Sz+dcFiYaKdfohhjD6dhffv2tY7H5LrOW7Ln6bxnyzEV5gwpzGHrg1AR5IWFSjVixIhsqRvM
JwmQQBEQ0C/kMLNJOOuRNZ+pdmyLIOt8RMQISIc7vwozTASl04P6hRnhdDdxLqNnjfUMs16v
3L9/f/tcHTeV50o5U4lbkuKkU27t8Tqo/yHyQudeHAglYibxtcwo30TEUg+jfFNnVVQx9ftZ
FFj32do0+9RTT/WXtIQpzO7SB52eDGChPZaBRK1DaKVU35dMQU0n/LXXXvPfCxhwe+WVV6wj
MjCRDZ8LlHeHNvVO9jy5P5v2Oa8wN2zYMNBzpBai/sG4irGYXrjX9f36u235Xaul0+ExCZBA
ySQg7QhePPAo7G7fffed/1JKpc1y7+d5bhCQDnd+FGbtZwP1D50kzCJipkP/YeZEK7tBRFev
Xm3rqTuoA+cxMnstznHwXuUa5iCK4ddEvuExgkNEyUWnV6+fHDdunN+u4BuzyTbKNxmhgoVT
vgXjVxh36/5/mMLsmmaLAqkVZhlUQlizZs1Cs6r7AmKRWlwKs7QbyHOQozAUYsiQIX4bgk9d
yUaFWUhk+V6PuKbiiEt3KNxRWKncQW7aBZOuOHoERsK5JwESyG0CeiS3Y8eOcTCwhkg++YAX
F759yI0EgghIhzs/CrPujElHL2yfaGYE9VQ+sxi0zk46X1Da4HW1Tp06tqOVzhIDKWcQg5J8
Ld1y4/M0MmAB2UIGWsbov+hZoyCGlG8Qlcxeo3wzyzMTqaWiMOM5rmk2fl9aYdZLrxA2a9as
PNnTaaBdFJ8Duo0uyhlmac+RX/hCcDc4DMTgqLQlekBB6z1heXbTi/p5Ts4wo+G/+uqrfSEn
64DC9EgqhLvGUBRmhIuHUC10/ZJBHBkx0nF4TAIkkNsE9CAe2gk4S8KsMrxSduvWzW9/0OlN
NsuX2yRzu/TS4S4shVlmh4MoQylD5ymsjsL7NmZW5F2KPeLDAiu/m5Qzv/dle/yClBvykUEK
LYPWrVsnVZbBjfIt/NpD+RY+4/w+IVWFGelq02z8xrTCjPAXXnghrv0bNmxYDI4W169fH/cZ
JtyrBx2LS2EeOXKkn1+sWYb5NdY1b9iwwToC0+2Im2cqzJB4CdmGDx/uV4REL+3Zs2f78VAh
3BF2rTA3bdo0zxogPWIUtj6xhCBlMUiABApAQJtsuS8inGNWiN6xCwA4B26VDnd+FOaixoKB
IJh6f/nll0mXQ4XlTcoZFl5Sr2ei3PjMF/jDxB5KcKY3yjd9opRv+uwK604ozPJJPD2DGvQ8
d+DbVZgxgda5c+c4nSLoXQ+9QvsTKC6FGd9i1zPIQXnV1/CNaPhbwUaFOaiGZOk11zwCQsdI
K9YYQ0nGSJA42ZEKgXXKuhKj6FphRrwmTZrYjgBmhrBgX+7FfsaMGVlKi9kmARIoCgJfeJ6K
9Wd9pP0488wzYwjjRgKJCEiHO8oKc6L8pxom5Uw1fkmJlyvlzpVyuvUyV8qdTeXUM8x6ja4r
OznXk2Rh/kZgji1KuLzjsYdPh5kzZ0pS/h7vfokXtqxzypQpfpyggXUdjplidwsLh9l1gwYN
/LQlH1CkMQP9+++/xxo3bmzDYV0kXwvS6SWyTHLzEeXzUsicByAnN29Gx3iVNqWyexXBLFiw
wOy5555x8du2bWsGDhwYdy3oxDPrNu3atQsK4jUSIAESiCPgvSBNqVKl7B8CDj300LhwnpBA
EAFvVN9enrF+ht0/1fypoGhZf03K6XXksr4s+SlArpQ7V8rpyj5Xyp0r5XTl6557Fh7Gm3W2
f6VLlzb777+/GyUy557puPGW1RjkE2pj5cqV/f5JZDJZyBnJaYUZbFEJnn32WeN9Ny0QtTeK
Yvr06WMaNWpkypQpkyeOKMyI55lr2HQ+//xzPx6uv/jii8ZbO+Rf4wEJkAAJkAAJZJqAdESp
MGeabDTSE/mW9IGCXCmnW6typdy5Uk5XvjzPbgI5rzCL+Dy7e+OZURvP/MLssccexluLYCpV
qmT23ntviRK41wozZoUw+uKtETLlypWzaR1yyCFml112CbyXF0mABEiABEggUwSkI5qp9KKe
TklXHF3+lK9LpGSdU74lS54sTckiQIW5gPLUCrPnRCOPyXYBk+ftJEACJEACJJASAXa4U8KU
tZEo36wVXUoZp3xTwsRIJFAsBKgwFxA7FeYCAuTtJEACJEACJEACJEACJEACJBBRAlSYCygY
KswFBMjbSYAESIAESIAESIAESIAESCCiBKgwF1AwLVq0ME8//bSBcy+aZBcQJm8nARIgARIg
ARIgARIgARIggQgRoMJcQGFs27bNOvdCMhUrVixgarydBEiABEiABEiABEiABEiABEggKgSo
MEdFEswHCZAACZAACZAACZAACZAACZBApAhQYY6UOJgZEiABEiABEiABEiABEiABEiCBqBCg
whwVSTAfJEACJEACJEACJEACJEACJEACkSJAhTlS4mBmSIAESIAESIAESIAESIAESIAEokKA
CnNUJMF8kAAJkAAJkAAJkAAJkAAJkAAJRIoAFeZIiYOZIQESIAESIAESIAESIAESIAESiAoB
KsxRkQTzQQIkQAIkQAIkQAIkQAIkQAIkECkCVJgjJQ5mhgRIgARIgARIgARIgARIgARIICoE
qDBHRRLMBwmQAAmQAAmQAAmQAAmQAAmQQKQIUGGOlDiYGRIgARIgARIgARIgARIgARIggagQ
KPXTzxtiUckM80ECJEACJEACJEACJEACJEACJEACUSFAhTkqkmA+SIAESIAESIAESIAESIAE
SIAEIkWg1OYt2znDHCmRMDMkQAIkQAIkQAIkQAIkQAIkQAJRIECFOQpSYB5IgARIgARIgARI
gARIgARIgAQiR4AKc+REwgyRAAmQAAmQAAmQAAmQAAmQAAlEgQAV5ihIgXkgARIgARIgARIg
ARIgARIgARKIHAEqzJETCTNEAiRAAiRAAiRAAiRAAiRAAiQQBQJUmKMgBeaBBEiABEiABEiA
BEiABEiABEggcgSoMEdOJMwQCZAACZAACZAACZAACZAACZBAFAhQYY6CFJgHEiABEiABEiAB
EiABEiABEiCByBGgwhw5kTBDJEACJEACJEACJEACJEACJEACUSBAhTkKUmAeSIAESIAESIAE
SIAESIAESIAEIkeACnPkRMIMkQAJkAAJkAAJkAAJkAAJkAAJRIEAFeYoSIF5IAESIAESIAES
IAESIAESIAESiBwBKsyREwkzRAIkQAIkQAL5JzB18vj835TFd9S97Moszn3+s0755p9ZNt1B
+WaTtJjXXCNAhTnXJM7ykgAJkAAJlEgC7HCXSLH6haJ8fRQl8oDyLZFiZaFKCIGcVZg3bFhv
YrGYKVWqlKlUac+k4vxl82azfcd2s+suu5q/VayYNH5xRpCyJcoDylG6TBmzxx57JIrGsAwS
2Lxpk9nxxw5TevfSplz58hlMmUmVVALbtm0z33+/1pT36stee+1dUovJcmWIgHS4S/rMa66U
060WuVLuXCkn5Zt9FiLox636dJX5cd06Kz7oEPsfeKA5/PDDTcWKlVyR+ufS//MvOAfok++2
226mbLlyTshfp3/88YfZtGmjfyE/+ojWC8qULpPwOf4DeBBHICcV5t9+/dVUr3aK+fKLLyyM
OfMWmRNOPDEOjHtyW8vm5vnnhpsaNc82r02bYXbddVc3SiTO3bIly1T5ChXM/Q90M01vaWbK
eAo0t8IhsHXrVlOrxhnm45UrTd16/zQvj37V7LLLLoXzMKZaYgg88nAf8+D9nW27M3X6m6wz
JUayhVOQXFE0cqWcbi3JlXLnSjkp3+xRmH/77Tfz9JNDTJf77nHF5p+3u7ODaXtHO7PPPvv6
13Cg+39xASEnLW9rbVrd1tYc5inhepsw/lXT+LqG/qUj//EPM3/RUqto+xcDDlZ98ok57eQT
/JAevfuatre38895kBqBnFSYUXnPO6eGWfbRR5YSKt37Hyw0//d//xdKrcOdd5gnhww2V13d
wAx//sXIdlzdsoUWyAnAD3OuN3DAmU8HTIZOMZBxbq2zfIV51CtjIzvokqEiM5kCEpj51pvm
8nqX2FSi3u4UsKi8PUMECqJowOLq1bGjTcc725sff/zB5ujkk08x3Xr0Muedf0FgDhfMn2fu
ubuDmTvnfRuODuNdHe4OtMJauGC+mTxpojnrrBrm4rqXBqaX6sWClDPVZ0QxXkHKTflGUaLx
eaJ843lE5ey///3R1L3oAtt/SyVP7iRcuv3yKa+/Yc45t7b/SFdhRsDij1aavx95pB8n6GDI
oIHm7g7t/aBH+j9umt/a0j/nQWoEqDD/j1PPPg+bNm3vCKUmCnPUZwf1D/Ogg6qYES++lEcx
22333c26tWvNCG/GfPKkCX6ZH+ja3dzVsZN/zoPMEfj999/NP+teZDuWNzS5yQwc/GRkB10y
V2qmlC4B/C6vu7aBf3vU2x0/ozwoVgLpdrh37Nhh7mp/uxn2zNOB+e/Vt59p3eb2uDA9oANL
JSxbwobB13kLl8QNQG/fvt3UOe8cs3DhAvOfqdPNubXPi0srvyfpljO/z4la/HTLTflGTZLB
+aF8g7kU51UMNLVs0cyMfGGEn42+/fqbWueca/baey/z6y+/mlnvvG3uaHubH+5OQOl+OSJh
hrdy5cp+fBz8svkX8+mqT8zjj/X3r++7735m0dJl/rLRIIV50JCnzI03NfXvcQ902ythVJiF
RP72VJgVL3dUSAWZbFSY6ze41jw74gW7TluXRR+/PGqkada0ib3k/jh1PB6TAAkUPgGsM+r5
UDfzxOBBcQ+jwhyHgychBNLtcI984Xlza/ObbaoPdnvIXNOwkdm6ZasZ0L+fXYqEgGkzZtql
ATj+888/7QDgu7PeMVCmb2vd1vz003/9WZjRY8fbpSeIi+21/0w21za4yl7LhHVNuuXcmZvs
/Z9uuSnf7JA55Rs9Oa1b97058rCDbcagCM+YOcvsv/8BeTKKeJgUwbI7bOMm/sfUuehie6wV
ZkxkLVq63JQtW9aGuf+++vJLzwK2pm/lA4tW9OWxuQozLID28CxjsVwL65+DthXLl5vqp58c
F0SFOQ5HyidUmBWqRKbZqSjMWJC/du13Ztvv26ySijWqB1WpElqRMeqLTSo6RujXejO/6IzA
PLzyQQf5YSqbCQ/1DzOVTrbu+CRSmFG2b77+2vzqmRbv7s1Ql69Q3hx4YOWEyjgyCgVg7Xdr
7WwqnCPss+8+oc6LXB5r1nxjNm7Y6eCggjeDcfAhhyR9HtaZfPftt6Z06dIGs7rlypfzRvIO
CmWGZyJfsiYdpjc///SzjZ9KGfG8b9esMRjFgxwT3QOGGK0UeYdmKp8Bwhhpw4nbPvvsE2gS
mc9kGb2ICcx+d5ZVONzH4nd5+hlncN27C4bneQik0+HWMxB3tL/LdPdMsGXD++Ga+leaaVNf
My1atjIPPzLAtpeyxOSn//5k5i1a7K/ZE8VMd8j0O+md9+aaU089TZJPe59OOdN+WIRuTKfc
lG+EBJgkK5RvEkDFEKzX/yabzdVWN9oqR7eBifrZUrzRr4wyNze5wZ7qttRVmCV+IrPsAY/2
y7PuWqcpaXCfnEDOK8wdO91rsA7rrTdnWFq6kmt8yRTm11+bYlrd2twfFZJ78eMYOPgJU++y
y+WS3X/91Vfm+GN2rjt4653ZZtHChdYkLi6SdzL4iadN4xubJFUU5T79w0xFYYaSBScCEyeM
M2E/5Pffm21aNGvqO0mTZyF+74f7mQbXNMyTv2+/XWO6d30wzoxF7ruwzkWmR6++5rjjj5dL
RvPATMYb06eZfn17++E4OO20081TQ4ebo485Ju46TqAcDx821FoCuIG47zFPBiedFD/KJrMe
GDV8dfxk89iAR/zZFEkDZXx+5Chzdq1z5JLdJ3IAgXueeOqZuHV6Wi4YLRz23PNm0sTxvgMH
d0ZGHgb5tG7VwubLlQ/y0KdXD/Nov74S3d+jznS6t7M55JBD/Ws8iDaB8ePGmhuub2QzWe2M
6gYv51YtbrFmrKn8lqNdOuauKAik0+HWbS8cyBxz7LFxWZ32+lRT/8rLDWZGFny41BsUrGCd
2MAPyLrv18WZDMKkG6aJumMp9VravUw4O0ynnHGFytKTdMpN+WaPsCnf6MlKK8zugKKbW/R7
293exhxx+BG2/ye+H3T/z+3HuWng/JOPPzann7LTEbFWbkVhRhvc+f4HzSN9+1idQ7e3Oj39
XPQH35v9roFVkE5Tx+dxYgI5rzA/PWy4N+J9ul85gSvINDtMYYZCc1+njmbg4wMSkoYC8/ig
J/zZxc9WrzYnnxjfMQlLID+VW/9AUnEUpF+mQT/kZ5560rS/o01Y1ux1eNt7qFcfX2nWJiyJ
btSds/zwmP7m2+asGjX9pDHDCocM4sTND3AOhjz5jB18kMvS+Mh5ov3MWe+Z06udYaPoWflE
92hTGpmRES/ZMEuE6c1JJ+xU/sNkpVkiDkzsMRuOTw3VOf/cPIMYOj9oVN+e9X7gAIOOx+No
EEB9bH97W0/ZeNJccmk9A4uEGtVP8x3F0bN6NOQU5Vyk0+FesmSxOfvMaqEDpp9/9pltp/B+
kNlkec+s/3m956V1ianwt79ZLNKmyjtLt3tY13zsccdlBF865czIg4s5kXTKTfkWs9Dy8XjK
Nx+wiigqrA7POPVkfzIMSjOsbapU2WmmnUo2pL1EHzWon+2mIZM5uC5tKY6lfcUxJtoe8SZL
pkyeZM70HCkGmWXLbx99QXzdp12b2+wAvE4TaXFLjUDOK8xScTBL90CX+yw1mGbj5Q7TY9nC
FGYxQZN4mI28pG49E/NM2WbMmG6ub3iNBMVVfFdBRIUe8PggU/u88+1s6SBPAZd1jJgBneN5
8UacZJv+YWJWKmi9GDriv3uewpct+8jcfNMNvtLlxv9g7hxzoeeoRTYonFdedbU1r5437wPT
9MbGfiOilcOhTz/ljbK1trdhPdz1jW8we+65l1m//mfz3LPDTI/uXW0YnF9hZAwm0S4PRBg7
bqK56JK6ZsuWLQY8und9wN6HmQ7ppGHAoknj6824V8fYMDRGY8ZNMFW92eRfftlsZ2Y73/uX
IzNtEqgbH9yMdJ94eqg56eST7cxJ0yb/9pVwnVc4rql99ln2eVdeVd90eaCrNRff5s1yT5s2
1XKR9NwZGTSYMluIcsvsPmSLMrmNsMzOID1ZE+Mq7JiNhCXCP446yirSGOSQWWfX+YTNNP9F
kgB+l5h9Q73A5v6WqTBHUmyRylQ6HW4x/9Ptqi6Ubpul/dSKsB74vL/zvab/Iw/777oXRjzn
WV41M7r9xAAnrGOwpbKsR+dFjtMpp9ybzft0yk35Zo/EKd9oyko+76hzd/4FF9q+/unVqlkv
1XvttbcOjjvW7/KwdlZucD1yY1Kv0XX/tsG6zwodZenSxeaWm260YR8uXWGgu+itd8+HbH+7
hvc53PHemuoLzqtl+7Si9+i4PE5OgArz/9yrowMglQnYXNPsIIUZL/5aNar7Cues9z8wp5xy
ahz15cuWmTO9bz5jgzI3Z/5C6zBAd0IQ5t6rlSI9so+4iTb9w0wUzw2DwgalHAoWNiiija6t
b0evcI7RLChmesPs51nVTrNKswwyYH2umBAjrYWLl9n1xHIfynW+59AASicaDlEoXR5BnlQf
6vagNUFGWqKgywgaruF578yek2eNNJTpG/99HaJYZfWll8fYmX7d+CAvcxcs8r0RIi6c2FQ7
5SRbPnwGBQoLyicdEMRZuvwTc/gRR+DQ3zDwIgqrWCtouYjCDOVIzB1xs24Yca7rgM6fVtgx
6wzzbnddtB4ACjP3xjO4RZdAWJ2Jbo6Zs+ImkE6HW9rBMCsXXQ/x+cUTq1a1xZR3Ipa8PPLY
QPPxihW+4zAo0fACW/Osavb9+NGKVabKwQfbQU9pG5EI2rVJU143Rx19dL7QpVPOfD0gopHT
KTflG1FhBmSL8g2AEoFLO3bsML16dDd9e/cMzQ0ccN3aqrVnil3X9+kgkXUbimuwkjzggANt
P1viwD8Q/Jh0vOuv7yO7s9HyW8Y9aIsrVqzoL+10lWA9qIm+5TXXNrK6CKwc3biSB+4TE6DC
rL5HphURYBNlB8fSOdDKjl7bcGur26xDFMR1N3i9xY8Nm4zQawUxbG2X/nHIfW7a7rn7w3TD
g85hLt61ew+r0Eu4zh86Us+9MNKf+ZI42GvlURwP3Nayub8WuM/Dj5qr6te3jYPcB3Ni6zis
fAVTpkwZe1k/D4yDZsaxPuSYI3cq9BIH3w0V5wgvvTLGXHb5v+Qx/l47PdENkOaLGVrMgrgb
zNExY6vv02WG3Ju3aGVH9mRmEM7bNm3eZBV3OOHCpuWi69CmjRv9TmUNbxQQ390T5VdMIXE/
1p/c12XnDLvURVzXHVicy4bn1apxhjXn1c56JJz76BMIqzPRzzlzWFwECrvDrWeT3ZkQKbMM
Nsu3P29vd6d5qGdv6/0d70EMzna8+x7rpwLr6aA0y8CppJFsn045k6WZDeHplFvec6kMiFC+
xVsLKN/i5Z/s6VA2R774vIEjrUSb2xfV7/JE97lhEya/Zi64sI5/WX7LuACdAEr6xReeZz9X
6ppliz6D9nbuvEVm/wMO8Lxv1+AMs08z/wdUmJXCDHxauZVZUyh3oqRoZUdXXr3G1RXD0qVL
TM3qp9vLMjOqFcSwBfs6/XQVZihLu+2609386s9WW2+nOn+i5OprONbKGpTFXn0fNlt+2xIX
DR6oMVsga4clj3r9hdyAH3bD6/9tzjmntl3HJkqhhGsewkjCZI9G5xKvcUBDIC9/fIz9ySGD
4xRaia/32nReOgWa77tz5tnGR9+DY5G7Vpix9viEY+NNX9AoNW/R0lzsmZBXrXpSHvN53WDq
OoRniBkjjrVZjTZt14qx5Anx8e3wE7wZH3hmlw2K+2bPHP2ejnfZSzK4IJ7AJR730SaQqM5E
O+fMXXERSKfDLcs+3AE7KYMeqHQtjTDzAkcyP/74gx1QPemkU+zgoVjnbNm6xS412Wefff0B
PHlX6vZclpvIM5Pt0ylnsjSzITydclO+2SDZnXmkfLNDVlg+ha/GfPLxSrsMD5Mq7jby5dHm
8iuutJf1u9yNF3SOSawOHe/JY72o+6zS35ZlL0hH9x+lXyn9P0wcUWEOop36NSrMjsIcZpot
SopWdoIqbxB6rQyKKUTQNffeVNPX9+kfJmau3e8wQ9mrf9Xl/rfiwsyYtXt8nX6iY/kBw5wb
Aw9YPxG2YcYUzsLEWYzmoZVD936RA/INE/KuD3ZJSWEOYhl0Lex5WmFGnP9MnmgaXVPfje6f
Q6HH+m0x19Zy0XUIN2iz8n6PPmYdSsDr98UX1LaDA3rgBo3epRdfaEcU/YclOZAGkwpzElAR
C05UZyKWVWYnIgTS6XDL58zcNk6KJIOnMlNx6GGHSVDoXpaEiGWMWGNhNln8T+DmQQMfswN7
MisdmqATkE45nSSy8jSdclO+2SNqyjd7ZKVzCp8M8z6YaydY5DvM6KNinTE+Eavf5bgPyjS+
5Yy+smwwyYaJ9RF//3ueZYUSJ6jP+sXnn5uqx+9c0iL6hTbHhg6AL9noPEg8SZf71AhQYXYU
ZmATUwZBuGjJcvPiCyPsbKpWdoIqr9yj99LhwDWpqFpBlGv6Hhynmr6+T/8owhQljP7XrlXD
X3tdwzEFRnpaIURHqt5ll5lNGzfpR8UdYybh4X79je5MoYwow+CBj/vOwfRNUARh9g6zZc0j
TGFG43Jnu7bWRBrrqeH1795OHfKYTOtnyHEQy6BrEl/2oqAHdSZhkjh1yhQz/NmhZr7nBM3d
0MGUdeGuXLQDJ20yLsrxypUrfKsEbYGg08HzrvjXVf43pN3ny3n1M88yrVon9nQucbmPDgEt
a93uRCeHzEnUCKTT4Za2F+1VkONBcf4Y1AYGlf+77741R//9sECfHW49HjP6ZeskUc/GBKXp
XkunnG4a2XieTrkp3+yRNOUbLVnBguaVUS8ZtGmlS5c2be9oH7g0UXKtJ9zQns5buNgcfPAh
ccoqBg0XLV1uypYtK7elvA/qs2pfN2KWvXDBfOuwV7fpuj8RpnOknJEcjUiFOUBhRl3Qptkw
J97DGyWaO+d938MxHDbpyuuaqun6JB0OXJOOgbzEcC2s8ur0ZfYW8RNt+kfhdk70fXpWE9fd
PGgl33VGpdNJ9Xjt2u/MSs8pDL49jG91yibl0jzCzPNQNlmXG2SSLZ88kbT1Hh7H4UwBDcjs
9+dZr4ap8E2kMOv04QDu01Wr7Lo8Wa+OcOGaTC7aZPyDBYvNdM/bdpf77rH5dTuxkic0vB9+
tMKOYOq88LhkEEhWZ0pGKVmKTBJIp8OtZyOeGvqsue76xnFZck37klmqSHw9a6xnmPV65cGD
HjedOtyZx8lmXAYCTtIpZ0AyWXcpnXJTvtkjZso3WrLCb6e657T3yy++sE5lZcY4US71UjrR
C/S7PNWBx6BnhPVZdf8RjmhHvfSi9ZukJ810HqRfGvQMXgsnQIU5RGHWI0Uan1ZCpROA8CZN
bzYDB+ddx4AwUXBwLLOnWkEMq7xhPw6kE7bpH4XOa1B8MZuTML2eWedP/+gkruyxfuKVl18y
xx1/vLm1ZWtTsVJF+zmsxYs/NDW8byW3v6ujRPX3iC+u8MXZln6e/gSJf5N3oHmLo7SxY17x
nX7BMdnV9f/6jJfcq5mgsVqybKU1BU+Fr8hOGrnynqMyrFdZMH+e2bhpo+85W56FvfaMLvnc
tm2bv34kSC4yK4P7sX4FznDQSMv9GKCRTRyR4VwaZAmTPRR4fKf5jOpn2u+MN72lWcKRUbmP
++gQ0PU2qM5EJ6fMSVQIpNPhRt5FycWA4mTPa7V8c37ypAnmumsb2OKlMnAq5oHSXlaqtKe9
d/OmTabGmafbNk2caaJ+i0+KsEFSe3PAv3TLGZBUVl1Kt9yUb3aImfKNlpxg1ShffUHOZMIr
LJeI3+KWplZhDZthdtvGsLSCrof1Wb/+6ivfWzaWOk6f/rpddvniqFesFSLS0v2JMJ0j6Jm8
9hcBKswhCjMQuabZuKY7ru5npabNmGlqeObNetNpYEZQPl2kFcSwyhv249Dpu8f6R6Hz6sbD
uR55xjniyyeXYIpS96IL/LWyI158yTraQjzZ9GelpHHY2/sWnYzIobx6vZrcp8slP2jNA/GC
PrN1a7ObbUOEcPnslOaLhujdOR9Yr6uIIxtmtO9oe5s91QqozofMdMs9sncV5ooVK5mWLZqZ
kZ6JPjYZAJH42OvZefFQjTXJ4nAhSC5oaHW6kl5QR1LWpCGOrOUGf73J2kBcEy+1OpzH0SeQ
n99y9EvDHBYFgXQ73GjLLzzvHH+ZDtoTePuXDaZ+k1+bZpfPyDV3r5fM6GUkEk8rbd2697TL
ft56c4Ztr4PeE3Jf0D7dcgallU3X0i035ZsdUqZ8oycnvTwRuUNf+F9XXm30JAauo8/87NBn
7LJBnKNvhuWccBqs3+WFoTCLWTacL8oaajxHPmOL/Og8hOkciMctnAAV5gQKM7Bp02ycu8rO
y6NGmmZNmyDIbkOefMZc+s9/2k8DwXFW4+saSpDRnQitIIZV3lQUOj/x/x3oH4WbVzcuzmFm
jplI2bSHaq2YIRxOrK5vfIMpV7ac98H0Jfabm5gFxaaVMukY4bp8o/OYo48x27ZvM2/PfMvc
cH0jBFlz48WeSTGcH2geNtD7N3bcRFOtenW7drrrA10MZpOxYZ0vPF3D07araKKRQIN2wokn
2gZs+LCh9tuf9kbvn1bEU+HrKsyYMdHfTsbzHh88xNQ651yzS6ldLBco59JoiWKfilxcR2sY
cNDmi1IGdzADeRg1eqw54YQT7eesRgx/1n6sXuIvW/lp3Npyuc59tAmkUmeiXQLmrqgJpNvh
Rj6hVDW/+SYDJVZvGPR7qGefhMqy3H9WtdMMvp4A3w3uIB4c4+CrBs89O8xPHm3X1OlvmqOP
Oca/lspBQcqZSvpRjVOQclO+UZXqX/mifP9iEZUjUUZh9Scb2ra7O93r9TOrWsUZ/b1+fXvH
+eoZ8+oEc8ml9ewt+l2ONm/R0mVGrG8kzVT2ifqs2iwbaenJIZzrPITpHIjHLZxAzirMshY2
WcUJmoXVDpugsD3U7cGEHzQHfnyvt3ffR3wHTVpBDMtDoh9HmEjxo5CyQWEO+p6xe68ohbiO
H/Os9+eaKlUOttFgfgwT4EQbFLv3Pphv9t57HxsNzrDOO6emP1sRdi++/Swm25pHWHxcR/7c
WeSNGzeYK+rVtRYBie51TbZT4Svmz7qRg8zF7CbR82ooZ2q6HoXJRcdBuuJhNugZ+NTLJXXO
T8pYTN6D0uC1aBPQ9SGszkS7BMxdURMoSIdb8rpmzTf2M3X4dMrfKv7NDmhKWCb233+/1rNu
+s3stvtu9j3jztSk8oxMlDOV50QtTibKTflGTap/5Yfy/YtFlI7Qx2x6Y2PPt8zrKWULE2dY
WiebfpfrvqSEp7pP1GfVZtlIb/TY8XaCT9LWukGYziFxuQ8mkLMKs5jHpqJQaLNfd9RGsL7/
3mxze5tW/syiXMcMa/eeve0MpFzDXn/bMsjRCuLomUw4Gzj2uONwOeGmR5GwFhjrqpN1SFyv
2dpZCx6G9botW9ySp2wI6/Pwo3b9tuvxD+bqcLSFGXp3A5MevfuammfX8oO0wnz3PffZ2eMe
3bv64TjAoEOX+7t6nbiKcddxgllXrI2G2ba7wUFY5/sfNP846qi4IDR+V//rMntN1tXFRfBO
ZLbcNX22z/O8J8IxF8xg9IbRx/6PDbSu/MVBjv5MVCK5DHi0n00T6SWTOWZsnn5yiB9f5+HC
OheZLg9289Yvn6Yv8ziLCOC3LGs8E9WZLCoSs1rIBDLR4S7kLGYk+VwppwsrV8qdK+WkfHd+
p9jlENXzd96eaSfIYJkZtMEap3WbO6w5tg7X/T/0Jed6X4cpV768jpLSsdYJ3D4rJnIaXVvf
TJk8yU4szf9wiT+JhcS1bpCK3pNShnIsUk4qzIUpY4yew4QDf3BDv99++xfm44o0bZkZKF+h
vNm6ZavZ/4ADTJkyZRLmAUodZpxhPg0lE/Fhgu1uWmEW03Xch4YGMx3lvcZlL299dLINcTEY
gU9VoYHAd+2w7riwNjzvhx/Wme3btpvdS+9u8ImoypUPsuUtrGe66YLxuu+/N2XLlbUzN3C8
lgorNx2ekwAJZDeBXFE0cqWcbm3MlXLnSjkp3+xSmEVemBDa5DkyxGbXKHv94f322y8tJVjS
5D76BKgwR19GOZFDrTDTXCQnRM5CkgAJZJhArigauVJOt3rkSrlzpZyUb3YqzK7ceJ4bBKgw
54acI19KKsyRFxEzSAIkEHEComhEPJsZy17dy3Krw035ZqzqRDIhyjeSYmGmSMASoMLMihAJ
AlSYIyEGZoIESCCLCbDDncXCSyHrlG8KkLI4CuWbxcJj1ks8ASrMJV7E2VFAKszZISfmkgRI
gARIgARIgARIgARyiQAV5lySdsTLutlzogBPf2XLlStSp1kRx8LskQAJkAAJkAAJkAAJkAAJ
FBMBKszFBJ6PJQESIAESIAESIAESIAESIAESiDYBKszRlg9zRwIkQAIkQAIkQAIkQAIkQAIk
UEwEqDAXE3g+lgRIgARIgARIgARIgARIgARIINoEqDBHWz7MHQmQAAmQAAmQAAmQAAmQAAmQ
QDERoMJcTOD5WBIgARIgARIgARIgARIgARIggWgToMIcbfkwdyRAAiRAAiRAAiRAAiRAAiRA
AsVEgApzMYHnY0mABEiABEiABEiABEiABEiABKJNgApztOXD3JEACZAACZAACZAACZAACZAA
CRQTASrMxQSejyUBEiABEiABEiABEiABEiABEog2ASrM0ZYPc0cCJEACJEACJEACJEACJEAC
JFBMBP4fx/BRetGKoRkAAAAASUVORK5CYII=
--------------E141655F6BC3F6F8ABEBA096--
--------------5323FA587AD2C243D1E46CA3--
5:28 p.m.
On Mon, Apr 17, 2017 at 9:12 AM, FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Should I understand then that fecing is not mandatory, just advised
and
there are no downtimes to the rest of the cluster if a pending manual
action has to be done to confirm a host has been rebooted in order to move
the SPM to another live host. Perhaps only effect is not be able to add new
hosts and connect to storage or something ?
A missing SPM will not affect existing VMs. Running VMs (on hosts other
than the SPM host) will continue to run normally and you can even start
existing VMs. Without an SPM you will not be able to manipulate the
storage (Add/remove domains, add/remove disks, migrate disks, etc).
Fernando
On 17/04/2017 10:06, Nir Soffer wrote:
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
> But actually, it didn't work well. After main SPM host went down I see
> this
> [image: Screen Shot 2017-04-16 at 10.22.00 PM.png]
>
> 2017-04-17 05:23:15,554Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
> Init: could not find reported vds or not up - pool: 'STG' vds_spm_id:
'1'
> 2017-04-17 05:23:15,567Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
> selection - vds seems as spm 'tank5'
> 2017-04-17 05:23:15,567Z WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds
> is non responsive, stopping spm selection.
>
> So that means only if BMC is up it's possible to automatically switch
> SPM host?
>
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
>
> Thanks
>
> On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Oh, fence agent works fine if I select ilo4,
>> Thank you for your help!
>>
>> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
>>> konrasko(a)gmail.com> wrote:
>>>
>>>> Makes sense.
>>>> I was trying to set it up, but doesn't work with our staging
hardware.
>>>> We have old ilo100, I'll try again.
>>>> Thanks!
>>>>
>>>>
>>> It is absolutely necessary for any HA to work properly. There's of
>>> course the "confirm host has been shutdown" option, which serves as
an
>>> override for the fence command, but it's manual
>>>
>>>
>>>> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>
>>>>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>>>>> konrasko(a)gmail.com> wrote:
>>>>>
>>>>>> Fence agent under each node?
>>>>>>
>>>>>
>>>>> When you configure a host, there's the power management tab,
where
>>>>> you need to enter the bmc details for the host. If you don't have
fencing
>>>>> enabled, how do you expect the system to make sure a host running a
service
>>>>> is actually down (and it is safe to start HA services elsewhere), and
not,
>>>>> for example, just unreachable by the engine? How do you avoid a
splitbraid
>>>>> -> SBA ?
>>>>>
>>>>>
>>>>>>
>>>>>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> "Corner cases"?
>>>>>>>> I tried to simulate crash of SPM server and ovirt kept
trying to
>>>>>>>> reistablished connection to the failed node.
>>>>>>>>
>>>>>>>
>>>>>>> Did you configure fencing?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Apr 16, 2017 7:01 AM, "Nir
Soffer" <nsoffer(a)redhat.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> When you set up a storage domain, you
need to specify a host
>>>>>>>>>>>> to perform the initial storage
operations, but once the SD is defined, it's
>>>>>>>>>>>> details are in the engine database, and
all the hosts get connected to it
>>>>>>>>>>>> directly. If the first host you used to
define the SD goes down, all other
>>>>>>>>>>>> hosts will still remain connected and
work. SPM is an HA service, and if
>>>>>>>>>>>> the current SPM host goes down, SPM gets
started on another host in the DC.
>>>>>>>>>>>> In short, unless your actual NFS
exporting host goes down, there is no
>>>>>>>>>>>> outage.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> There is no storage outage, but if you
shutdown the spm host,
>>>>>>>>>>> the spm host
>>>>>>>>>>> will not move to a new host until the spm
host is online again,
>>>>>>>>>>> or you confirm
>>>>>>>>>>> manually that the spm host was rebooted.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In a properly configured setup the SBA should
take care of
>>>>>>>>>>> that. That's the whole point of HA
services
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> In some cases like power loss or hardware
failure, there is no
>>>>>>>>>> way to start
>>>>>>>>>> the spm host, and the system cannot recover
automatically.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> There are always corner cases, no doubt. But in a
normal
>>>>>>>>> situation. where an SPM host goes down because of a
hardware failure, it
>>>>>>>>> gets fenced, other hosts contend for SPM and start
it. No surprises there.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Nir
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Nir
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM,
Konstantin Raskoshnyi <
>>>>>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Fernando,
>>>>>>>>>>>>> I see each host has direct connection
nfs mount, but yes, if
>>>>>>>>>>>>> main host to which I connected nfs
storage going down the storage becomes
>>>>>>>>>>>>> unavailable and all vms are down
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM
FERNANDO FREDIANI <
>>>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That doesn`t make much sense make
a whole cluster depend on
>>>>>>>>>>>>>> a single host. From what I know
any host talk directly to NFS Storage Array
>>>>>>>>>>>>>> or whatever other Shared Storage
you have.
>>>>>>>>>>>>>> Have you tested that host going
down if that affects the
>>>>>>>>>>>>>> other with the NFS mounted
directlly in a NFS Storage array ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-04-15 12:42 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In ovirt you have to attach
storage through specific host.
>>>>>>>>>>>>>>> If host goes down storage is
not available.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31
AM FERNANDO FREDIANI <
>>>>>>>>>>>>>>> fernando.frediani(a)upx.com>
wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Well, make it not go
through host1 and dedicate a storage
>>>>>>>>>>>>>>>> server for running NFS
and make both hosts connect to it.
>>>>>>>>>>>>>>>> In my view NFS is much
easier to manage than any other
>>>>>>>>>>>>>>>> type of storage,
specially FC and iSCSI and performance is pretty much the
>>>>>>>>>>>>>>>> same, so you won`t get
better results other than management going to other
>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>>>> I have one nfs
storage,
>>>>>>>>>>>>>>>>> it's connected
through host1.
>>>>>>>>>>>>>>>>> host2 also has access
to it, I can easily migrate
>>>>>>>>>>>>>>>>> vms between them.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The question is - if
host1 is down - all infrastructure
>>>>>>>>>>>>>>>>> is down, since all
traffic goes through host1,
>>>>>>>>>>>>>>>>> is there any way in
oVirt to use redundant storage?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Adam Litke
5:31 p.m.
This is a multi-part message in MIME format.
--------------52C86E47EED6E2100703EC63
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
Thanks for the clarification Adam.
That makes total sense.
Reagrds
Fernando
On 17/04/2017 11:28, Adam Litke wrote:
On Mon, Apr 17, 2017 at 9:12 AM, FERNANDO FREDIANI
<fernando.frediani(a)upx.com <mailto:fernando.frediani@upx.com>> wrote:
Should I understand then that fecing is not mandatory, just
advised and there are no downtimes to the rest of the cluster if a
pending manual action has to be done to confirm a host has been
rebooted in order to move the SPM to another live host. Perhaps
only effect is not be able to add new hosts and connect to storage
or something ?
A missing SPM will not affect existing VMs. Running VMs (on hosts
other than the SPM host) will continue to run normally and you can
even start existing VMs. Without an SPM you will not be able to
manipulate the storage (Add/remove domains, add/remove disks, migrate
disks, etc).
Fernando
On 17/04/2017 10:06, Nir Soffer wrote:
> On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi
> <konrasko(a)gmail.com <mailto:konrasko@gmail.com>> wrote:
>
> But actually, it didn't work well. After main SPM host went
> down I see this
> Screen Shot 2017-04-16 at 10.22.00 PM.png
>
> 2017-04-17 05:23:15,554Z ERROR
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5)
> [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init: could not
> find reported vds or not up - pool: 'STG' vds_spm_id: '1'
> 2017-04-17 05:23:15,567Z INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5)
> [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM selection - vds
> seems as spm 'tank5'
> 2017-04-17 05:23:15,567Z WARN
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5)
> [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is non
> responsive, stopping spm selection.
>
> So that means only if BMC is up it's possible to
> automatically switch SPM host?
>
>
> BMC?
>
> If your SPM is no responsive, the system will try to fence it.
> Did you
> configure power management for all hosts? did you check that it
> work? How did you simulate non-responsive host?
>
> If power management is not configured or fail, the system cannot
> move the spm to another host, unless you manually confirm that the
> SPM host was rebooted.
>
> Nir
>
>
> Thanks
>
> On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi
> <konrasko(a)gmail.com <mailto:konrasko@gmail.com>> wrote:
>
> Oh, fence agent works fine if I select ilo4,
> Thank you for your help!
>
> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny
> <dyasny(a)gmail.com <mailto:dyasny@gmail.com>> wrote:
>
> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin
> Raskoshnyi <konrasko(a)gmail.com
> <mailto:konrasko@gmail.com>> wrote:
>
> Makes sense.
> I was trying to set it up, but doesn't work with
> our staging hardware.
> We have old ilo100, I'll try again.
> Thanks!
>
>
> It is absolutely necessary for any HA to work
> properly. There's of course the "confirm host has
> been shutdown" option, which serves as an override
> for the fence command, but it's manual
>
> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny
> <dyasny(a)gmail.com <mailto:dyasny@gmail.com>> wrote:
>
> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin
> Raskoshnyi <konrasko(a)gmail.com
> <mailto:konrasko@gmail.com>> wrote:
>
> Fence agent under each node?
>
>
> When you configure a host, there's the power
> management tab, where you need to enter the
> bmc details for the host. If you don't have
> fencing enabled, how do you expect the system
> to make sure a host running a service is
> actually down (and it is safe to start HA
> services elsewhere), and not, for example,
> just unreachable by the engine? How do you
> avoid a splitbraid -> SBA ?
>
>
> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny
> <dyasny(a)gmail.com
> <mailto:dyasny@gmail.com>> wrote:
>
> On Sun, Apr 16, 2017 at 11:13 PM,
> Konstantin Raskoshnyi
> <konrasko(a)gmail.com
> <mailto:konrasko@gmail.com>> wrote:
>
> "Corner cases"?
> I tried to simulate crash of SPM
> server and ovirt kept trying to
> reistablished connection to the
> failed node.
>
>
> Did you configure fencing?
>
>
>
> On Sun, Apr 16, 2017 at 8:10 PM
> Dan Yasny <dyasny(a)gmail.com
> <mailto:dyasny@gmail.com>> wrote:
>
> On Sun, Apr 16, 2017 at 7:29
> AM, Nir Soffer
> <nsoffer(a)redhat.com
> <mailto:nsoffer@redhat.com>>
> wrote:
>
> On Sun, Apr 16, 2017 at
> 2:05 PM Dan Yasny
> <dyasny(a)redhat.com
> <mailto:dyasny@redhat.com>>
> wrote:
>
>
>
> On Apr 16, 2017 7:01
> AM, "Nir Soffer"
> <nsoffer(a)redhat.com
>
<mailto:nsoffer@redhat.com>>
> wrote:
>
> On Sun, Apr 16,
> 2017 at 4:17 AM
> Dan Yasny
> <dyasny(a)gmail.com
>
<mailto:dyasny@gmail.com>>
> wrote:
>
> When you set
> up a storage
> domain, you
> need to
> specify a
> host to
> perform the
> initial
> storage
> operations,
> but once the
> SD is
> defined, it's
> details are
> in the engine
> database, and
> all the hosts
> get connected
> to it
> directly. If
> the first
> host you used
> to define the
> SD goes down,
> all other
> hosts will
> still remain
> connected and
> work. SPM is
> an HA
> service, and
> if the
> current SPM
> host goes
> down, SPM
> gets started
> on another
> host in the
> DC. In short,
> unless your
> actual NFS
> exporting
> host goes
> down, there
> is no outage.
>
>
> There is no
> storage outage,
> but if you
> shutdown the spm
> host, the spm host
> will not move to
> a new host until
> the spm host is
> online again, or
> you confirm
> manually that the
> spm host was
> rebooted.
>
>
> In a properly
> configured setup the
> SBA should take care
> of that. That's the
> whole point of HA
> services
>
>
> In some cases like power
> loss or hardware failure,
> there is no way to start
> the spm host, and the
> system cannot recover
> automatically.
>
>
> There are always corner
> cases, no doubt. But in a
> normal situation. where an
> SPM host goes down because of
> a hardware failure, it gets
> fenced, other hosts contend
> for SPM and start it. No
> surprises there.
>
>
> Nir
>
>
>
> Nir
>
>
> On Sat, Apr
> 15, 2017 at
> 1:53 PM,
> Konstantin
> Raskoshnyi
> <konrasko(a)gmail.com
>
<mailto:konrasko@gmail.com>>
> wrote:
>
> Hi Fernando,
> I see
> each host
> has
> direct
> connection
> nfs
> mount,
> but yes,
> if main
> host to
> which I
> connected
> nfs
> storage
> going
> down the
> storage
> becomes
> unavailable
> and all
> vms are down
>
>
> On Sat,
> Apr 15,
> 2017 at
> 10:37 AM
> FERNANDO
> FREDIANI
>
<fernando.frediani(a)upx.com
>
<mailto:fernando.frediani@upx.com>>
> wrote:
>
> Hello
> Konstantin.
>
> That
> doesn`t
> make
> much
> sense
> make
> a
> whole
> cluster
> depend
> on a
> single
> host.
> From
> what
> I
> know
> any
> host
> talk
> directly
> to
> NFS
> Storage
> Array
> or
> whatever
> other
> Shared
> Storage
> you have.
> Have
> you
> tested
> that
> host
> going
> down
> if
> that
> affects
> the
> other
> with
> the
> NFS
> mounted
> directlly
> in a
> NFS
> Storage
> array ?
>
> Fernando
>
> 2017-04-15
> 12:42
> GMT-03:00
> Konstantin
> Raskoshnyi
>
<konrasko(a)gmail.com
>
<mailto:konrasko@gmail.com>>:
>
> In
> ovirt
> you
> have
> to
> attach
> storage
> through
> specific
> host.
>
> If
> host
> goes
> down
> storage
> is
> not
> available.
>
>
> On
> Sat,
> Apr
> 15,
> 2017
> at
> 7:31
> AM
> FERNANDO
> FREDIANI
>
<fernando.frediani(a)upx.com
>
<mailto:fernando.frediani@upx.com>>
> wrote:
>
> Well,
> make
> it
> not
> go
> through
> host1
> and
> dedicate
> a
> storage
> server
> for
> running
> NFS
> and
> make
> both
> hosts
> connect
> to
> it.
> In
> my
> view
> NFS
> is
> much
> easier
> to
> manage
> than
> any
> other
> type
> of
> storage,
> specially
> FC
> and
> iSCSI
> and
> performance
> is
> pretty
> much
> the
> same,
> so
> you
> won`t
> get
> better
> results
> other
> than
> management
> going
> to
> other
> type.
>
> Fernando
>
> 2017-04-15
> 5:25
> GMT-03:00
> Konstantin
> Raskoshnyi
>
<konrasko(a)gmail.com
>
<mailto:konrasko@gmail.com>>:
>
> Hi
> guys,
>
> I
> have
> one
> nfs
storage,
> it's
>
connected
> through
> host1.
> host2
> also
> has
> access
> to
> it,
> I
> can
> easily
> migrate
> vms
between
> them.
>
> The
> question
> is
> -
> if
> host1
> is
> down
> -
> all
>
infrastructure
> is
> down,
> since
> all
> traffic
> goes
> through
> host1,
> is
> there
> any
> way
> in
> oVirt
> to
> use
>
redundant
> storage?
>
> Only
>
glusterfs?
>
> Thanks
>
>
>
_______________________________________________
> Users
> mailing
> list
>
Users(a)ovirt.org
>
<mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
>
<http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
>
_______________________________________________
> Users
> mailing list
> Users(a)ovirt.org
>
<mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
>
<http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
_______________________________________________
> Users mailing
> list
> Users(a)ovirt.org
>
<mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
>
<http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
_______________________________________________
> Users mailing list
> Users(a)ovirt.org
> <mailto:Users@ovirt.org>
>
http://lists.ovirt.org/mailman/listinfo/users
>
<http://lists.ovirt.org/mailman/listinfo/users>
>
>
>
_______________________________________________
Users mailing list
Users(a)ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
<http://lists.ovirt.org/mailman/listinfo/users>
--
Adam Litke
--------------52C86E47EED6E2100703EC63
Content-Type: multipart/related;
boundary="------------627018FE70FE823EEDB7FEB5"
--------------627018FE70FE823EEDB7FEB5
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>Thanks for the clarification Adam.</p>
<p>That makes total sense.</p>
<p>Reagrds<br>
Fernando<br>
</p>
<div class="moz-cite-prefix">On 17/04/2017 11:28, Adam Litke
wrote:<br>
</div>
<blockquote
cite="mid:CAG8F9UE5HVYeY0iwQ+1cJmaMHDx0nLnZywrJayGq8WBdTDV+Ew@mail.gmail.com"
type="cite">
<div dir="ltr"><br>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Mon, Apr 17, 2017 at 9:12 AM,
FERNANDO FREDIANI <span dir="ltr"><<a
moz-do-not-send="true"
href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000">
<p>Should I understand then that fecing is not
mandatory, just advised and there are no downtimes to
the rest of the cluster if a pending manual action has
to be done to confirm a host has been rebooted in
order to move the SPM to another live host. Perhaps
only effect is not be able to add new hosts and
connect to storage or something ?</p>
</div>
</blockquote>
<div>A missing SPM will not affect existing VMs. Running
VMs (on hosts other than the SPM host) will continue to
run normally and you can even start existing VMs. Without
an SPM you will not be able to manipulate the storage
(Add/remove domains, add/remove disks, migrate disks,
etc).<br>
</div>
<div><br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div bgcolor="#FFFFFF" text="#000000"><span
class="HOEnZb"><font
color="#888888">
<p>Fernando<br>
</p>
</font></span>
<div>
<div class="h5">
<div class="m_7118500493255933245moz-cite-prefix">On
17/04/2017 10:06, Nir Soffer wrote:<br>
</div>
<blockquote type="cite">
<div dir="ltr">
<div class="gmail_quote">
<div dir="ltr">On Mon, Apr 17, 2017 at 8:24 AM
Konstantin Raskoshnyi <<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr">But actually, it didn't
work
well. After main SPM host went down I see
this
<div><img
src="cid:part3.327B103E.0FE862E1@upx.com"
alt="Screen Shot 2017-04-16 at
10.22.00 PM.png"
style="max-width:100%"></div>
<div><br>
2017-04-17 05:23:15,554Z ERROR
[org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-<wbr>03a970dbbec1]
SPM Init: could not find reported vds or
not up - pool: 'STG' vds_spm_id:
'1'<br>
</div>
<div>2017-04-17 05:23:15,567Z INFO
[org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-<wbr>03a970dbbec1]
SPM selection - vds seems as spm
'tank5'</div>
<div>2017-04-17 05:23:15,567Z WARN
[org.ovirt.engine.core.<wbr>vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5)
[4dcc033d-26bf-49bb-bfaa-<wbr>03a970dbbec1]
spm vds is non responsive, stopping spm
selection.</div>
<div><br>
</div>
<div>So that means only if BMC is up it's
possible to automatically switch SPM
host?</div>
</div>
</blockquote>
<div><br>
</div>
<div>BMC?</div>
<div><br>
</div>
<div>If your SPM is no responsive, the system
will try to fence it. Did you </div>
<div>configure power management for all hosts?
did you check that it </div>
<div>work? How did you simulate non-responsive
host? </div>
<div><br>
</div>
<div>If power management is not configured or
fail, the system cannot</div>
<div>move the spm to another host, unless you
manually confirm that the</div>
<div>SPM host was rebooted.</div>
<div><br>
</div>
<div>Nir</div>
<div> </div>
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div dir="ltr">
<div><br>
</div>
<div>Thanks</div>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">On Sun, Apr 16,
2017 at 8:29 PM, Konstantin Raskoshnyi <span
dir="ltr"><<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>Oh, fence agent works fine if I
select ilo4,</div>
<div>Thank you for your help!</div>
<div
class="m_7118500493255933245m_-7049131619673457077HOEnZb">
<div
class="m_7118500493255933245m_-7049131619673457077h5">
<div><br>
<div class="gmail_quote">
<div>On Sun, Apr 16, 2017 at
8:22 PM Dan Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16, 2017 at
11:19 PM, Konstantin
Raskoshnyi <span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>Makes sense. </div>
<div>I was trying to
set it up, but
doesn't work with
our staging
hardware. </div>
<div>We have old
ilo100, I'll try
again. </div>
<div>Thanks!
<div>
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886h5"><br>
</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div>It is absolutely
necessary for any HA
to work properly.
There's of course
the "confirm host
has been shutdown"
option, which serves
as an override for
the fence command,
but it's manual</div>
</div>
</div>
</div>
<div>
<div class="gmail_extra">
<div class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0
.8ex;border-left:1px
#ccc
solid;padding-left:1ex">
<div>
<div>
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886h5">
<div
class="gmail_quote">
<div>On Sun,
Apr 16, 2017
at 8:18 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16,
2017 at 11:15
PM, Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Fence
agent under
each node?</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>When you
configure a
host, there's
the power
management
tab, where you
need to enter
the bmc
details for
the host. If
you don't have
fencing
enabled, how
do you expect
the system to
make sure a
host running a
service is
actually down
(and it is
safe to start
HA services
elsewhere),
and not, for
example, just
unreachable by
the engine?
How do you
avoid a
splitbraid
-> SBA ?</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743HOEnZb">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743h5">
<div><br>
<div
class="gmail_quote">
<div>On Sun,
Apr 16, 2017
at 8:14 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16,
2017 at 11:13
PM, Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>"Corner
cases"? </div>
<div>I tried
to simulate
crash of SPM
server and
ovirt kept
trying to
reistablished
connection to
the failed
node. </div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>Did you
configure
fencing?</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210HOEnZb">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210h5">
<div><br>
</div>
<div><br>
<div
class="gmail_quote">
<div>On Sun,
Apr 16, 2017
at 8:10 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">On
Sun, Apr 16,
2017 at 7:29
AM, Nir Soffer
<span><<a
moz-do-not-send="true"
href="mailto:nsoffer@redhat.com"
target="_blank">nsoffer(a)redhat.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><span>
<div>On Sun,
Apr 16, 2017
at 2:05 PM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@redhat.com"
target="_blank">dyasny(a)redhat.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div><br>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Apr 16, 2017
7:01 AM, "Nir
Soffer" <<a
moz-do-not-send="true" href="mailto:nsoffer@redhat.com"
target="_blank">nsoffer(a)redhat.com</a>&gt;
wrote:<br
type="attribution">
<blockquote
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quoted-text">
<div>On Sun,
Apr 16, 2017
at 4:17 AM Dan
Yasny <<a
moz-do-not-send="true"
href="mailto:dyasny@gmail.com"
target="_blank">dyasny(a)gmail.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>When you
set up a
storage
domain, you
need to
specify a host
to perform the
initial
storage
operations,
but once the
SD is defined,
it's details
are in the
engine
database, and
all the hosts
get connected
to it
directly. If
the first host
you used to
define the SD
goes down, all
other hosts
will still
remain
connected and
work. SPM is
an HA service,
and if the
current SPM
host goes
down, SPM gets
started on
another host
in the DC. In
short, unless
your actual
NFS exporting
host goes
down, there is
no outage. </div>
</blockquote>
<div><br>
</div>
</div>
<div>There is
no storage
outage, but if
you shutdown
the spm host,
the spm host</div>
<div>will not
move to a new
host until the
spm host is
online again,
or you confirm</div>
<div>manually
that the spm
host was
rebooted.</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
<div><br>
</div>
</div>
<div>
<div>In a
properly
configured
setup the SBA
should take
care of that.
That's the
whole point of
HA services </div>
</div>
</blockquote>
<div><br>
</div>
</span>
<div>In some
cases like
power loss or
hardware
failure, there
is no way to
start</div>
<div>the spm
host, and the
system cannot
recover
automatically.</div>
</div>
</div>
</blockquote>
<div><br>
</div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div>There are
always corner
cases, no
doubt. But in
a normal
situation.
where an SPM
host goes down
because of a
hardware
failure, it
gets fenced,
other hosts
contend for
SPM and start
it. No
surprises
there. </div>
</div>
</div>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><span
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191HOEnZb"><font
color="#888888">
<div><br>
</div>
<div>Nir</div>
</font></span>
<div>
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191h5">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div><br>
</div>
<div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div
class="gmail_quote"><font
color="#888888">
<div><br>
</div>
<div>Nir</div>
</font>
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354elided-text">
<div> </div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div
class="gmail_extra"><br>
<div
class="gmail_quote">On
Sat, Apr 15,
2017 at 1:53
PM, Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko(a)gmail.com</a>&gt;</span>
wrote:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hi
Fernando,</div>
<div>I see
each host has
direct
connection nfs
mount, but
yes, if main
host to which
I connected
nfs storage
going down the
storage
becomes
unavailable
and all vms
are down</div>
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354m_6867735599077560094m_2305649857973554660HOEnZb">
<div
class="m_7118500493255933245m_-7049131619673457077m_-3447473577979970513m_1774108031932592886m_3846525510791616528m_1120357608056684743m_362375030716496662m_-7954130722235415210m_2173925241577336935m_-6434846878319371191m_5622908635907641408m_8346864507156293354m_6867735599077560094m_2305649857973554660h5">
<div><br>
</div>
<div><br>
<div
class="gmail_quote">
<div>On Sat,
Apr 15, 2017
at 10:37 AM
FERNANDO
FREDIANI <<a
moz-do-not-send="true" href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div>
<div>Hello
Konstantin.<br>
<br>
</div>
That doesn`t
make much
sense make a
whole cluster
depend on a
single host.
From what I
know any host
talk directly
to NFS Storage
Array or
whatever other
Shared Storage
you have.<br>
</div>
Have you
tested that
host going
down if that
affects the
other with the
NFS mounted
directlly in a
NFS Storage
array ?<br>
<br>
</div>
</div>
<div>Fernando<br>
</div>
<div
class="gmail_extra"><br>
<div
class="gmail_quote">2017-04-15
12:42
GMT-03:00
Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>In ovirt
you have to
attach storage
through
specific
host. </div>
<div>If host
goes down
storage is not
available. </div>
<div><br>
<div
class="gmail_quote">
<div>On Sat,
Apr 15, 2017
at 7:31 AM
FERNANDO
FREDIANI <<a
moz-do-not-send="true" href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;
wrote:<br>
</div>
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div>Well,
make it not go
through host1
and dedicate a
storage server
for running
NFS and make
both hosts
connect to it.<br>
</div>
In my view NFS
is much easier
to manage than
any other type
of storage,
specially FC
and iSCSI and
performance is
pretty much
the same, so
you won`t get
better results
other than
management
going to other
type.<br>
<br>
</div>
Fernando<br>
</div>
<div
class="gmail_extra"><br>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">2017-04-15
5:25 GMT-03:00
Konstantin
Raskoshnyi
<span><<a
moz-do-not-send="true" href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
</div>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>Hi guys,
<div>I have
one
nfs storage,</div>
<div>it's
connected
through host1.</div>
<div>host2
also has
access to it,
I can easily
migrate
vms between
them.</div>
<div><br>
</div>
<div>The
question is -
if host1 is
down - all
infrastructure
is down, since
all traffic
goes through
host1,</div>
<div>is there
any way in
oVirt to use
redundant
storage?</div>
<div><br>
</div>
<div>Only
glusterfs?</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
</div>
<br>
</blockquote>
</div>
</div>
<div
class="gmail_extra">
<div
class="gmail_quote">
<blockquote
class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">______________________________<wbr>_________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/<wbr>mailman/li...
<br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/<wbr>mailman/li...
<br>
</blockquote>
</div>
<br>
</div>
______________________________<wbr>_________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/<wbr>mailman/li...
</blockquote>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Users mailing
list<br>
<a
moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a
moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users" rel="noreferrer"
target="_blank">http://lists.ovirt.org/<wbr>mailman/li...
<br>
</blockquote>
</div>
<br>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</blockquote>
</div>
</div>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</div>
</div>
</div>
<br>
______________________________<wbr>_________________<br>
Users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Users@ovirt.org">Users@ovirt.org</a><br>
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer"
target="_blank">http://lists.ovirt.org/<wbr>mailman/li...
<br>
</blockquote>
</div>
<br>
<br clear="all">
<br>
-- <br>
<div class="gmail_signature"
data-smartmail="gmail_signature">
<div dir="ltr">Adam Litke</div>
</div>
</div>
</div>
</blockquote>
<br>
</body>
</html>
--------------627018FE70FE823EEDB7FEB5
Content-Type: image/png
Content-Transfer-Encoding: base64
Content-ID: <part3.327B103E.0FE862E1(a)upx.com>
iVBORw0KGgoAAAANSUhEUgAAA8wAAACoCAYAAAA1pWe/AAAME2lDQ1BJQ0MgUHJvZmlsZQAA
SImVVwdYU8kWnltSCAktEAEpoTdBepXeO9LBRkgChBIgIajYkUUF14KKBURFV0RUXAsga0HE
ziLY+wMVlZV1sWBD5U0K6Pra9873zZ3/njnnzH/mnjvfDACK1qy8vGxUCYAcfoEgOtCHmZiU
zCT1ARyQAAowoMliC/O8o6LCAJSx/u/y7iZAxP01S3Gsfx3/r6LM4QrZACBREKdyhOwciI8A
gKuz8wQFABA6od5gdkGeGA9BrCqABAEg4mKcLsXqYpwqxZMkNrHRvhB7AUCmsliCdAAUxLyZ
hex0GEdBzNGaz+HxIa6G2IOdweJAfB/iSTk5uRArkiE2Tf0uTvrfYqaOx2Sx0sexNBeJkP14
wrxs1tz/czn+t+Rki8bm0IeNmiEIihbnDNdtT1ZuqBhTIT7OT42IhFgF4gs8jsRejO9miILi
ZPaDbKEvXDPAAPBTc1h+oRBrQcwQZcV5y7AtSyDxhfZoBK8gOFaGUwW50bL4aCE/OyJMFmd5
Bjd4DNdwhf4xYzZpvIBgiGGloUeKMmITpDzRjkJefATEChB3C7NiQmW+D4syfCPGbASiaDFn
Q4jfpgkCoqU2mHqOcCwvzIrNkswFawHzKsiIDZL6YolcYWLYGAcO189fygHjcPlxMm4YrC6f
aJlvaV52lMweq+FmB0ZL1xk7KCyMGfO9WgALTLoO2KNMVkiUbK53eQVRsVJuOArCgC/wA0wg
gi0V5IJMwOsabB6Eb9KRAMACApAOuMBSphnzSJCM8OEzBhSBPyHiAuG4n49klAsKof7LuFb6
tARpktFCiUcWeApxDq6Je+BueBh8esFmizvjLmN+TMWxWYn+RD9iEDGAaDbOgw1ZZ8MmALx/
owuFPRdmJ+bCH8vhWzzCU0IP4RHhBqGXcAfEgyeSKDKrWbxiwQ/MmSAc9MJoAbLsUr/PDjeG
rB1wH9wd8ofccQauCSxxe5iJN+4Jc3OA2u8Zisa5fVvLH+cTs/4+H5lewVzBQcYidfzL+I5b
/RjF97s14sA+9EdLbDl2GDuPncYuYsexZsDETmEtWCd2QozHK+GJpBLGZouWcMuCcXhjNtYN
1gPWn3+YmyWbX7xewgLunALxz+CbmzdXwEvPKGB6w92Yywzms60mMW2tbZwAEO/t0q3jDUOy
ZyOMS990+W0AuJRBZfo3HcsAgGNPAaC/+6YzeA3LfQ0AJ7rZIkGhVCfejgEBUIAi/Cs0gA4w
AKYwH1vgCNyAF/AHISASxIIkMBOueAbIgZxng/lgCSgF5WAN2AC2gG1gJ9gD9oNDoBkcB6fB
OXAZdIMb4B6si37wAgyBd2AEQRASQkPoiAaiixghFogt4ox4IP5IGBKNJCEpSDrCR0TIfGQp
Uo5UIFuQHUg98ityDDmNXER6kDtIHzKAvEY+oRhKRVVRbdQYnYw6o95oKBqLzkDT0Xy0CC1B
V6Gb0Fp0H9qEnkYvozfQXvQFOowBTB5jYHqYJeaM+WKRWDKWhgmwhVgZVonVYgewVvidr2G9
2CD2ESfidJyJW8LaDMLjcDaejy/EV+Jb8D14E96BX8P78CH8K4FG0CJYEFwJwYREQjphNqGU
UEnYTThKOAv/m37COyKRyCCaEJ3gf5lEzCTOI64kbiU2EtuIPcTHxGESiaRBsiC5kyJJLFIB
qZS0mbSPdIp0ldRP+kCWJ+uSbckB5GQyn1xMriTvJZ8kXyU/I4/IKckZybnKRcpx5ObKrZbb
Jdcqd0WuX26EokwxobhTYimZlCWUTZQDlLOU+5Q38vLy+vIu8lPlefKL5TfJH5S/IN8n/5Gq
QjWn+lKnU0XUVdQ6ahv1DvUNjUYzpnnRkmkFtFW0etoZ2kPaBwW6gpVCsAJHYZFClUKTwlWF
l4pyikaK3oozFYsUKxUPK15RHFSSUzJW8lViKS1UqlI6pnRLaViZrmyjHKmco7xSea/yReXn
KiQVYxV/FY5KicpOlTMqj+kY3YDuS2fTl9J30c/S+1WJqiaqwaqZquWq+1W7VIfUVNTs1eLV
5qhVqZ1Q62VgDGNGMCObsZpxiHGT8WmC9gTvCdwJKyYcmHB1wnv1iepe6lz1MvVG9RvqnzSY
Gv4aWRprNZo1HmjimuaaUzVna9ZontUcnKg60W0ie2LZxEMT72qhWuZa0VrztHZqdWoNa+to
B2rnaW/WPqM9qMPQ8dLJ1Fmvc1JnQJeu66HL012ve0r3D6Ya05uZzdzE7GAO6WnpBemJ9Hbo
demN6Jvox+kX6zfqPzCgGDgbpBmsN2g3GDLUNQw3nG/YYHjXSM7I2SjDaKPReaP3xibGCcbL
jJuNn5uomwSbFJk0mNw3pZl6muab1ppeNyOaOZtlmW016zZHzR3MM8yrzK9YoBaOFjyLrRY9
kwiTXCbxJ9VOumVJtfS2LLRssOyzYliFWRVbNVu9nGw4OXny2snnJ3+1drDOtt5lfc9GxSbE
ptim1ea1rbkt27bK9rodzS7AbpFdi90rewt7rn2N/W0HukO4wzKHdocvjk6OAscDjgNOhk4p
TtVOt5xVnaOcVzpfcCG4+Lgscjnu8tHV0bXA9ZDrX26Wbllue92eTzGZwp2ya8pjd313lvsO
914PpkeKx3aPXk89T5ZnrecjLwMvjtdur2feZt6Z3vu8X/pY+wh8jvq893X1XeDb5of5BfqV
+XX5q/jH+W/xfxigH5Ae0BAwFOgQOC+wLYgQFBq0NuhWsHYwO7g+eCjEKWRBSEcoNTQmdEvo
ozDzMEFYazgaHhK+Lvx+hFEEP6I5EkQGR66LfBBlEpUf9dtU4tSoqVVTn0bbRM+PPh9Dj5kV
szfmXaxP7OrYe3GmcaK49njF+Onx9fHvE/wSKhJ6EycnLki8nKSZxEtqSSYlxyfvTh6e5j9t
w7T+6Q7TS6ffnGEyY86MizM1Z2bPPDFLcRZr1uEUQkpCyt6Uz6xIVi1rODU4tTp1iO3L3sh+
wfHirOcMcN25Fdxnae5pFWnP093T16UPZHhmVGYM8nx5W3ivMoMyt2W+z4rMqssazU7Ibswh
56TkHOOr8LP4Hbk6uXNye/Is8krzevNd8zfkDwlCBbuFiHCGsKVAFR5zOkWmop9EfYUehVWF
H2bHzz48R3kOf07nXPO5K+Y+Kwoo+mUePo89r32+3vwl8/sWeC/YsRBZmLqwfZHBopJF/YsD
F+9ZQlmSteT3YuviiuK3SxOWtpZolywuefxT4E8NpQqlgtJby9yWbVuOL+ct71pht2Lziq9l
nLJL5dblleWfV7JXXvrZ5udNP4+uSlvVtdpxdc0a4hr+mptrPdfuqVCuKKp4vC58XdN65vqy
9W83zNpwsdK+cttGykbRxt5NYZtaNhtuXrP585aMLTeqfKoaq7WqV1S/38rZerXGq+bANu1t
5ds+bedtv70jcEdTrXFt5U7izsKdT3fF7zr/i/Mv9bs1d5fv/lLHr+vdE72no96pvn6v1t7V
DWiDqGFg3/R93fv99rccsDywo5HRWH4QHBQd/OPXlF9vHgo91H7Y+fCBI0ZHqo/Sj5Y1IU1z
m4aaM5p7W5Jaeo6FHGtvdWs9+pvVb3XH9Y5XnVA7sfok5WTJydFTRaeG2/LaBk+nn37cPqv9
3pnEM9c7pnZ0nQ09e+FcwLkz573Pn7rgfuH4RdeLxy45X2q+7Hi5qdOh8+jvDr8f7XLsarri
dKWl26W7tWdKz8mrnldPX/O7du568PXLNyJu9NyMu3n71vRbvbc5t5/fyb7z6m7h3ZF7i+8T
7pc9UHpQ+VDrYe0/zP7R2OvYe6LPr6/zUcyje4/Zj188ET753F/ylPa08pnus/rnts+PDwQM
dP8x7Y/+F3kvRgZL/1T+s/ql6csjf3n91TmUONT/SvBq9PXKNxpv6t7av20fjhp++C7n3cj7
sg8aH/Z8dP54/lPCp2cjsz+TPm/6Yval9Wvo1/ujOaOjeSwBS3IUwGBD09IAeF0HAC0Jnh26
AaAoSO9eEkGk90UJAv8JS+9nEnEEoM4LgLjFAITBM0oNbEYQU2EvPnrHegHUzm68yUSYZmcr
jUWFNxjCh9HRN9oAkFoB+CIYHR3ZOjr6ZRckeweAtnzpnU8sRHi+324lRt39L8GP8k+I32zs
Bfxd5wAAAAlwSFlzAAAWJQAAFiUBSVIk8AAAAZ1pVFh0WE1MOmNvbS5hZG9iZS54bXAAAAAA
ADx4OnhtcG1ldGEgeG1sbnM6eD0iYWRvYmU6bnM6bWV0YS8iIHg6eG1wdGs9IlhNUCBDb3Jl
IDUuNC4wIj4KICAgPHJkZjpSREYgeG1sbnM6cmRmPSJodHRwOi8vd3d3LnczLm9yZy8xOTk5
LzAyLzIyLXJkZi1zeW50YXgtbnMjIj4KICAgICAgPHJkZjpEZXNjcmlwdGlvbiByZGY6YWJv
dXQ9IiIKICAgICAgICAgICAgeG1sbnM6ZXhpZj0iaHR0cDovL25zLmFkb2JlLmNvbS9leGlm
LzEuMC8iPgogICAgICAgICA8ZXhpZjpQaXhlbFhEaW1lbnNpb24+OTcyPC9leGlmOlBpeGVs
WERpbWVuc2lvbj4KICAgICAgICAgPGV4aWY6UGl4ZWxZRGltZW5zaW9uPjE2ODwvZXhpZjpQ
aXhlbFlEaW1lbnNpb24+CiAgICAgIDwvcmRmOkRlc2NyaXB0aW9uPgogICA8L3JkZjpSREY+
CjwveDp4bXBtZXRhPgqcwRHqAAAAHGlET1QAAAACAAAAAAAAAFQAAAAoAAAAVAAAAFQAAEPb
KmoCuAAAQABJREFUeAHsfQl8FUW2/glb2EEQHJxxQX0JA2MQBWXIk4lIBMU8QEFcAozIIItm
UASREPkLhEE0CpFFEWHQABPJyOYCwgiDI4Iw+ieKAk/FyAhCZElYL1veV91d3dV97w25N7m5
uX1P/35J3+6u9Tt1qs45daoqpgQX8cUIMAKMACPACDACjAAjwAgwAowAI8AIMAI2BGJYYbbh
wQ+MACPACDACjAAjwAgwAowAI8AIMAKMgIYAK8zcEBgBRoARYAQYAUaAEWAEGAFGgBFgBBgB
HwiwwuwDFH7FCDACjAAjwAgwAowAI8AIMAKMACPACMTsPXSS1zBzO2AEGAFGgBFgBBgBRoAR
YAQYAUaAEWAEHAiwwuwAhB8ZAUaAEWAEGAFGgBFgBBgBRoARYAQYAYFAzNHiyN4le/9PP2qU
bPHrK5mijAAj4ECA+cMBCD8yAiFEgPkthOBWgaSZvlWACCEsAtM3hOBWgaSZvlWACBFcBFaY
I5h4XHRG4GII8ABxMYT4OyNQcQgwv1UcllUxJaZvVaRKxZWJ6VtxWFbFlJi+VZEqkVMmVpgj
h1ZcUkYgYAR4gAgYMo7ACASNAPNb0NBFRESmb0SQKehCMn2Dhi4iIjJ9I4JMVbaQrDBXWdJw
wRiB8iPAA0T5MeQUGIGyIsD8VlakIjMc0zcy6VbWUjN9y4pUZIZj+kYm3apKqVlhriqU4HIw
AiFAgAeIEIDKSTICfhBgfvMDjEteM31dQkg/1WD6+gHGJa+Zvi4hZJiqwQpzmIDnbBmBykCA
B4jKQJnzYAR0BJjf3N0SmL5MX3cj4O7aMf+6m76hrh0rzKFGmNNnBMKIAA8QYQSfs446BJjf
3E1ypi/T190IuLt2zL/upm+oa8cKc6gR5vQZgTAiwANEGMHnrKMOAeY3d5Oc6cv0dTcC7q4d
86+76Rvq2rHCHGqEOX1GIIwI8AARRvA566hDgPnN3SRn+jJ93Y2Au2vH/Otu+oa6dqwwhxph
Tp8RCCMCPECEEXzOOuoQYH5zN8mZvkxfdyPg7tox/7qbvqGuHSvMoUaY02cEwogADxBhBJ+z
jjoEmN/cTXKmL9PX3Qi4u3bMv+6mb6hrxwpzqBHm9BmBMCLAA0QYweesow4B5jd3k5zpy/R1
NwLurh3zr7vpG+rascIcaoQ5fUYgjAjwABFG8DnrqEOA+c3dJGf6Mn3djYC7a8f86276hrp2
rDCHGmFOnxEIIwI8QIQRfM466hBgfnM3yZm+TF93I+Du2jH/upu+oa4dK8yhRpjTZwTCiAAP
EGEEn7OOOgSY39xNcqYv09fdCLi7dsy/7qZvqGvHCnOoEeb0GYEwIsADRBjB56yjDgHmN3eT
nOnL9HU3Au6uHfOvu+kb6tqxwhxqhDl9RiCMCPAAEUbwOeuoQ4D5zd0kZ/oyfd2NgLtrx/zr
bvqGunasMIcaYU6fEQgjAjxAhBF8zjrqEGB+czfJmb5MX3cj4O7aMf+6m76hrh0rzKFGmNNn
BMKIAA8QYQSfs446BJjf3E1ypi/T190IuLt2zL/upm+oaxf1CnPx/t301Y5dVFjsIaou4I6l
Zi3j6abWcRRbI9Twc/qMQGgRCGiAOOehwoICOl09FoUCP9S7iq5oJn77vgp/2E2nwS9m2EtO
096CA+CjWMSuQ9dd3dx3xKr6tvggfVt4imKrX7zuWhXOFZn11auEeLGX0RUtGjlqCFx/AK4a
VkbI83XoiquaV14fE2jdjBpIGnvOE8p7VeWV14FgpDwGxG+yUiZtjBfnPVS7WRw1aygDGHev
9ibeN6Yrrna2N0c8fqwwBIKirzN30Pvz7Vto739Ef6F/bHZdAv0OMkdDXzKHs33Y0qtDtevF
UmzDRtTQR1ct+ZfAv41+dRU1rGeLrD14Dh+kvcXo97Q+H/1Xs+htT0HR9wTGjZ/FuIGREP36
ddf6Hvf0MUCEIWqOvtQnrb3J4543inwhcKrU8c9AMSj6KhQIVF8oLiyggydMNldSws9Y8G5s
LDUE73rpGl59fRWWLey1cvVT9CrMhVtpyoBu9PrX/uibQOPemEt/6hrnI4CHPn/nRXp2RwL9
PSNFjnk+wpXtlWf/Vpo56EW6bs6b1PNqH6Ne2ZLhUIyAFwIBDRAn8unu1km0w0xlFH1WkE7N
zGflx+GN1LJdL+tF0lzaPu40tb0jzXiXTO/uzqU2ATbnwi/y6Olxu2jcqnS6zpfwaOVY4b++
XdKPkseu1dNtnU07P0gtlbc9u3KolVlfWZxU2vhdNl2hlt2JlRH0uQ/304D4AAGS2QR4D7Ru
WvKefBoUl0TrZXk/QHlbV055A6xelQkeEL8ZpbbRRtakx0LaMztFPmn34k+zqO39mbZ3RMHx
mSMRfiwjAsHQ10waQvD6NzJo0JQc85X9h2+Zw2f7sEfUntr0mEbTXxhM10ml2MG/bZ7bTO/+
0SnPeGjFnS1opJSDytDv+cjaNa+Coa+TPo/l7KRRtzqUZs9u9KUdK7QvDedYGQzBneNlZY5/
srzB0FeLG5S+4KFc8NZYyVuyED7u/cYtp+ce7WzKG06s9ChVU7bwUR3XvopOhfnwVhrUrpvZ
eZVG3UFzvqCMu64ygxR+vYqevXMgrRZvoCTsXNjHbORmoLL+OHeQVs8eTcOyVmkxpkKA7ldJ
AnRZi8jhIhuBQAeI1WOb0LAlVp2nrd5PPWDEqVvHeid+7V2RRp3TLMEvbcEe+mOztXTj3UOM
gCmU98VC+i30q1gIcJrzhj0J+1NhPs2ckERZ74nXKVC2FwasbNsTDPzp23fSKPkJo07d5tLn
L4O3q5FX3WXKnu/yqFUXWV/5lmja6sPUA11G3br6O9+KDtHEFfvp3v+KpRrAthbyCeUVaN20
spzbTWOv7Ui5RsHM8oKmtWqEsrSRm3ag/CZqaqONWfURtOrLSXQtsK6DP3Ftyk6ih7Ly9Qfz
fwqtAp9dgzC1wWchbkZmrtH6Ixj6alid2E1ZrTvSzDIA1/qJ5fTeyM5mSN/tw/zs+DEKBrt0
3WDn4N9kKMzT+8ZR9Zrok2vJaFCYh0Nh1vpdvCtDvydjuvEeDH296ZMKnsym3zZRxj0HLWRf
Kpy5vGYXLwZsFRgrL1ZEX9+d46XEoDLGP1meYOhLQesLDt6ShfB3H5RLeyYka1+dWMkoVVG2
kGWLhntUKsw7/tqP7p5gzCRpVE6m4aNT6PJaRfTR0gz6aLdK+lT6Ozq/G9D5VYPLks1i1GMu
bcvsQ3SBqGFTIoxDAV1OK1Lmyv105+WxVA2CdiNpJQ4oRQ7MCNgRCHSAKP5iFrXtlWEmkjh6
A83ol0B1Mc1s6cxFlDuwJY3dIIOl0tJN2fTrfXnUqY9UIPV3LSGYxTYmqn8R5rBb6aEwb1tI
LZB87UugeFaScmYTfDrNhYLSh4Rc2QB1N+VLWWXc7YNaAt7oykzPzM2U0S2O6qBPqAsNZtM0
KDqznIoOUUbufuoJjag6gG1cX0k4BD8DrZtWBIeQNzlvP3WH8aR6bZS3QQgK6YIkA+U3UWUb
beLQjnbrbUXDu2UsNb5UCN4HaeZVrShLw8hqa8K4tHTTQiorn7kA4rBWIRj6Ejn7S70KiQ+O
og4Niba+mkWfOGr17Dv7aeBNkAXw3tY+tHAJlNjpMvrllwO0y2gravQuEzfTKw/Eoe+xG7y6
pG+mab3jNC2uKeQZ/XII9WXo92RMN96Doa83fYjihy6nJcM70yXSu93Rl5p9fxB9aVUYK4Oh
vX28rNzxT5Y3GPqWR1+wGaNEIeKSKfHSA+DdfPCuLJV1z1y6n/rcHEslNmO81d9XRdnCKr37
f0WhwuwcILJpfXYqWfKfh1ZntqDxyyzii86tV6tYatIYcQfCGrvB+NYpG0J1KtWKgVANoUYV
qj3FRVTs8WBdC0yINWpTw7ribqUpfjkV5jGL99N9cRUkQJ8jKsbaJ7FeRpQhNrY2ZvpQBr6i
CoGABwjHwE40itZsSafLwCCmEQcuxnfDHXuHRPLh5bRthD4jUg1tvD6mk4tP4qMxrexTYT5R
RIUnwR8Nm2tr775dMhDu0LqnhVACFm1cSPEwHPmMK/Mt693BCwReaOiDF2yCjxQcwdsNwdu+
9H1LALAGNK1IXaFsT+2jzTA3qneQsqDozPRRVlNoUhVmrPMqPlIE0xx4FvjF1sP6pouxrQdY
ij0YcMXGYi2jj8WMzrptgzGALiBOtSK6gP6hob98gJ1Y/3j4CFGJQU9/Cr7Z5/kpg1ZA+c+g
v3iMrYs2cBEDoZm26E/90E8mHc57wPyGwpq0icfDLqv0QvGZdhcML8JodARLINorSyDMYJbC
XBuGqXqyoZrtCHBVR5tocrFGZCRoxvOxLlZ8w7gm2qbPdXdmmfQfHtDYAx4XLVPyuSOI70dH
3+A7UHjeBkNfT/48apUyRilwKs15/wXq0FzSxENb3xxCw7Jl/4egvRfStmdTqDEU2z2q54vo
G2EgiZfCBta771r3Ij2UrptStEwMuaRRw900EW7A0kNEKswx6KOboE3pl1MeMgyFpfR7MqYb
78HQ1+RfByDC4NXrhlidJx3jqs++X8QvQ78Y0rHSUQftUeP705Alwcnof5s1kVYAX4Ht74oL
IYMacazxUg/jFwN7EhX6FDh9nfwRoL6gem9ImULW6Aw8TGfeR+MXW8b0RBi1ZtyLPv9oHrXR
vNcqULaQ+fI9aARYYY4bQfOnj6cEY/CqBqHw7A95NKD3Uvp118ugSDeiLiPGU+LlBTTllo70
ji+oMbh9OT2FxCTR3k/zaMrEIbTax7qFfuNysU4hGeIGhKQVGZScNstXajTl3f30wKVQSjr2
M5WS6f/EbBRmd/QLSv3YFqbrbL+XN9PUe2A5FhfWSW1aOY+mPJFpxtU/4H/rFJr+7ETq+XvL
xdz8xj9ciUDgA4T3jKgY+O+8FgYjY1ZiL9puZ6XtSkPPGfCNNcM8gpZumUQtwU+1i1bR724f
qOPbOpWe63cZTZhgCXi33Uq0/mNf8I+glbsm0a/+lUk3PyLDj6C1302y1jdDEJkCt+HXjejT
P9JnbfVH8MI7fngBaz/HvTHJtkeBTfCRg1spgqMpALTCoLZTH/TikfEu0mfXr4ES2OTsWig6
/fDWGvj0MA4Le8xuWjErk0bOUoRmo04EzObPfIFuAw1sF9w8c1/KoLHzVG8ZESKFpuakU79b
jT4Bb2x16zqN5vQmmj5ijKqfUfdRC+n5tBRqKDNxCHlyhrlG7G76y/VSEB9F727uS5smdaQp
78mI4p5M01fNpZ4JDuHKT5nbPID1l+Ow/tLMXE/rW/SnWT770wR67OWJ9Ng91rovNfdw/Q6G
30zayIYh24pieDnzqcEDrVEz29jiVJg9tGnJZHporGNsaY1xbuZTaEMKPTCL0VITyojaDJ5L
03sSTUkZYluq1B1r66Zjbd2OJZl071jJgzq6j2VvoFE9Rbu2X9+uy6GsrDSvMfC2wdk07s+p
dhr/sIpa/sF/39AOSX9hJN9mcC69m5Fsy2wHDG13m4Y2R99gC1kxD8HQ12uZy7LD1OUKqzwx
mEaOiTlIs9q1ojfEa3gZpPacSCP7dda8T375UFkq4lCYY9A/lZQUUc6DLWm6nLEy2k0s+pRp
N0k+JWKF2cLc369g6GvyrzPRuEm0PncEXYlxs5qjL/VSFsvUL0J5g9w3cokzI/E8gl4He/5p
lMH3SdNo+8LBZl++KbsflnMY40RSNr6lWt9UDyjw2B7JY4fzKXdeFo31GpOSMb5Mso0vogSr
J2E51zzxi6jfqGnU7N9jaOYG7RH/RtB7H7alHncMkS8sD6tz+TS1Y5Jp2BEB7OO4GaXcPwKn
r0NhLqu+cEUsvOM8tC5NWe4gZQqjFhrvnthKf+7czfQwkTx6/uc8ukUsb6sI2SLE3mvlJkoE
JRCFCrNQCJrARdJOpfiuo+hPD9xKba9ri919daGi6BfonyVGuPO76WkozP+wR9OfEudS/uI+
VOJzUxZHhOHLac/Tne0CrCPIhGVQmButsq2PtK9vtjNx96mbaQ5csMS1HmtQB/nsUK1Mxi3f
Q39qpwhO1if+5TIEAh8gvD0f4tPW0KIBHaguXIzrVMOyBHhZWO7Yo2glZqAvh2J85kcozPfI
ATGFVn69kFpj9vTcHrnW11Ia2wDnHQbWvX5PtPxTX8Cn0NtfLaRL1mL2+QmpSDrWN0MQGQaF
WdtTAEmofOJ7zac9nxFv7KSnujbXXtoEHzm4lUVhtiepPWkC0XXYd/b/Z1KiqezbA1pC00HK
aSPdbe1h1Kf5mw/TbS2MN4UbaRBmHNerARy/u03dQK8+oCs0Zt1MpcwR2HhsPXINvfdEB/3J
ge1fsOlXX+EBQ3bMfaekv83ZdpgSmxkhsBZsGPaOkLTyjpdMizbnUiejjoVrMujmIY6O2hGp
9TiU91GjvI5v4XgMht9M2ngVWJ9NbAVPi+0z0a8bwqg9mKowF9HfB7ekCRvsIdSnJ3N20uPG
hkSmwUcNEODvse/soUdvkmOJh9ZPauGnnDLhBJrz0Rrqbhh/rDL47htS72lDOe/sMCI7eN/m
po4gDyyknVPLvwmnLKmve8D0dShKZCiztYzEa8Go1gD0FZcHs3H7TsIbQHijyQv9z6GNadRN
7q0AhXnpFrjgo781vW+Qh2o0JNl3XdhNU2/uSHlGWlIY5xlmCa73PWD6Ign//EvUM/MLeq73
VdSoBtzjldl+q+/H8pYzZe0X7XKfvfToK5Yl00O904zXCbToiw3USTNyO72ckunvX+bSjQ0R
9Bwmgq5tZxqc+2Vj8qVnHHlgyGolDVn2jMyn1iMW0jtjJL9ZZTPH9tbg6a91QzJ1zqaPn6lN
t94p5QMYjPMOU88rCujVW9qR2rV1n7oB8izihuAKhr5B6Qsoe/U6HvrnGEVhlryvyhSYZGj5
BwsTyaNn9+bR73tb71UoApMtQr/cSy2b239HpcJM+2HV7mhYtX1SOAEWssH0QI++9Bu4NwpL
EEFhnozBZ5mv8Jhh3pR+C+W0b0Wzle+padPo8jP5NO3VHOstrI4bYHU8+XEG3aXM0lkBiMRM
ThdapczWWbNRNTC4NqpndU4inmSymFNb6albrc3M7hmeTX9ocyV59n1MYzLV2YF0WvflKLpW
dJh8uRqBYAYIwmZ0WdeqbsS6UnxlA7Q9j90dOzENa5wH6IPb+f2wimKGSr8sQb7a4Ty6UbEs
2wEfSqN6vUpZy+1v9Sd9lprWpFHfZyUPWelqrqpQ3tSNqaQgUgPlvCFRdWFNoZHpyXQGm/bN
XmbNyApjwJLBHbS1ZjbBRwqd6uDmKKIl7IsPydSl0wH6aJMuIMi135/NakLDF+gRe6I/oNVj
aIUxEyTLevSLTOr6J4U/E0fQmKRf0a71GbRik5XpSGys9vAtEKjr+VgTmZhKw+OIZi+QOOll
WvhJLiX+xuHWiXCAjRIfSqeOdfIpa540Rog4qZS3LZt+ByU31iHsy/LaZ5hFHHEl0+NP3Uw7
Xsykj/QX2n/hVvwSvF8aNbD3WeJjq3vSqXujApquljkxm/69IJWaxBbR63e2pCnGbOpv+6RT
aqcO1OT8j7R0dJotj9c2HKaklvYlMUoRKvVnMPxmb3cplPjLKvrEaCOa90b8zzT7pnY0X6tJ
AtpxX/ogM8PwDrD4Yd+GNPqfpxT6x6fSyG6X0QfZWYonQTLlbsulm0Ffe/vVYYpPHkUpVx+g
F19X0jEQ/MPgdEo4/Bm98o7FP0IB3JzVh5rBHfwnGDg6Owwcib1TYRnLMeujJ4VNkb5BG8NY
5qsMRna4jaC8xfHU58E089XwV/fQoJsbUWMY76o7hE3BH6nX68q71jfUMKNV2I+A6eswOkk3
e1Ggmqi/r6UIRwq1bVG0MsdAMf4FdLUUZhgclq2gtqg/QSA/d6SAVs/oZvfueDCXtj2JmXgo
zM9DZlmqpaTICcCFXbINUBy3gOmL+Db+daQn+sX562EEvHI3jVQMu7Iv9VKqEN9/v9iXPp7g
f4Z56abBtKJTO5KcO2jWThresTnVL1lLrTQvJ6twGq/A+6dmUR61NbxMhBfUnDUbqEPj3TQd
k0MyHT1WMvWEV9IKZewU73s9v5my7o/T9vfxWq9rZUc9X9hJT7fcaJNpJ+duJvpbR9vyx8S0
5ZAn9OVdIrou7yoJlfNnMPQNSl8QZQd/rh+tKMxx6bRy3mN06YXTVLORh44WbKSsXvBGVeqU
+tJOGtm5OdkV5uBlC3/Lp5Qs+WcACESnwgyACj+dRzffP+aiUD02dzONwgY++gWhbwAY4J/G
o1zDLB7PYz3CX1+hjd98TB9ugNKba1nRD23JpG4jpEBsrEHCZg91fsmhNsqxNNK1VSRnd2+1
FGadAezCp1SYz/6QQ7/vI4WLBBozay71ah9HtcTs3+48Gj9/OyXc0pbir0igtu3iqDYG7EtY
aRZwu/YKaoAAGs6NLrQBHrOLJ7faheLJcC/sbrgXVsO6mxu7DjGwtAR5OpRH7bvJ9+LzCFq0
bjy1pALavq8xdWjdnPYsG0h9M6XiBh4x1jCL0Hve960wazMscDv0pTBXh6tTzltv0eebcmj9
11C8N8E9vJZIDZvszEqiYQsMy7fBw7Xx7dA/FNfHgBXmETR5YmMa/2ymnomI/3JnmnNLK3pL
f0OT31xDO3GUnRREpNB0FLyZu2oVXGlX0fnec2l+ur7ZGGEGbTaMcLqiBAVXrG/Cpj01gWdb
Bc9EbDAzY7AuZJyBm22nfhbWQjDqD2+Sw+uVuqE8wyFMDbpFn1nfA0Wnb/oso5RG/wTPgKZY
A+kT21qYuUqwXD0JRsA1b42gpuhn6GQ+TeqcRCuM1GTfVOvwKkq4wzJSdhm9gaZhMzlxib6p
04NWmWe8j5n0y/NpzA1JpjDRc/RyGtm7MzUQNEQe0yfOo4atb6ar4+Op7fUdqCmMOdZGRlqy
YfkXDL/ZBO7e02jyZWtp/Ku6UqrRvHMB+Ee49YtrFM1/M54GDZB4GXxWvYBmYqbmr1oY/Euc
RutnDNb35nDQRAhlT8Krou4v0vNDj6S2o/x5/WiQUQbxtUs66NVb0MtDb8PNcNomPY6czaxd
o4Dm3GDNVAlFYUbem5SoLSPyXqMrZt4mYuYtFq6H9p3mnX0D0XzVEC3W9qZjVgsKekGe6o6N
ZRBbsrWZVyrFyGWUOuhbwPR1GJ0kP4gyOvc9kYU6cwriBNy05e7otvYhA5Vy14wK7WE44Bnm
UlDy/Slg+iIZG30enEuL2u2ih0ZLeQ8BHlxO/37qKpp2YzvT7Vj2/dULV1E7nLoir4v2izAM
Hljhe78PkcbWeRjbXjXGNsNwUowxu8sw2b/rOUmvsZ9WD6Se441xNw59xuLBdAaG2m6jrfCJ
Q3PphcHJ2h49Z/ZupNG9e5kuxGIsX7l1EsU399AH6npdZJOauYFGdo2nPV9vodhrO9OlB9Ul
W3o51P+tHs6l+SP0fMR74QnR+JKK3fk/GPqKsgStLzgwEWn5v2CwWCcMFlgWapthDl62YIXZ
P9rBfIlahVmAdf6Xg7T2vaW0NMe5M7YdyueW7aEBNwrLNRTVYVCY3ze+SxcLGdwYqI//THT6
5EH66YedtOuLj+mDNdgFc5cMZCkSNYtV656lFIuQwSjMMUc30k1d1Vk1Pc/45FS6u1My3dy+
M13bQrfAy9LUxUwDZGO+XIpAsAMEqWsLgU38ULhlD06g959qQc9uMMASa7QWj9CEcjETUu+I
Kvxa7TwGCt5NioI3CAL7cFhR1cufUizC+PumbXLkT2FGoxY7T8OYS0eOEh07WkB7vs2n/G1r
aTVmNHfKzCUPo/xHMJNjHisVhMK8KC+ZZveRwgQ29lnQmaY/PMSY3YPSvr4/rbjNstybQpNR
1lNF0AVPeejQvgL6HmXdumkpzVcs+lLYPvJ5FiUPMRRzYwYjAQqjdsV4aPe2LVTSvBVd3rS5
6e7500dp1HOMoaordBNxYn4CfXo6FDAopnUcmwaZ5XUozKqhT6S35pkmlG5MQsoy2415MCgu
3kmJaAJnzsdSrVPbaVivXiZN9JkRopybWtF0kaB6xafQ4B7J1LHjrdTmmqtsm7HVAgYNYIgM
5xUMv9kEbhhMlnY7QH2HZujV6JpNc7oV0DApgPfOpfUDTtNtvaWQbfAZdmgf2ymJ1hmV7zkR
wuqtV9GZMx6qVesUrbytHb0sgZGb9NkMWcbskpi5xOXkOXWjqR8+GEh9MgwhW/JJ0UYacHsv
c3l16gtf0MjbrtITw3+xRnc26PmGfGPwXW2H94nvvgHGtGeN/Iw1vL9tUESLrm9p7Bou+6cO
WurV4NEcKkNwwPR1KszGRm52t2gJiu+7rX34DmK+TYTHzAwsn9GuEswwd+AZZhOcMvwImL5I
00YftOttU5Ppbawpn2Z4iYhsM15dTgeG9qK5RhlkX3r+Pzl0Sy85yVGWfrE5HYSL/l2qi76x
S75I+szueTA+jjFyGUFrtk2iH2Y0oUffMl7JG5TjTYv708rUFjTVGAyl98NWGMuGmcYya8mV
jHr4s0y6Y7hlEJiMTXHvbEX08ThlNlVsFrotnYzuRIvqlGllevpdL6sML+SJxk0qVlkW+QRD
X1nOoPSFABTmka/upNT2ulx0FmPy780xGUbEcsoWsg58Lx8CUa0wC+hOFUNI9aCjOVlEP323
nbZuybO7UCOMcK98JTWBLhG7ZKsMIIUFaS0W5yrPe4Vm/GWWKfx5k8dSJJwzb7ITFce21IE1
TrW8m9804dpeDimUxtTw0MfPo9Na4p2r9SaFXng7m26Pa0Q10CnFQqkI9RmwVt78q7IRCH6A
sLvEEqXT0vc704y7utG/jErIAVY8Ctep2rbZIqudOxVmuXmUiBcLJUes2PsaO8H2esZQ6MQ6
PXlUDuw7P7+rKLLKt4sqzKfyaeHMLPp/f5XCtsjRcSk8bBNElPcX3yVbpIkZro3j6ZPOPpQ8
8bkrZsYyE2gSZgHl7KvK02e/X0VzXsmiBR8aswMijuOSfP4DZgX6yFkBBQ8hZDSCkIEbnTkO
IwFmquRVgI2D7h1n4CuNBPhYDYqxfaZRwb4BlqHEWzPJZnkdCrN8L/Kq29RDHz5mCU6+yyxL
5fsuZ9L3woDRW3Ux9hF8+IubafDtOFsWla6JvrEO6hPOKxh+swvcC7EPRSMaCsOnaWNVKiRc
GzPabKT2d9kNHPYj3ZQIvn4aXhXCUGIJZdYMrbcbsNUmhKJn81aQfPKfVdS+l1TiiVQjinQ9
Xo9Ngcx12Ea8mP+gDHBLlJevvuH0wY30wK3WrvzD4TUx6NefUfs75Ky7voxJHHkmrlC5Y4u0
A6avB+v9sXZ1tYiMSy7V8Kkwix2SPbWxC7FeDz2GQyGTL9U7Ngnr0iaZuvf+E3WBt468qmHd
7PM3WPwredGet12WkB4D4uQPf/2eTN+N94DpCxBs/Cv5Aft5tDf38/BGSvaZP64dSPc8U8r4
pEQ1+0V4QvV+2vdYWd9xlNjkNzfQvgFJtqWCepIpNGfxI7T0wV7m8pZp8Ozpgubz0UtNwL9G
xnE4CWYxToLBo+gX6mN8qeVY0qjXBfGebUFj3jfiKWNMTE2MS/BkLP4SHmj/Y/G6EdK8CcVb
29sA3hXBHNNqJlTKj2DoqyZXLn1BTcj4HY9jplolJtN9fftjlt7i+5LCPOpgrvcun2wR6iMr
fVTLta+iT2HGZjlTJrxBe09g7c+GfJqDjXTugFnrxAm4QZzDjpOC1Ee30qSu3UzBVg4idWpj
Ef/TljAo3+uDSxEtwZq7CV872krnFOq6cZVp+VfPzXQqErITFW4UdQ4EozBjbRIs758smkeT
4BrqS+DSS6ds+uAoLj+6C4HyDBA2QcALFvuMVH14KpDt7EBFyHbMMMt2LpQ1eU6lPS8rrlCK
968KQmE+vZWe/G9rPb8s/h/uS6V/vi2FDbyVAg4ExEJY7u+UlnvlvT/B0b7+EmXGZjzHXk2i
QdLdW2aKu3BBzejmoTHtO5oCisThJGYFOpuzAlak5L4ptHapJUxJgdeuMFuKTiyEkvrWmEvH
YQwU2nNNCCx7VaODrBs+l0a32lCYJwWgMOvnM9sFcLPMK6HkT7TqYtXS+5c0xIj2cSg/h2ZN
TKPlymyNM8ZYbGL4aBXZxDAYfrO1fWOGan77lj6EXCiGYgkEtnFqb24Io/PKr/fmwBXfmqly
YmR7NuhPjk36pJGqFoyoBz7wzXPCFfpnlR9lWgVwt7zXEoalwiyE5SaIIy5fCjP8qm3xJE+o
fYPm2aUYquPhuvn0ddj0zlxGkI7ZrFH6bBbaeyhd8wOmLzZVysKmStaxcvrM26UwPFjriAU6
4BvjyMo2Sal03z39qVePDtQQ4WztAx4li7AmVhy5Z16os/PSZujgITIO62ZzjY+Sr0pVmKWi
wwqzE1K/zzb6yL4V+O1ZnER9s3wbQGU7//E9KMwTAusXC7Cnx73pcgyzj5XiWDkbn6mlfmgu
zbh6I/05U8ZVP8JbbJvwFvPQmiktKP0d45uiMNeBnFxXHAzuR2Fem96CnlljxJM44FF6MdrH
SyOcejM8ny6DLAEROCRXwPxbkfqCcHl/y1gmI2vng3eFh0xdKMzWhFnwsgW7ZEugK+YedQqz
8+zj257bQPP/mGCiefYMZmdOF9GcPi1pltQ4DeaPrY4Z3PGKwqwMLrGHcuh6ZT3ygIzl9FBy
Z2oqBraDsL7fJa3vVgdXmsLcGOvL1N3zpOVdZwD77J8USs2BEOc/f/dtEZ33/Ezbv/yM8jeu
pZx1ykYtKFIGznzu0xbnaUI44su9CAQ8QKhQOM9bVr8p6/dlu7MPiP7bubQkqxt62IQOZdZU
KMyH31ePsbIURG2G+Zh9EzLfgkgCTXhtLnW7QV/PfwjrebuNNoQGObBXkMJ86bYsum1EpoqU
9nsydgTtfsVuGoPNVD4yvuplJfrwiRY07mPjZasR9Grm49T+NzD1YzD9KBPW/mX6N8nnh5HH
HUOtPGaswU7UEGY0PGoKhbUbdj29lXreHE9tfp9CneIb2YVupc7CGHDBj6EjYIUZUk7j+r4V
5uNfz6KkAYabMaozAycBiDKju9WuY8fhj14rFi7kteFCrGv9Wrtq4KED3xVR0blT9P2OrfTl
F9i0bbFdwEzE+tpX7kugRmgrQp4L5xUMv9naPmgjzsi2rbM3K2S0faxva99HKqcGn3lgILqt
G200wg566Qsanvgrgke2dnng738GvhwN6gBfcbA32tYZm7Jq8WttrBvcv9K3wuxlwJJtCTPF
7ZWZYumSrRtRRBEcO/UaY6dTYfbVN4jYxaWcQCEVQRFO7VPEc0VfwdB307QknMphKU7C7XzE
bc3NY/q0MjqWwBBNou0FI7Sjf2ztQ+kbfdUtBv1YDZC3oVii4ZjdJqPPjq2hKOsOl3EZhmeY
r/QFr893NvpIfhAGB2yelQ6Dhdx0TY0sx6mTu2fBWBpYv/jD6jR4GEmlV+Fb9H9CYfbHK73g
nTLmyg99Gtak54Mw3KyFl+IzZqEtl+wGUGRhw6TCj3HEXariko1NartfjXEMCjMOLdAvRTaW
Bme7fIBgndJp8m1HaXzmLCMS/LTAG091a06NQiSXBsq/IdEXzNraf4jj5WphDK0PncGOFWgM
Y3zgsgVOtNDGZHs+/BQ8AlGnMJNj10oB3aDMNTSoZwf6tRhkiuFWvWQ0DZtiCWVmZ4L1gRvg
dvLUewbgUmm4ALfS43l0vbnbIKzAm2AFFr0LLvuaEKuD81KYocT2jEMjR7zG9ewb7vTE2qeM
u+B6CAY4++UsbFhmdbJSkN63NoP+5xnZ+WAdzZZcukZasA6upYfu6mfOOssO2zkrpZeY/7sF
gUAHCHu9rVkP+3t9YBO7OYpLzEg1QLv06uQNt2p7O7fav9qZ24QOCIXapl8Q/GpDqaqHmTB5
XqzITwrVQnjfPkc5XxLfZLv+cW0aXN0MoUJbrzVYG+zF5nzz+2M3ezljqQg4FTHD3BIKfHuv
fQQMRQdbU0+GwrxcVAKXqTArgoa66zjOk6GHulmuuZLPY7Ab+U3mbuRICOte14/FTsVQfs9j
A622pisXUXec0T7nnriwK8wxh1ZhHbs0GhLZNrdRNiqLj0uh4c/PpUScEmDNohJNfHs/9bgm
VvcAAg1nYzO1+RqK1mZooVy7amR10Vsw/GZr+4ageQzrBLsp6wS1jPFt29Q+jv0tDH5ybvqF
tc/rp6bqm36dgadAp46ax5RwAbx96DQa1Bnrm/0pzOCr0rw6bN8k/1wooNk3y528RWn1Tb86
t4qlxjB6bHptCD2kjKlSofZXBrVv0OruVOy0l+Kf3dNFzmaZnyv4RzD0dc7IiSKlTlxDjz/U
gUQPuvfrPJrgOP9aW8v9h+ZUHcrtPrjg9njSW0HSxm4oSBA/tKual7WoiN50eL11GbqQxj50
B/3XlTjyrljssD2QRs6zlHmz/xEKH/oTJB9VVzD0tfGv5AfgJzZ1O+3H0CPHqerF2PSrSwD9
Is73dSrMtrFSEOxEPg1rbW2YKAmoeadclk9/xl4Hn8iXxl1OyIjH09sz6L8fkXIk+ldj06+m
zeCBuX8jjezYy1xioG36tWUSjpXEUhxlHJNjlboBn10+EOMfjpW61nGGuFjatCmbrv+1vlTL
UcxyPwZM3wrVF7ARKIyhQi2oD7lG5S0n79qx0hXmwGULVpjL3WAcCUSfwgwAnDsAOzDxehz5
Jo6raI3FlFCYP32xBT2+xBEE5zBvm3Aaa6oUl7hOo7D5xq30w4fP0svvWAOS6pJdG1v6/+72
IY7EiCYs309/bI2NXpS1TyJQl4cn0Q01P6OX5lrKvPYeu+dOw+65x+HamWRz7Uyh8a+k0rUl
B2jl22m07F8yK2s3PmsGQH7ju5sQCHiAcFS+cB2syV7nCMMgtNFwCzQEAzEIeHXyF1GYa0DJ
lpbkvSvUWWRZCAwUOxbS9T/lUCuVtyAkDxqdSjV2jKG578uw+l0KIk5Xt8SH52JDDWxI9coQ
Wm5scqLFUASccivMWn0P0pvY1TdbLZa0tuNoukwozMaEsaUwqzPMiDdoYi51qL6bZqRn2PZC
kEJITI0iWjuypXIWtsgsmXrfR7TsbdWTBEeabMilG9B1/YhZCXONuFLnypph1sqMjXDGfm0B
E987nf54A9FbEzLNzaJEPRZvzaWbqm+lR2/sRhus4DQ0fS7dAqX5q/dzKOvvVj3lrsDS00GJ
Uuk/g+E3nwI3vDsG3GFtoiUq0jMTRlOc2GDfPMcyQO1Z2Y/6TrRwobhUenZgAn2zaAwtVXAf
nbuHBmMTy3NQmNv42NXeaxZZmdX0+qa0pcJPsKQhTSp2OvSJ90Jp/yaHViv5W0K2/1lutW+Q
RPw8O4nudbq4yvwRqDLoHwx9RfnLcia8rKfYCX0NzrbXdp3HywMfQ2GWS0UctBAziqVdhetw
lrmi/JQWVnwzTz1Q+vWLxXHT92Do65N/TYMDjM7qRrEGWHKcql6niNb0DaBfFN4f8LhKSrMU
Wj1Jfaxsr83MIs+xzr1ssJu1odiuGQmXa1MWFLGtJQ3Clb+JmKxRzozW00+me+8n+vvflP4F
H3pic78MbXM/zEwrLtlyrCpdYcYEEc5jP7Mby0mUo+OE0SYbG4w2wbhV0Vcw9C2PvmCfYDMU
ZrNt+K+db1kqUNmCFWb/CAf3JSoVZuEe9uaAVjThnxcHTZ0JEZ3JCQxenR1CAcWn0yerUmgB
Opl5F08SGyrsofuw6ZbT5VFGHTEfbim3N6d/zUii/i+pyrYMYb/Lzknb9Aud1sj37N+9nuQu
qfjgZcn3CswvIhmBYAYIW31hrR4Ea/V69aVxtIt4FQNNWQ5szk5e7qzrb4ZZCN+mwOc4U1XP
LoHmbdpAtzc/SK9c14pe0l+W+l8KIuexAdktd3sbo7wjG7PZ9cq/hlnWd/sbSfTIHItvpaIj
znJ/BgqzFDlkWX+Ei/g90kXcu4DWG3i0rM9O1dY1Njm3kXq0tytUVkD9l+kZg0fbrIRUMoyB
uzSX7KewhlluWCTLW70W6oFjpbzea+5fvl2yNWUGZX4YZd7gLKjybGKFd3vXYXObsXYFTAlq
/ISwtwXrV9E3V4bC5J2//U0w/OZb4D5Ii7Cr9MtK8nIWyDkrK9td7QYHafGjrWjKBiWS8yd4
dxOOZQLb0jls+tXR3InVOE4MH3SlGEfXPCENs85viru2rS156H2xx8ffnJmqzwlwx19DiZgp
E9dZ26ZflvJv6xtkdHgiqJ4m4rW2o7pxPJr0dJHBQ3EPhr56OTz0Lxjr+r8oMfVXugSalreG
uhgbmGnr+DHDbO7eL7xvYJgT3ms+MfJK1pfy5BVIe5E4Gjts9+ug/dYUpya+w7n5bTD09c2/
ygw91sE6+2qzLxV95qnA+sWaWK7XtvsQBxmMsbIFPA7gaXDIaeiWZ3MjlrNfNb0K8E0uafDs
whraO5x52LMUewksMI+B8tg2/bJkUsv93y4fSA8rodR5aPPkFjTMnIhCH/E+xv04RT6wZx30
UzD0DV5fcCzhtPWV9hlmZ4XsWFk8H6hswfK9E9nyPUepwixAw9mQy5bSqyPTzDWFKpTxnUZQ
/wGDqXv7q8zX1Wpjk6JaBfT6kHYOoUR3I2l5fCu9PqkbvfaxGQU/Uugvix6jg7O70cuGH4w8
01IMeIc3Yw1LquVeLWJKobF6rSL65+whNNLc4l98TaAnps+lDueX0oOj9HUk0l1bFxg99MXy
tyjrz2O83G5EWTJmp2NtI3oi46oKR7HIsvC94hEIboCwl2P9JGxkpbjtibN9U6/Xzb+xuNWv
pYe3d/I4RgkW7ZZQZGoXrYInhXQ7M3gFcewCn7fbpkhVKgmxJfk0+8kkmqPyFmbQsjMfozPL
h9BTi3QFVYYXA8XZ7/PoKbg6ynWdWimTJ9G8gVfS4tSBJt+L9fw942PJNkMGN+dN4jzkUqzB
/up7fGcWJaVm6qCIOmC9bnehIOBc1Odvtta0yfeirD99mEl3P2GtCxORuwxfSA+3+pH6p8n+
AQMnzqdu1dAQQk7spr9mZtBzi6QKbmSJo5cmPzHR1nf9tBHHSkm3TmfdbOsnFfo4Nv0ysYXC
rJ7DbL7XFGai1dgNeZhhOZTrS01lFmV+LX0ITV1mGRS0Uvsos3A5PfXVKsp+cSDlbDLqptwe
Gb+cUlNwNjPamLgi1iV7BRQiaYSVxii0u68X96MBWZK2Fj/ZlUzrvc5PRfSP2S/S4OedM1AJ
NHziC5R6Vwd9aQLwOrcPCrO5ay3obpxjLNI5+RE8S4bI9mjPQ8xwJcsZLuk9YfKJh/694i2a
PXMMfSSXPWjUgWvnw9k08uFUaqlsWFUHLqltTJdUpe2hDKYxzYhP5HQx1vlB2wAL+fs719iM
XgE/ytuf7vl0Lc3P7uezPaeOXkj9e6dQU6M/FcWt2xR9g9hkTbYPnHtr9qs+MfJVSQ99/rd5
9NLTGT5kAoRHPzr5yVG2/kIqTr5Sc/O7YOhr845y9q0GWDalGu+cfSYF1C96KP+v9iUOIhuZ
ptiYr/6prTToemvTS/W4tmqOo9zkEieRhrqkwbMvnxbMzaLnFziNPMk4Jmsa9VRkY7G532ev
lNLvi8Rt4wzRNGxg2OUKY9IGRgPVqCCMvdkPYpa5go02wdBXFD04fcFDGzOVmX5prDT7Sj1l
X/8rUrbgXbJ9IRzcuyhWmHXAjv4ijpY6qB3BUqsO3EOwC02DBo2w8YwdUGFxVc+FO7IXR0Cc
wBmX1RtRg8a6tVzEqIvOquiHg1R8Hms7a+FbA+ubPUV0FFDAG4t109ikq2BPEVR4nEmKvBsY
eUsB8NjPRbT/KHZvqY5NW0TZDAHRmZ4plOJDEep1sriIjsldX1DOpko5tbiwRF6CAdlr6ZMz
YX6OWASCHyAcVQZfHMa+TCXqa3T8jbFOy9kcz57AVgAnrYDaAI4B1fnerjDr4c8XFtF3h7zb
ujZ7BFY68AM2gNLOlr0IbxnKm8jzp70Htc2latVBnLr++VEq/yePoE/AjvnaVQZB3FkvWd9j
h+ByKhcYIjFzx2CkfRh5SCylFfgk6v0f1F+s3vLVB+kFEglBIALuoibncXTU0YNFdAjnN4vL
2R+Jd+psUWl1c9bDpI+f8pK/9yJTXKIPOmdUUu2bRJkL9xdRsSgzGo+vftLECuloZUZfJuuo
4dPYux8UykWdMHdm5eE3f7Q5UmitUxW4SmXGSS/Z7sSi1iMYMwqP6PiKcaMpxg0boypCm990
JPaSF/Bs5uH85uATbfzBMUnHZLtsgDPBHWOqPG4KSZWpb6BiuxJAyqyZfVdtkWJorvLQV5bo
2GGi40cNbETn6WdcV/ERcW3tA88qLcT3i12SJh70n2cgnwg+UuUNGd8p68j30XAvD31t9HHw
g8DuAvq9I8ftKMq+X7wNtF8sdazE+Ccu7PVHJzF2mxf6x8boJ0Wzs5UXz+o4IcNrxxMe99Cx
Y6fRZvRxuSn6XtuFNNVjoPz1+zKOs79RMSjGmHFWDoyIUBOeXw0VA5tMI9h7eegr8gxWX1Ax
EeN3WY17TqwkzwcqWwSLF8ezIxD1CrOAQ2zQKhRlhU8tlNC4a2CgF+fI2a6z6PwQ74IjkhAy
a0PAKIag7vwmdq+8gHzke1WIxDHQdErt2JCZKTRCACqCgH1OEb5FWYSQUBP5e1AWcanpiefj
R1EvfHMUUXzSNhZzylDaB/7nKgTKO0CoYBRDATyrtEFp0FHDiN/OTl6ei3oePHFUUaRNhcyR
gBAodYHO+iCVhAunwQsQOiQPyRDiDF5YnMzyyfDi+6lj+EM8Gx+Ar4VR7CziyPfSgHUSfHPK
4Cl1DZbMy3n3V195ZqMMb5bJoWia7xHwOPjcoygoIq4QZISRzKP0D1K5F99Fn1KMOp5XaCPe
i0v0OQ3Qd0k9srS6+aWPv/L6e69nTWp7cfZNwoJxtBhlluAbcWSZnf2tRkNBKx/hq9WAoRKG
R7H7b7iv8vCbkzbSGOXkB8lP/tqdxMAZT74XeNW7BG3KeOFMR+ySXc/A8hR44aTkBYSXeYuo
zvI6N4nyN66K3WBrg151ZQGQlv+256EdX+7CgHWUVjzdi15X1kKPwd4i94m9RXCp/KC9CNG/
8tBXLVJp7Vnwex3gUwdGRvWy4Y0PKi3UcKX9Lk0mEPFEf+HkvdLSc9u38tDXRh+ML05+EFid
QP9+Wunf1b5fwzLAftEXj9vSRJ95GP2s7DZVw5KT733tGyDKdBaK/gmM277GF9h6qLFDNi61
30d6Tl5XDUNOo4IvJV7DKch/5aGvzNJfv6Z996MvOBVmX21Dpq/enTSSPB+MbKGmy7+DQ4AV
ZgU3MRl7HpbXahiwLkD4FDtU1nIMWkpw7acWB2GrCUbBjDGimtcZCOlCIBQ74NVExyKFVjOA
8wc6UqH8CrlXCDWxjrzP49sZ0dkaTFnzognqGcgyak+IG4tyljGqngD/j1gEKmKACEflz0Nw
0JRm0dbBCzUNAV6WRSi6woBUZt5CRA8Gfo23wKSxirAu06wyd/C4VNjL0gfJcpuY4YXoj2pG
AJ9rdBSzXShvWWip1lHUWwj4Ze0HRfhQX1WN3y5gzNAMqsBXYCyMRNUrufMXRzWe02Y0QWMf
41ppNHEe62KFlefG4g3qU1meUhVNX7X9BzquW1gE/kvQRMg40vAYKF0CzzEyYlQ0fYOttdou
LtYvmn0i+NvXWBlsGbziCfkUf1qbEX2JQ971Cl8FX1QkfYPRF6ogJFykABBghTkAsDgoIxBp
CFTkABFpdefyMgKVjQDzWwUjDjfsu7EWc4ctWfvGYdIzxBYkRA9M3xABW0WSZfpWEUKEqBhM
3xABGyXJssIcJYTmakYnAjxARCfdudbhQYD5raJxh0v2ug9pF9bpHz1emy69Io4S2iWYG72J
3Oo3g8dIRWfrJz2mrx9gXPKa6esSQvqpBtPXDzD8ukwIsMJcJpg4ECMQmQjwABGZdONSRyYC
zG8hohuWYBzCWkznpa5/dH4LxTPTNxSoVp00mb5VhxahKAnTNxSoRk+arDBHD625plGIAA8Q
UUh0rnLYEGB+Cx30YrMdsfGQ2MBI7DMSi91zK3ujN6Zv6OhbFVJm+lYFKoSuDEzf0GEbDSmz
whwNVOY6Ri0CPEBELem54mFAgPktDKBXYpZM30oEOwxZMX3DAHolZsn0rUSwXZgVK8wuJCpX
iRGQCPAAIZHgOyMQegSY30KPcThzYPqGE/3Q5830DT3G4cyB6RtO9CM/b1aYI5+GXANGwC8C
PED4hYY/MAIVjgDzW4VDWqUSZPpWKXJUeGGYvhUOaZVKkOlbpcgRcYVhhTniSMYFZgTKjgAP
EGXHikMyAuVFgPmtvAhW7fhM36pNn/KWjulbXgSrdnymb9WmT1UvHSvMVZ1CXD5GoBwI8ABR
DvA4KiMQIALMbwECFmHBmb4RRrAAi8v0DRCwCAvO9I0wglWx4rLCXMUIwsVhBCoSAR4gKhJN
TosRKB0B5rfS8Yn0r0zfSKdg6eVn+paOT6R/ZfpGOgXDW35WmMOLP+fOCIQUAR4gQgovJ84I
2BBgfrPB4boHpq/rSGqrENPXBofrHpi+riNppVaIFeZKhZszYwQqFwEeICoXb84tuhFgfnM3
/Zm+TF93I+Du2jH/upu+oa4dK8yhRpjTZwTCiAAPEGEEn7OOOgSY39xNcqYv09fdCLi7dsy/
7qZvqGvHCnOoEeb0GYEwIsADRBjB56yjDgHmN3eTnOnL9HU3Au6uHfOvu+kb6tqxwhxqhDl9
RiCMCPAAEUbwOeuoQ4D5zd0kZ/oyfd2NgLtrx/zrbvqGunasMIcaYU6fEQgjAjxAhBF8zjrq
EGB+czfJmb5MX3cj4O7aMf+6m76hrh0rzKFGmNNnBMKIAA8QYQSfs446BJjf3E1ypi/T190I
uLt2zL/upm+oa8cKc6gR5vQZgTAiwANEGMHnrKMOAeY3d5Oc6cv0dTcC7q4d86+76Rvq2rHC
HGqEOX1GIIwI8AARRvA566hDgPnN3SRn+jJ93Y2Au2vH/Otu+oa6djEluEKdCafPCDACjAAj
wAgwAowAI8AIMAKMACPACEQaAqwwRxrFuLyMACPACDACjAAjwAgwAowAI8AIMAKVggArzJUC
M2fCCDACjAAjwAgwAowAI8AIMAKMACMQaQiwwhxpFOPyMgKMACPACDACjAAjwAgwAowAI8AI
VAoCrDBXCsycCSPACDACjAAjwAgwAowAI8AIMAKMQKQhwApzpFGMy8sIMAKMACPACDACjAAj
wAgwAowAI1ApCLDCXCkwcyaMACPACDACjAAjwAgwAowAI8AIMAKRhgArzJFGMS4vI8AIMAKM
ACPACDACjAAjwAgwAoxApSDACnOlwMyZMAKMACPACDACjAAjwAgwAowAI8AIRBoCrDBHGsW4
vIwAI8AIMAKMACPACDACjAAjwAgwApWCACvMlQIzZ8IIMAKMACPACDACjAAjwAgwAowAIxBp
CLDCHGkU4/IyAowAI8AIMAKMACPACDACjAAjwAhUCgKsMFcKzJwJI8AIMAKMACMQWgSWLl0a
2gyqWOp9+/atYiUKbXGYvqHFN9ypM33DTQHOnxHwjwArzP6x4S+MACPACDACjEDEIMACd8SQ
KqiCMn2Dgi1iIjF9I4ZUXNAoRIAV5igkOleZEWAEqjYC+/fvpwsXLmiFLCkpod/85jdVu8Bc
uiqBgBS41x1Zp5XntSGvVYlyVXQhZD2jdYbZ7fVm+rrbcyLS6HvkyBES43BMTAxdcsklF+3O
jh07RmfPnqXq1atTo0aNLhrejQHOnz9PRUVFWtUEZgK7SL9YYY50CnL5GQFGwDUI/Pzzz5SW
lkZSoJAV69ixIy1ZsoSuvvpq+YrvjIAXArLdsMLsBY0rXlX9SNYAAAObSURBVEj6ssLsCnJ6
VYLp6wVJ2F+cOHGCEhIS6Pvvv9fKsn37du25tIINHjyY3njjDbr11ltp/fr1muJcWng3fsvL
yyPZT23dupXat28f+dWE1SQqr8OHD5ccOnSoBJagMtUflhIt/NGjR8sUngMxAowAIxAIAhiQ
S5o3b16CUcXnX4MGDUp27NgRSJIcNsoQePvtt0vE35DXhmh/Za3+vn37SnJyckr69+9va4P3
339/ycaNG/0ms3nz5pLExESzvT799NMl/sbILVu2lDzzzDMl7777rt/0yvpB1rOs4d0Srjz1
hsdKCYxuNvreeOONJWvXrvULD9PXLzQh+cD0DQms5Ur01KlTJVCYzT4uLi6u5OTJk6Wm+fjj
j2vh77vvvhLMtJYa1q0fYfwxMYPC7IpqCjeDqLuOHz9ecs0112jETElJuWiDFgzTunVrLTws
RiXnzp2LOsy4wowAIxA6BEQf06FDB3OAGT16dElhYWGJUGTGjRtnvhf9VnFxcegKwilHNAJS
4A5UYRaCnT9DjXg/atQor3FSKFoyjjDmyN+ijToFyjNnzpjt+x//+Ee5MZb1LHdCEZZAsPWG
e2jJ0KFDTRpJWsn7Sy+95IUE09cLkpC/YPqGHOKAM3AqzIJnsrKySk1HKsxl0S9KTSiCP7LC
HMHEU4uuMkBZLECBhlfz4t+MACPACFwMgXXr1pnC7KBBg7yUk4yMDPP7ypUrL5Ycf49SBKTA
HajC/Mgjj2jta8KECdqM8v/+7/9qM4/SsCyExA8++MBEVcyaJCUlaXGEsiVmLw8ePGgalp1t
dMWKFVpYIUBWhMFZ1tMsUJT8CLbeCxYsMPuPKVOmlBQUFJTs3LmzRNJd0Ff1JGD6hqdBMX3D
g3tpuaryvzQwiTtcs/1GY4W5pIQVZr/NI7I+qAxQFgtQoOEjCw0uLSPACIQbATGjLAZhMVMn
XLOd1969e02BVwi5QkHhixFwIiAF7kAVZuEmLZYoOS+xdEl6V82cOdP8LLy0xHuxhEAoyvKS
ipkaVh0/K8o1T9ZT5hst92Dqrc7ujxkzxgaVUIx79Oih9S2PPfaY2a8wfW0wVdoD07fSoC5z
Rmr/pSrMpblml0VhFobDH3/8sUQYJ7/99ltt3BeeIP4u8U1+Fx48X331Vck333xjW1YqvqsG
SWEUy8/P18L98ssvXknv3r1b+y7CYP8Ur+/OF8K7TZRVprlnzx6/S3DcqDD/HwAAAP//guTa
mAAAMLtJREFU7Z0JvFVT+8dXhvJveMtMMr5es4yJEhlCevGiKN5IVEpFUUKh0iiKBlNJSFQa
36REJEqTSgPJHImXJpSK89+/1ftsz1l373POPffce/e557c/n3v3sNZee63vs87a61nrWc82
sRzctmzZEqtatWrMGBO77LLLYn/88UdCCvmNnzAxBpIACZCAIqDbl5o1a8a2b9+uQnce/vnn
n7HGjRvbNmu//faL/fzzz3ni8AIJjB49Ooa/5k81t3+ZIHL33Xfbejdo0CA/Oamzbl184okn
bNxnnnnGj4v84F3bsGHDpO9a/6YkB1LOJNFKXHA65f7iiy8sf8hg+fLleZhMmTLFhlepUiW2
adMmG0755sFUJBco3yLBnK+HyG8Bv5/OnTvH6tSp4/+eHn300cC02rRpY+OE6ReTJ0+Ooe1E
mvoP1yZMmJAnzdWrV9t4FSpUiI0bN87XX+TeOXPmxMaPH2/jHHHEEbGPPvooVrdu3bi0Effm
m2+2/YsffvghrhySTseOHWNbt27N8/zPP/88JmWSuHrftGnT2Lp16+LuGzNmjP/8+fPnx4Vl
64nJ1owXJN/6BxBWoXX6yeLrDu5vv/0WW7x4cWzp0qW20q5fv14nxWMSIAESiCOwcePG2HHH
HWdfLj169IgL0yfywkLHFvdwIwGXgHS4M6Uwb9u2LVatWjVbN4MUZrcuSidJ4v7yyy9+3V62
bJmb3bTPpZxpJ5ClN6ZT7kWLFln5uYMbguDTTz/1w9GRxiZ9HspXKBXNnvItGs75eYr8FqAg
Pv/887EVK1bY34sojEuWLMmTnLyrXf0CA9/t27ePu1/S0XsooFqvWLVqVdw90l+QexYuXBiT
tleuhe3RDgQp6xK/Q4cOceWBLiNhifa1atWKy7PODxXmOKTZdaJ/AG6FDipJovgy+o4K/t57
7wVWLIxC7dixIyhpXiMBEshxAitXrvTbDT0z52LRL6BPPvnEDeY5CdjZZXS6M6Uwi7KFjtLU
qVN9wloR1rOW8j4UhXnYsGG2bmNmA51FbLCOWLNmjf2Ta37CKR6ko1ikmHSko6VT7pEjR1oZ
uMqvFFR3xqVjS/kKnaLdU75FyzuVp+n+v7RrvXr18t/ZRx11VAwDi3oLU5iHDx/u34c2FfLG
JBt+b6+++mpcmDwL6erfKO7DTPPQoUNj999/fwzPh3Kt+weIg4FOTN5h0u65556LSxvhF198
sQ3/6aef4sJ1O4H2+YorrvDv7dmzZ+zrr7+2s9AbNmyITZo0yQ9DmtJ+IM86P/o6wrJ14wxz
AU2yW7VqFVdhUGmC/q655pq40ZdsrTDMNwmQQGYJ6Jehfkm6TymJLyC3jDwvGAHpcGdCYf7x
xx9jMO/D+wx7dOz0Jp1CdMzmzZsX051BKNHoUMn9n332mX3/derUKe79iM4ZBozyu0k583tf
tsdPp9zSbqAPErT8TCsE6GDLRvkKiaLbU75FxzrVJ+nfh7yfoeDKsk60j65ptvx29IQcBgql
PcQ9CxYsyJMFPZuLWeC1a9faOLqPgHsxoyybTMbJ7xzhUKLd9loGMxGOpV+uko+ZZYRBGYcJ
Njb93LD2A3UW9+FPK8Y6P/q65Dsb91SYC6gwyw9DKkyDBg18W36YaugfSKLZo2ysPMwzCZBA
wQnol5K8kINSLYkvoKBy8lr6BKTDXVCF+ddff/VNsfFu07PIkjuY77qmgYgrnccBAwbYTpSY
+D3wwAN+h6x3796x2rVr23MozbJ2VtJOtpdyJotX0sLTKbe0G2EdXq0QaDlTvkVfeyjfomee
7In696HfzxgklH4/9to0W/QCrTBrSzKEh23STiJNUTR1H0Fb6+g05HeO+15++WUdZI9ljTPC
33zzzYTh8lws/Zo+fXps8ODBsQ8//DDPPbigyyX34brOj76OsGzdqDBnUGGuV69enllk7XAD
yvPmzZuzta4w3yRAAoVAQL8M9QvZfVRJfAG5ZeR5wQhIh7sgCjOUZcxAoGOFv2nTpoVmCqaA
6HyNGjXKdtJkqQBmpzFDgtkKmPCh0ynK9dy5c216uCbro7W5d+jDVICUU13KicN0yo17IEd3
jaEA++abb3xZw3mQ3ihfTaPwjynfwmec3yeEKcxIRyu32jQ7SGHW729pA4PyAsVU2l60q9h0
HyFs4k2nry1F5Bk6XM9QB4WHKbgw3/74449jb7zxhlWimzdv7ucVedb36efp6/K8bNxTYc6g
whxWKbp06eJXqrA42Vh5mGcSIIGCE9AjtGEvQzxFjxDrmaCC54AplBQC0uFOV2GGoqtNDV9/
/fW00MgaP6yxwyZ1XK+Pw3XMRqOjJbPSuJbKJuVMJW5JipNOud9++23LGAMYQd71xemXNsVM
xozyTUYovXDKNz1uhXlXIoU5zDQ7mcKcSA/QyrEMoOtrYX2EZApquuFYxzx27NiEjsJEwdfl
Sva8wpRZYaVNhTlDCjNmj901AyK0iRMn+gpz0MiPxOOeBEgg9wjotU3oiIZt0kl1lY6w+Lye
ewSkw52OwgxrKL2E6N13300LIBx6oQMVtAZPmygi8ZdeesnGhcOb/GxSzvzcUxLiplNu6WzL
bL/LQZyVhinUbnzK1yWSuXPKN3MsM5VSIoUZz3BNszEDK74adHuXqgIpA1hoQ4MUZq2U6jIm
Sz+dcFiYaKdfohhjD6dhffv2tY7H5LrOW7Ln6bxnyzEV5gwpzGHrg1AR5IWFSjVixIhsqRvM
JwmQQBEQ0C/kMLNJOOuRNZ+pdmyLIOt8RMQISIc7vwozTASl04P6hRnhdDdxLqNnjfUMs16v
3L9/f/tcHTeV50o5U4lbkuKkU27t8Tqo/yHyQudeHAglYibxtcwo30TEUg+jfFNnVVQx9ftZ
FFj32do0+9RTT/WXtIQpzO7SB52eDGChPZaBRK1DaKVU35dMQU0n/LXXXvPfCxhwe+WVV6wj
MjCRDZ8LlHeHNvVO9jy5P5v2Oa8wN2zYMNBzpBai/sG4irGYXrjX9f36u235Xaul0+ExCZBA
ySQg7QhePPAo7G7fffed/1JKpc1y7+d5bhCQDnd+FGbtZwP1D50kzCJipkP/YeZEK7tBRFev
Xm3rqTuoA+cxMnstznHwXuUa5iCK4ddEvuExgkNEyUWnV6+fHDdunN+u4BuzyTbKNxmhgoVT
vgXjVxh36/5/mMLsmmaLAqkVZhlUQlizZs1Cs6r7AmKRWlwKs7QbyHOQozAUYsiQIX4bgk9d
yUaFWUhk+V6PuKbiiEt3KNxRWKncQW7aBZOuOHoERsK5JwESyG0CeiS3Y8eOcTCwhkg++YAX
F759yI0EgghIhzs/CrPujElHL2yfaGYE9VQ+sxi0zk46X1Da4HW1Tp06tqOVzhIDKWcQg5J8
Ld1y4/M0MmAB2UIGWsbov+hZoyCGlG8Qlcxeo3wzyzMTqaWiMOM5rmk2fl9aYdZLrxA2a9as
PNnTaaBdFJ8Duo0uyhlmac+RX/hCcDc4DMTgqLQlekBB6z1heXbTi/p5Ts4wo+G/+uqrfSEn
64DC9EgqhLvGUBRmhIuHUC10/ZJBHBkx0nF4TAIkkNsE9CAe2gk4S8KsMrxSduvWzW9/0OlN
NsuX2yRzu/TS4S4shVlmh4MoQylD5ymsjsL7NmZW5F2KPeLDAiu/m5Qzv/dle/yClBvykUEK
LYPWrVsnVZbBjfIt/NpD+RY+4/w+IVWFGelq02z8xrTCjPAXXnghrv0bNmxYDI4W169fH/cZ
JtyrBx2LS2EeOXKkn1+sWYb5NdY1b9iwwToC0+2Im2cqzJB4CdmGDx/uV4REL+3Zs2f78VAh
3BF2rTA3bdo0zxogPWIUtj6xhCBlMUiABApAQJtsuS8inGNWiN6xCwA4B26VDnd+FOaixoKB
IJh6f/nll0mXQ4XlTcoZFl5Sr2ei3PjMF/jDxB5KcKY3yjd9opRv+uwK604ozPJJPD2DGvQ8
d+DbVZgxgda5c+c4nSLoXQ+9QvsTKC6FGd9i1zPIQXnV1/CNaPhbwUaFOaiGZOk11zwCQsdI
K9YYQ0nGSJA42ZEKgXXKuhKj6FphRrwmTZrYjgBmhrBgX+7FfsaMGVlKi9kmARIoCgJfeJ6K
9Wd9pP0488wzYwjjRgKJCEiHO8oKc6L8pxom5Uw1fkmJlyvlzpVyuvUyV8qdTeXUM8x6ja4r
OznXk2Rh/kZgji1KuLzjsYdPh5kzZ0pS/h7vfokXtqxzypQpfpyggXUdjplidwsLh9l1gwYN
/LQlH1CkMQP9+++/xxo3bmzDYV0kXwvS6SWyTHLzEeXzUsicByAnN29Gx3iVNqWyexXBLFiw
wOy5555x8du2bWsGDhwYdy3oxDPrNu3atQsK4jUSIAESiCPgvSBNqVKl7B8CDj300LhwnpBA
EAFvVN9enrF+ht0/1fypoGhZf03K6XXksr4s+SlArpQ7V8rpyj5Xyp0r5XTl6557Fh7Gm3W2
f6VLlzb777+/GyUy557puPGW1RjkE2pj5cqV/f5JZDJZyBnJaYUZbFEJnn32WeN9Ny0QtTeK
Yvr06WMaNWpkypQpkyeOKMyI55lr2HQ+//xzPx6uv/jii8ZbO+Rf4wEJkAAJkAAJZJqAdESp
MGeabDTSE/mW9IGCXCmnW6typdy5Uk5XvjzPbgI5rzCL+Dy7e+OZURvP/MLssccexluLYCpV
qmT23ntviRK41wozZoUw+uKtETLlypWzaR1yyCFml112CbyXF0mABEiABEggUwSkI5qp9KKe
TklXHF3+lK9LpGSdU74lS54sTckiQIW5gPLUCrPnRCOPyXYBk+ftJEACJEACJJASAXa4U8KU
tZEo36wVXUoZp3xTwsRIJFAsBKgwFxA7FeYCAuTtJEACJEACJEACJEACJEACJBBRAlSYCygY
KswFBMjbSYAESIAESIAESIAESIAESCCiBKgwF1AwLVq0ME8//bSBcy+aZBcQJm8nARIgARIg
ARIgARIgARIggQgRoMJcQGFs27bNOvdCMhUrVixgarydBEiABEiABEiABEiABEiABEggKgSo
MEdFEswHCZAACZAACZAACZAACZAACZBApAhQYY6UOJgZEiABEiABEiABEiABEiABEiCBqBCg
whwVSTAfJEACJEACJEACJEACJEACJEACkSJAhTlS4mBmSIAESIAESIAESIAESIAESIAEokKA
CnNUJMF8kAAJkAAJkAAJkAAJkAAJkAAJRIoAFeZIiYOZIQESIAESIAESIAESIAESIAESiAoB
KsxRkQTzQQIkQAIkQAIkQAIkQAIkQAIkECkCVJgjJQ5mhgRIgARIgARIgARIgARIgARIICoE
qDBHRRLMBwmQAAmQAAmQAAmQAAmQAAmQQKQIUGGOlDiYGRIgARIgARIgARIgARIgARIggagQ
KPXTzxtiUckM80ECJEACJEACJEACJEACJEACJEACUSFAhTkqkmA+SIAESIAESIAESIAESIAE
SIAEIkWg1OYt2znDHCmRMDMkQAIkQAIkQAIkQAIkQAIkQAJRIECFOQpSYB5IgARIgARIgARI
gARIgARIgAQiR4AKc+REwgyRAAmQAAmQAAmQAAmQAAmQAAlEgQAV5ihIgXkgARIgARIgARIg
ARIgARIgARKIHAEqzJETCTNEAiRAAiRAAiRAAiRAAiRAAiQQBQJUmKMgBeaBBEiABEiABEiA
BEiABEiABEggcgSoMEdOJMwQCZAACZAACZAACZAACZAACZBAFAhQYY6CFJgHEiABEiABEiAB
EiABEiABEiCByBGgwhw5kTBDJEACJEACJEACJEACJEACJEACUSBAhTkKUmAeSIAESIAESIAE
SIAESIAESIAEIkeACnPkRMIMkQAJkAAJkAAJkAAJkAAJkAAJRIEAFeYoSIF5IAESIAESIAES
IAESIAESIAESiBwBKsyREwkzRAIkQAIkQAL5JzB18vj835TFd9S97Moszn3+s0755p9ZNt1B
+WaTtJjXXCNAhTnXJM7ykgAJkAAJlEgC7HCXSLH6haJ8fRQl8oDyLZFiZaFKCIGcVZg3bFhv
YrGYKVWqlKlUac+k4vxl82azfcd2s+suu5q/VayYNH5xRpCyJcoDylG6TBmzxx57JIrGsAwS
2Lxpk9nxxw5TevfSplz58hlMmUmVVALbtm0z33+/1pT36stee+1dUovJcmWIgHS4S/rMa66U
060WuVLuXCkn5Zt9FiLox636dJX5cd06Kz7oEPsfeKA5/PDDTcWKlVyR+ufS//MvOAfok++2
226mbLlyTshfp3/88YfZtGmjfyE/+ojWC8qULpPwOf4DeBBHICcV5t9+/dVUr3aK+fKLLyyM
OfMWmRNOPDEOjHtyW8vm5vnnhpsaNc82r02bYXbddVc3SiTO3bIly1T5ChXM/Q90M01vaWbK
eAo0t8IhsHXrVlOrxhnm45UrTd16/zQvj37V7LLLLoXzMKZaYgg88nAf8+D9nW27M3X6m6wz
JUayhVOQXFE0cqWcbi3JlXLnSjkp3+xRmH/77Tfz9JNDTJf77nHF5p+3u7ODaXtHO7PPPvv6
13Cg+39xASEnLW9rbVrd1tYc5inhepsw/lXT+LqG/qUj//EPM3/RUqto+xcDDlZ98ok57eQT
/JAevfuatre38895kBqBnFSYUXnPO6eGWfbRR5YSKt37Hyw0//d//xdKrcOdd5gnhww2V13d
wAx//sXIdlzdsoUWyAnAD3OuN3DAmU8HTIZOMZBxbq2zfIV51CtjIzvokqEiM5kCEpj51pvm
8nqX2FSi3u4UsKi8PUMECqJowOLq1bGjTcc725sff/zB5ujkk08x3Xr0Muedf0FgDhfMn2fu
ubuDmTvnfRuODuNdHe4OtMJauGC+mTxpojnrrBrm4rqXBqaX6sWClDPVZ0QxXkHKTflGUaLx
eaJ843lE5ey///3R1L3oAtt/SyVP7iRcuv3yKa+/Yc45t7b/SFdhRsDij1aavx95pB8n6GDI
oIHm7g7t/aBH+j9umt/a0j/nQWoEqDD/j1PPPg+bNm3vCKUmCnPUZwf1D/Ogg6qYES++lEcx
22333c26tWvNCG/GfPKkCX6ZH+ja3dzVsZN/zoPMEfj999/NP+teZDuWNzS5yQwc/GRkB10y
V2qmlC4B/C6vu7aBf3vU2x0/ozwoVgLpdrh37Nhh7mp/uxn2zNOB+e/Vt59p3eb2uDA9oANL
JSxbwobB13kLl8QNQG/fvt3UOe8cs3DhAvOfqdPNubXPi0srvyfpljO/z4la/HTLTflGTZLB
+aF8g7kU51UMNLVs0cyMfGGEn42+/fqbWueca/baey/z6y+/mlnvvG3uaHubH+5OQOl+OSJh
hrdy5cp+fBz8svkX8+mqT8zjj/X3r++7735m0dJl/rLRIIV50JCnzI03NfXvcQ902ythVJiF
RP72VJgVL3dUSAWZbFSY6ze41jw74gW7TluXRR+/PGqkada0ib3k/jh1PB6TAAkUPgGsM+r5
UDfzxOBBcQ+jwhyHgychBNLtcI984Xlza/ObbaoPdnvIXNOwkdm6ZasZ0L+fXYqEgGkzZtql
ATj+888/7QDgu7PeMVCmb2vd1vz003/9WZjRY8fbpSeIi+21/0w21za4yl7LhHVNuuXcmZvs
/Z9uuSnf7JA55Rs9Oa1b97058rCDbcagCM+YOcvsv/8BeTKKeJgUwbI7bOMm/sfUuehie6wV
ZkxkLVq63JQtW9aGuf+++vJLzwK2pm/lA4tW9OWxuQozLID28CxjsVwL65+DthXLl5vqp58c
F0SFOQ5HyidUmBWqRKbZqSjMWJC/du13Ztvv26ySijWqB1WpElqRMeqLTSo6RujXejO/6IzA
PLzyQQf5YSqbCQ/1DzOVTrbu+CRSmFG2b77+2vzqmRbv7s1Ql69Q3hx4YOWEyjgyCgVg7Xdr
7WwqnCPss+8+oc6LXB5r1nxjNm7Y6eCggjeDcfAhhyR9HtaZfPftt6Z06dIGs7rlypfzRvIO
CmWGZyJfsiYdpjc///SzjZ9KGfG8b9esMRjFgxwT3QOGGK0UeYdmKp8Bwhhpw4nbPvvsE2gS
mc9kGb2ICcx+d5ZVONzH4nd5+hlncN27C4bneQik0+HWMxB3tL/LdPdMsGXD++Ga+leaaVNf
My1atjIPPzLAtpeyxOSn//5k5i1a7K/ZE8VMd8j0O+md9+aaU089TZJPe59OOdN+WIRuTKfc
lG+EBJgkK5RvEkDFEKzX/yabzdVWN9oqR7eBifrZUrzRr4wyNze5wZ7qttRVmCV+IrPsAY/2
y7PuWqcpaXCfnEDOK8wdO91rsA7rrTdnWFq6kmt8yRTm11+bYlrd2twfFZJ78eMYOPgJU++y
y+WS3X/91Vfm+GN2rjt4653ZZtHChdYkLi6SdzL4iadN4xubJFUU5T79w0xFYYaSBScCEyeM
M2E/5Pffm21aNGvqO0mTZyF+74f7mQbXNMyTv2+/XWO6d30wzoxF7ruwzkWmR6++5rjjj5dL
RvPATMYb06eZfn17++E4OO20081TQ4ebo485Ju46TqAcDx821FoCuIG47zFPBiedFD/KJrMe
GDV8dfxk89iAR/zZFEkDZXx+5Chzdq1z5JLdJ3IAgXueeOqZuHV6Wi4YLRz23PNm0sTxvgMH
d0ZGHgb5tG7VwubLlQ/y0KdXD/Nov74S3d+jznS6t7M55JBD/Ws8iDaB8ePGmhuub2QzWe2M
6gYv51YtbrFmrKn8lqNdOuauKAik0+HWbS8cyBxz7LFxWZ32+lRT/8rLDWZGFny41BsUrGCd
2MAPyLrv18WZDMKkG6aJumMp9VravUw4O0ynnHGFytKTdMpN+WaPsCnf6MlKK8zugKKbW/R7
293exhxx+BG2/ye+H3T/z+3HuWng/JOPPzann7LTEbFWbkVhRhvc+f4HzSN9+1idQ7e3Oj39
XPQH35v9roFVkE5Tx+dxYgI5rzA/PWy4N+J9ul85gSvINDtMYYZCc1+njmbg4wMSkoYC8/ig
J/zZxc9WrzYnnxjfMQlLID+VW/9AUnEUpF+mQT/kZ5560rS/o01Y1ux1eNt7qFcfX2nWJiyJ
btSds/zwmP7m2+asGjX9pDHDCocM4sTND3AOhjz5jB18kMvS+Mh5ov3MWe+Z06udYaPoWflE
92hTGpmRES/ZMEuE6c1JJ+xU/sNkpVkiDkzsMRuOTw3VOf/cPIMYOj9oVN+e9X7gAIOOx+No
EEB9bH97W0/ZeNJccmk9A4uEGtVP8x3F0bN6NOQU5Vyk0+FesmSxOfvMaqEDpp9/9pltp/B+
kNlkec+s/3m956V1ianwt79ZLNKmyjtLt3tY13zsccdlBF865czIg4s5kXTKTfkWs9Dy8XjK
Nx+wiigqrA7POPVkfzIMSjOsbapU2WmmnUo2pL1EHzWon+2mIZM5uC5tKY6lfcUxJtoe8SZL
pkyeZM70HCkGmWXLbx99QXzdp12b2+wAvE4TaXFLjUDOK8xScTBL90CX+yw1mGbj5Q7TY9nC
FGYxQZN4mI28pG49E/NM2WbMmG6ub3iNBMVVfFdBRIUe8PggU/u88+1s6SBPAZd1jJgBneN5
8UacZJv+YWJWKmi9GDriv3uewpct+8jcfNMNvtLlxv9g7hxzoeeoRTYonFdedbU1r5437wPT
9MbGfiOilcOhTz/ljbK1trdhPdz1jW8we+65l1m//mfz3LPDTI/uXW0YnF9hZAwm0S4PRBg7
bqK56JK6ZsuWLQY8und9wN6HmQ7ppGHAoknj6824V8fYMDRGY8ZNMFW92eRfftlsZ2Y73/uX
IzNtEqgbH9yMdJ94eqg56eST7cxJ0yb/9pVwnVc4rql99ln2eVdeVd90eaCrNRff5s1yT5s2
1XKR9NwZGTSYMluIcsvsPmSLMrmNsMzOID1ZE+Mq7JiNhCXCP446yirSGOSQWWfX+YTNNP9F
kgB+l5h9Q73A5v6WqTBHUmyRylQ6HW4x/9Ptqi6Ubpul/dSKsB74vL/zvab/Iw/777oXRjzn
WV41M7r9xAAnrGOwpbKsR+dFjtMpp9ybzft0yk35Zo/EKd9oyko+76hzd/4FF9q+/unVqlkv
1XvttbcOjjvW7/KwdlZucD1yY1Kv0XX/tsG6zwodZenSxeaWm260YR8uXWGgu+itd8+HbH+7
hvc53PHemuoLzqtl+7Si9+i4PE5OgArz/9yrowMglQnYXNPsIIUZL/5aNar7Cues9z8wp5xy
ahz15cuWmTO9bz5jgzI3Z/5C6zBAd0IQ5t6rlSI9so+4iTb9w0wUzw2DwgalHAoWNiiija6t
b0evcI7RLChmesPs51nVTrNKswwyYH2umBAjrYWLl9n1xHIfynW+59AASicaDlEoXR5BnlQf
6vagNUFGWqKgywgaruF578yek2eNNJTpG/99HaJYZfWll8fYmX7d+CAvcxcs8r0RIi6c2FQ7
5SRbPnwGBQoLyicdEMRZuvwTc/gRR+DQ3zDwIgqrWCtouYjCDOVIzB1xs24Yca7rgM6fVtgx
6wzzbnddtB4ACjP3xjO4RZdAWJ2Jbo6Zs+ImkE6HW9rBMCsXXQ/x+cUTq1a1xZR3Ipa8PPLY
QPPxihW+4zAo0fACW/Osavb9+NGKVabKwQfbQU9pG5EI2rVJU143Rx19dL7QpVPOfD0gopHT
KTflG1FhBmSL8g2AEoFLO3bsML16dDd9e/cMzQ0ccN3aqrVnil3X9+kgkXUbimuwkjzggANt
P1viwD8Q/Jh0vOuv7yO7s9HyW8Y9aIsrVqzoL+10lWA9qIm+5TXXNrK6CKwc3biSB+4TE6DC
rL5HphURYBNlB8fSOdDKjl7bcGur26xDFMR1N3i9xY8Nm4zQawUxbG2X/nHIfW7a7rn7w3TD
g85hLt61ew+r0Eu4zh86Us+9MNKf+ZI42GvlURwP3Nayub8WuM/Dj5qr6te3jYPcB3Ni6zis
fAVTpkwZe1k/D4yDZsaxPuSYI3cq9BIH3w0V5wgvvTLGXHb5v+Qx/l47PdENkOaLGVrMgrgb
zNExY6vv02WG3Ju3aGVH9mRmEM7bNm3eZBV3OOHCpuWi69CmjRv9TmUNbxQQ390T5VdMIXE/
1p/c12XnDLvURVzXHVicy4bn1apxhjXn1c56JJz76BMIqzPRzzlzWFwECrvDrWeT3ZkQKbMM
Nsu3P29vd6d5qGdv6/0d70EMzna8+x7rpwLr6aA0y8CppJFsn045k6WZDeHplFvec6kMiFC+
xVsLKN/i5Z/s6VA2R774vIEjrUSb2xfV7/JE97lhEya/Zi64sI5/WX7LuACdAEr6xReeZz9X
6ppliz6D9nbuvEVm/wMO8Lxv1+AMs08z/wdUmJXCDHxauZVZUyh3oqRoZUdXXr3G1RXD0qVL
TM3qp9vLMjOqFcSwBfs6/XQVZihLu+2609386s9WW2+nOn+i5OprONbKGpTFXn0fNlt+2xIX
DR6oMVsga4clj3r9hdyAH3bD6/9tzjmntl3HJkqhhGsewkjCZI9G5xKvcUBDIC9/fIz9ySGD
4xRaia/32nReOgWa77tz5tnGR9+DY5G7Vpix9viEY+NNX9AoNW/R0lzsmZBXrXpSHvN53WDq
OoRniBkjjrVZjTZt14qx5Anx8e3wE7wZH3hmlw2K+2bPHP2ejnfZSzK4IJ7AJR730SaQqM5E
O+fMXXERSKfDLcs+3AE7KYMeqHQtjTDzAkcyP/74gx1QPemkU+zgoVjnbNm6xS412Wefff0B
PHlX6vZclpvIM5Pt0ylnsjSzITydclO+2SDZnXmkfLNDVlg+ha/GfPLxSrsMD5Mq7jby5dHm
8iuutJf1u9yNF3SOSawOHe/JY72o+6zS35ZlL0hH9x+lXyn9P0wcUWEOop36NSrMjsIcZpot
SopWdoIqbxB6rQyKKUTQNffeVNPX9+kfJmau3e8wQ9mrf9Xl/rfiwsyYtXt8nX6iY/kBw5wb
Aw9YPxG2YcYUzsLEWYzmoZVD936RA/INE/KuD3ZJSWEOYhl0Lex5WmFGnP9MnmgaXVPfje6f
Q6HH+m0x19Zy0XUIN2iz8n6PPmYdSsDr98UX1LaDA3rgBo3epRdfaEcU/YclOZAGkwpzElAR
C05UZyKWVWYnIgTS6XDL58zcNk6KJIOnMlNx6GGHSVDoXpaEiGWMWGNhNln8T+DmQQMfswN7
MisdmqATkE45nSSy8jSdclO+2SNqyjd7ZKVzCp8M8z6YaydY5DvM6KNinTE+Eavf5bgPyjS+
5Yy+smwwyYaJ9RF//3ueZYUSJ6jP+sXnn5uqx+9c0iL6hTbHhg6AL9noPEg8SZf71AhQYXYU
ZmATUwZBuGjJcvPiCyPsbKpWdoIqr9yj99LhwDWpqFpBlGv6Hhynmr6+T/8owhQljP7XrlXD
X3tdwzEFRnpaIURHqt5ll5lNGzfpR8UdYybh4X79je5MoYwow+CBj/vOwfRNUARh9g6zZc0j
TGFG43Jnu7bWRBrrqeH1795OHfKYTOtnyHEQy6BrEl/2oqAHdSZhkjh1yhQz/NmhZr7nBM3d
0MGUdeGuXLQDJ20yLsrxypUrfKsEbYGg08HzrvjXVf43pN3ny3n1M88yrVon9nQucbmPDgEt
a93uRCeHzEnUCKTT4Za2F+1VkONBcf4Y1AYGlf+77741R//9sECfHW49HjP6ZeskUc/GBKXp
XkunnG4a2XieTrkp3+yRNOUbLVnBguaVUS8ZtGmlS5c2be9oH7g0UXKtJ9zQns5buNgcfPAh
ccoqBg0XLV1uypYtK7elvA/qs2pfN2KWvXDBfOuwV7fpuj8RpnOknJEcjUiFOUBhRl3Qptkw
J97DGyWaO+d938MxHDbpyuuaqun6JB0OXJOOgbzEcC2s8ur0ZfYW8RNt+kfhdk70fXpWE9fd
PGgl33VGpdNJ9Xjt2u/MSs8pDL49jG91yibl0jzCzPNQNlmXG2SSLZ88kbT1Hh7H4UwBDcjs
9+dZr4ap8E2kMOv04QDu01Wr7Lo8Wa+OcOGaTC7aZPyDBYvNdM/bdpf77rH5dTuxkic0vB9+
tMKOYOq88LhkEEhWZ0pGKVmKTBJIp8OtZyOeGvqsue76xnFZck37klmqSHw9a6xnmPV65cGD
HjedOtyZx8lmXAYCTtIpZ0AyWXcpnXJTvtkjZso3WrLCb6e657T3yy++sE5lZcY4US71UjrR
C/S7PNWBx6BnhPVZdf8RjmhHvfSi9ZukJ810HqRfGvQMXgsnQIU5RGHWI0Uan1ZCpROA8CZN
bzYDB+ddx4AwUXBwLLOnWkEMq7xhPw6kE7bpH4XOa1B8MZuTML2eWedP/+gkruyxfuKVl18y
xx1/vLm1ZWtTsVJF+zmsxYs/NDW8byW3v6ujRPX3iC+u8MXZln6e/gSJf5N3oHmLo7SxY17x
nX7BMdnV9f/6jJfcq5mgsVqybKU1BU+Fr8hOGrnynqMyrFdZMH+e2bhpo+85W56FvfaMLvnc
tm2bv34kSC4yK4P7sX4FznDQSMv9GKCRTRyR4VwaZAmTPRR4fKf5jOpn2u+MN72lWcKRUbmP
++gQ0PU2qM5EJ6fMSVQIpNPhRt5FycWA4mTPa7V8c37ypAnmumsb2OKlMnAq5oHSXlaqtKe9
d/OmTabGmafbNk2caaJ+i0+KsEFSe3PAv3TLGZBUVl1Kt9yUb3aImfKNlpxg1ShffUHOZMIr
LJeI3+KWplZhDZthdtvGsLSCrof1Wb/+6ivfWzaWOk6f/rpddvniqFesFSLS0v2JMJ0j6Jm8
9hcBKswhCjMQuabZuKY7ru5npabNmGlqeObNetNpYEZQPl2kFcSwyhv249Dpu8f6R6Hz6sbD
uR55xjniyyeXYIpS96IL/LWyI158yTraQjzZ9GelpHHY2/sWnYzIobx6vZrcp8slP2jNA/GC
PrN1a7ObbUOEcPnslOaLhujdOR9Yr6uIIxtmtO9oe5s91QqozofMdMs9sncV5ooVK5mWLZqZ
kZ6JPjYZAJH42OvZefFQjTXJ4nAhSC5oaHW6kl5QR1LWpCGOrOUGf73J2kBcEy+1OpzH0SeQ
n99y9EvDHBYFgXQ73GjLLzzvHH+ZDtoTePuXDaZ+k1+bZpfPyDV3r5fM6GUkEk8rbd2697TL
ft56c4Ztr4PeE3Jf0D7dcgallU3X0i035ZsdUqZ8oycnvTwRuUNf+F9XXm30JAauo8/87NBn
7LJBnKNvhuWccBqs3+WFoTCLWTacL8oaajxHPmOL/Og8hOkciMctnAAV5gQKM7Bp02ycu8rO
y6NGmmZNmyDIbkOefMZc+s9/2k8DwXFW4+saSpDRnQitIIZV3lQUOj/x/x3oH4WbVzcuzmFm
jplI2bSHaq2YIRxOrK5vfIMpV7ac98H0Jfabm5gFxaaVMukY4bp8o/OYo48x27ZvM2/PfMvc
cH0jBFlz48WeSTGcH2geNtD7N3bcRFOtenW7drrrA10MZpOxYZ0vPF3D07araKKRQIN2wokn
2gZs+LCh9tuf9kbvn1bEU+HrKsyYMdHfTsbzHh88xNQ651yzS6ldLBco59JoiWKfilxcR2sY
cNDmi1IGdzADeRg1eqw54YQT7eesRgx/1n6sXuIvW/lp3Npyuc59tAmkUmeiXQLmrqgJpNvh
Rj6hVDW/+SYDJVZvGPR7qGefhMqy3H9WtdMMvp4A3w3uIB4c4+CrBs89O8xPHm3X1OlvmqOP
Oca/lspBQcqZSvpRjVOQclO+UZXqX/mifP9iEZUjUUZh9Scb2ra7O93r9TOrWsUZ/b1+fXvH
+eoZ8+oEc8ml9ewt+l2ONm/R0mVGrG8kzVT2ifqs2iwbaenJIZzrPITpHIjHLZxAzirMshY2
WcUJmoXVDpugsD3U7cGEHzQHfnyvt3ffR3wHTVpBDMtDoh9HmEjxo5CyQWEO+p6xe68ohbiO
H/Os9+eaKlUOttFgfgwT4EQbFLv3Pphv9t57HxsNzrDOO6emP1sRdi++/Swm25pHWHxcR/7c
WeSNGzeYK+rVtRYBie51TbZT4Svmz7qRg8zF7CbR82ooZ2q6HoXJRcdBuuJhNugZ+NTLJXXO
T8pYTN6D0uC1aBPQ9SGszkS7BMxdURMoSIdb8rpmzTf2M3X4dMrfKv7NDmhKWCb233+/1rNu
+s3stvtu9j3jztSk8oxMlDOV50QtTibKTflGTap/5Yfy/YtFlI7Qx2x6Y2PPt8zrKWULE2dY
WiebfpfrvqSEp7pP1GfVZtlIb/TY8XaCT9LWukGYziFxuQ8mkLMKs5jHpqJQaLNfd9RGsL7/
3mxze5tW/syiXMcMa/eeve0MpFzDXn/bMsjRCuLomUw4Gzj2uONwOeGmR5GwFhjrqpN1SFyv
2dpZCx6G9botW9ySp2wI6/Pwo3b9tuvxD+bqcLSFGXp3A5MevfuammfX8oO0wnz3PffZ2eMe
3bv64TjAoEOX+7t6nbiKcddxgllXrI2G2ba7wUFY5/sfNP846qi4IDR+V//rMntN1tXFRfBO
ZLbcNX22z/O8J8IxF8xg9IbRx/6PDbSu/MVBjv5MVCK5DHi0n00T6SWTOWZsnn5yiB9f5+HC
OheZLg9289Yvn6Yv8ziLCOC3LGs8E9WZLCoSs1rIBDLR4S7kLGYk+VwppwsrV8qdK+WkfHd+
p9jlENXzd96eaSfIYJkZtMEap3WbO6w5tg7X/T/0Jed6X4cpV768jpLSsdYJ3D4rJnIaXVvf
TJk8yU4szf9wiT+JhcS1bpCK3pNShnIsUk4qzIUpY4yew4QDf3BDv99++xfm44o0bZkZKF+h
vNm6ZavZ/4ADTJkyZRLmAUodZpxhPg0lE/Fhgu1uWmEW03Xch4YGMx3lvcZlL299dLINcTEY
gU9VoYHAd+2w7riwNjzvhx/Wme3btpvdS+9u8ImoypUPsuUtrGe66YLxuu+/N2XLlbUzN3C8
lgorNx2ekwAJZDeBXFE0cqWcbm3MlXLnSjkp3+xSmEVemBDa5DkyxGbXKHv94f322y8tJVjS
5D76BKgwR19GOZFDrTDTXCQnRM5CkgAJZJhArigauVJOt3rkSrlzpZyUb3YqzK7ceJ4bBKgw
54acI19KKsyRFxEzSAIkEHEComhEPJsZy17dy3Krw035ZqzqRDIhyjeSYmGmSMASoMLMihAJ
AlSYIyEGZoIESCCLCbDDncXCSyHrlG8KkLI4CuWbxcJj1ks8ASrMJV7E2VFAKszZISfmkgRI
gARIgARIgARIgARyiQAV5lySdsTLutlzogBPf2XLlStSp1kRx8LskQAJkAAJkAAJkAAJkAAJ
FBMBKszFBJ6PJQESIAESIAESIAESIAESIAESiDYBKszRlg9zRwIkQAIkQAIkQAIkQAIkQAIk
UEwEqDAXE3g+lgRIgARIgARIgARIgARIgARIINoEqDBHWz7MHQmQAAmQAAmQAAmQAAmQAAmQ
QDERoMJcTOD5WBIgARIgARIgARIgARIgARIggWgToMIcbfkwdyRAAiRAAiRAAiRAAiRAAiRA
AsVEgApzMYHnY0mABEiABEiABEiABEiABEiABKJNgApztOXD3JEACZAACZAACZAACZAACZAA
CRQTASrMxQSejyUBEiABEiABEiABEiABEiABEog2ASrM0ZYPc0cCJEACJEACJEACJEACJEAC
JFBMBP4fx/BRetGKoRkAAAAASUVORK5CYII=
--------------627018FE70FE823EEDB7FEB5--
--------------52C86E47EED6E2100703EC63--
4:26 p.m.
--_000_EF8A6CD043B349ADB14F59EB42E5B511acroniscom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
TmlyLA0KDQpJc27igJl0IFNQTSBtYW5hZ2VkIHZpYSBTYW5sb2NrPyBJIGJlbGlldmUgdGhlcmUg
aXMgbm8gbmVlZCB0byBmZW5jZSBTUE0gaG9zdC4gRXNwZWNpYWxseSBpZiB0aGVyZSBhcmUgbm8g
U1BNIHRhc2tzIHJ1bm5pbmcuDQoNCg0KRnJvbTogPHVzZXJzLWJvdW5jZXNAb3ZpcnQub3JnPiBv
biBiZWhhbGYgb2YgTmlyIFNvZmZlciA8bnNvZmZlckByZWRoYXQuY29tPg0KRGF0ZTogTW9uZGF5
LCAxNyBBcHJpbCAyMDE3IGF0IDE2OjA2DQpUbzogS29uc3RhbnRpbiBSYXNrb3NobnlpIDxrb25y
YXNrb0BnbWFpbC5jb20+LCBEYW4gWWFzbnkgPGR5YXNueUBnbWFpbC5jb20+DQpDYzogdXNlcnMg
PHVzZXJzQG92aXJ0Lm9yZz4sIEZFUk5BTkRPIEZSRURJQU5JIDxmZXJuYW5kby5mcmVkaWFuaUB1
cHguY29tPg0KU3ViamVjdDogUmU6IFtvdmlydC11c2Vyc10gc3RvcmFnZSByZWR1bmRhbmN5IGlu
IE92aXJ0DQoNCk9uIE1vbiwgQXByIDE3LCAyMDE3IGF0IDg6MjQgQU0gS29uc3RhbnRpbiBSYXNr
b3NobnlpIDxrb25yYXNrb0BnbWFpbC5jb208bWFpbHRvOmtvbnJhc2tvQGdtYWlsLmNvbT4+IHdy
b3RlOg0KQnV0IGFjdHVhbGx5LCBpdCBkaWRuJ3Qgd29yayB3ZWxsLiBBZnRlciBtYWluIFNQTSBo
b3N0IHdlbnQgZG93biBJIHNlZSB0aGlzDQoNCg0KMjAxNy0wNC0xNyAwNToyMzoxNSw1NTRaIEVS
Uk9SIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmlyc2Jyb2tlci5JcnNQcm94eV0g
KERlZmF1bHRRdWFydHpTY2hlZHVsZXI1KSBbNGRjYzAzM2QtMjZiZi00OWJiLWJmYWEtMDNhOTcw
ZGJiZWMxXSBTUE0gSW5pdDogY291bGQgbm90IGZpbmQgcmVwb3J0ZWQgdmRzIG9yIG5vdCB1cCAt
IHBvb2w6ICdTVEcnIHZkc19zcG1faWQ6ICcxJw0KMjAxNy0wNC0xNyAwNToyMzoxNSw1NjdaIElO
Rk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmlyc2Jyb2tlci5JcnNQcm94eV0g
KERlZmF1bHRRdWFydHpTY2hlZHVsZXI1KSBbNGRjYzAzM2QtMjZiZi00OWJiLWJmYWEtMDNhOTcw
ZGJiZWMxXSBTUE0gc2VsZWN0aW9uIC0gdmRzIHNlZW1zIGFzIHNwbSAndGFuazUnDQoyMDE3LTA0
LTE3IDA1OjIzOjE1LDU2N1ogV0FSTiAgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIu
aXJzYnJva2VyLklyc1Byb3h5XSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcjUpIFs0ZGNjMDMzZC0y
NmJmLTQ5YmItYmZhYS0wM2E5NzBkYmJlYzFdIHNwbSB2ZHMgaXMgbm9uIHJlc3BvbnNpdmUsIHN0
b3BwaW5nIHNwbSBzZWxlY3Rpb24uDQoNClNvIHRoYXQgbWVhbnMgb25seSBpZiBCTUMgaXMgdXAg
aXQncyBwb3NzaWJsZSB0byBhdXRvbWF0aWNhbGx5IHN3aXRjaCAgU1BNIGhvc3Q/DQoNCkJNQz8N
Cg0KSWYgeW91ciBTUE0gaXMgbm8gcmVzcG9uc2l2ZSwgdGhlIHN5c3RlbSB3aWxsIHRyeSB0byBm
ZW5jZSBpdC4gRGlkIHlvdQ0KY29uZmlndXJlIHBvd2VyIG1hbmFnZW1lbnQgZm9yIGFsbCBob3N0
cz8gZGlkIHlvdSBjaGVjayB0aGF0IGl0DQp3b3JrPyBIb3cgZGlkIHlvdSBzaW11bGF0ZSBub24t
cmVzcG9uc2l2ZSBob3N0Pw0KDQpJZiBwb3dlciBtYW5hZ2VtZW50IGlzIG5vdCBjb25maWd1cmVk
IG9yIGZhaWwsIHRoZSBzeXN0ZW0gY2Fubm90DQptb3ZlIHRoZSBzcG0gdG8gYW5vdGhlciBob3N0
LCB1bmxlc3MgeW91IG1hbnVhbGx5IGNvbmZpcm0gdGhhdCB0aGUNClNQTSBob3N0IHdhcyByZWJv
b3RlZC4NCg0KTmlyDQoNCg0KVGhhbmtzDQoNCk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6Mjkg
UE0sIEtvbnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21haWwuY29tPG1haWx0bzprb25y
YXNrb0BnbWFpbC5jb20+PiB3cm90ZToNCk9oLCBmZW5jZSBhZ2VudCB3b3JrcyBmaW5lIGlmIEkg
c2VsZWN0IGlsbzQsDQpUaGFuayB5b3UgZm9yIHlvdXIgaGVscCENCg0KT24gU3VuLCBBcHIgMTYs
IDIwMTcgYXQgODoyMiBQTSBEYW4gWWFzbnkgPGR5YXNueUBnbWFpbC5jb208bWFpbHRvOmR5YXNu
eUBnbWFpbC5jb20+PiB3cm90ZToNCk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDExOjE5IFBNLCBL
b25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdtYWlsLmNvbTxtYWlsdG86a29ucmFza29A
Z21haWwuY29tPj4gd3JvdGU6DQpNYWtlcyBzZW5zZS4NCkkgd2FzIHRyeWluZyB0byBzZXQgaXQg
dXAsIGJ1dCBkb2Vzbid0IHdvcmsgd2l0aCBvdXIgc3RhZ2luZyBoYXJkd2FyZS4NCldlIGhhdmUg
b2xkIGlsbzEwMCwgSSdsbCB0cnkgYWdhaW4uDQpUaGFua3MhDQoNCg0KSXQgaXMgYWJzb2x1dGVs
eSBuZWNlc3NhcnkgZm9yIGFueSBIQSB0byB3b3JrIHByb3Blcmx5LiBUaGVyZSdzIG9mIGNvdXJz
ZSB0aGUgImNvbmZpcm0gaG9zdCBoYXMgYmVlbiBzaHV0ZG93biIgb3B0aW9uLCB3aGljaCBzZXJ2
ZXMgYXMgYW4gb3ZlcnJpZGUgZm9yIHRoZSBmZW5jZSBjb21tYW5kLCBidXQgaXQncyBtYW51YWwN
Cg0KT24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgODoxOCBQTSBEYW4gWWFzbnkgPGR5YXNueUBnbWFp
bC5jb208bWFpbHRvOmR5YXNueUBnbWFpbC5jb20+PiB3cm90ZToNCk9uIFN1biwgQXByIDE2LCAy
MDE3IGF0IDExOjE1IFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdtYWlsLmNv
bTxtYWlsdG86a29ucmFza29AZ21haWwuY29tPj4gd3JvdGU6DQpGZW5jZSBhZ2VudCB1bmRlciBl
YWNoIG5vZGU/DQoNCldoZW4geW91IGNvbmZpZ3VyZSBhIGhvc3QsIHRoZXJlJ3MgdGhlIHBvd2Vy
IG1hbmFnZW1lbnQgdGFiLCB3aGVyZSB5b3UgbmVlZCB0byBlbnRlciB0aGUgYm1jIGRldGFpbHMg
Zm9yIHRoZSBob3N0LiBJZiB5b3UgZG9uJ3QgaGF2ZSBmZW5jaW5nIGVuYWJsZWQsIGhvdyBkbyB5
b3UgZXhwZWN0IHRoZSBzeXN0ZW0gdG8gbWFrZSBzdXJlIGEgaG9zdCBydW5uaW5nIGEgc2Vydmlj
ZSBpcyBhY3R1YWxseSBkb3duIChhbmQgaXQgaXMgc2FmZSB0byBzdGFydCBIQSBzZXJ2aWNlcyBl
bHNld2hlcmUpLCBhbmQgbm90LCBmb3IgZXhhbXBsZSwganVzdCB1bnJlYWNoYWJsZSBieSB0aGUg
ZW5naW5lPyBIb3cgZG8geW91IGF2b2lkIGEgc3BsaXRicmFpZCAtPiBTQkEgPw0KDQoNCk9uIFN1
biwgQXByIDE2LCAyMDE3IGF0IDg6MTQgUE0gRGFuIFlhc255IDxkeWFzbnlAZ21haWwuY29tPG1h
aWx0bzpkeWFzbnlAZ21haWwuY29tPj4gd3JvdGU6DQpPbiBTdW4sIEFwciAxNiwgMjAxNyBhdCAx
MToxMyBQTSwgS29uc3RhbnRpbiBSYXNrb3NobnlpIDxrb25yYXNrb0BnbWFpbC5jb208bWFpbHRv
OmtvbnJhc2tvQGdtYWlsLmNvbT4+IHdyb3RlOg0KIkNvcm5lciBjYXNlcyI/DQpJIHRyaWVkIHRv
IHNpbXVsYXRlIGNyYXNoIG9mIFNQTSBzZXJ2ZXIgYW5kIG92aXJ0IGtlcHQgdHJ5aW5nIHRvIHJl
aXN0YWJsaXNoZWQgY29ubmVjdGlvbiB0byB0aGUgZmFpbGVkIG5vZGUuDQoNCkRpZCB5b3UgY29u
ZmlndXJlIGZlbmNpbmc/DQoNCg0KDQpPbiBTdW4sIEFwciAxNiwgMjAxNyBhdCA4OjEwIFBNIERh
biBZYXNueSA8ZHlhc255QGdtYWlsLmNvbTxtYWlsdG86ZHlhc255QGdtYWlsLmNvbT4+IHdyb3Rl
Og0KT24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgNzoyOSBBTSwgTmlyIFNvZmZlciA8bnNvZmZlckBy
ZWRoYXQuY29tPG1haWx0bzpuc29mZmVyQHJlZGhhdC5jb20+PiB3cm90ZToNCk9uIFN1biwgQXBy
IDE2LCAyMDE3IGF0IDI6MDUgUE0gRGFuIFlhc255IDxkeWFzbnlAcmVkaGF0LmNvbTxtYWlsdG86
ZHlhc255QHJlZGhhdC5jb20+PiB3cm90ZToNCg0KDQpPbiBBcHIgMTYsIDIwMTcgNzowMSBBTSwg
Ik5pciBTb2ZmZXIiIDxuc29mZmVyQHJlZGhhdC5jb208bWFpbHRvOm5zb2ZmZXJAcmVkaGF0LmNv
bT4+IHdyb3RlOg0KT24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgNDoxNyBBTSBEYW4gWWFzbnkgPGR5
YXNueUBnbWFpbC5jb208bWFpbHRvOmR5YXNueUBnbWFpbC5jb20+PiB3cm90ZToNCldoZW4geW91
IHNldCB1cCBhIHN0b3JhZ2UgZG9tYWluLCB5b3UgbmVlZCB0byBzcGVjaWZ5IGEgaG9zdCB0byBw
ZXJmb3JtIHRoZSBpbml0aWFsIHN0b3JhZ2Ugb3BlcmF0aW9ucywgYnV0IG9uY2UgdGhlIFNEIGlz
IGRlZmluZWQsIGl0J3MgZGV0YWlscyBhcmUgaW4gdGhlIGVuZ2luZSBkYXRhYmFzZSwgYW5kIGFs
bCB0aGUgaG9zdHMgZ2V0IGNvbm5lY3RlZCB0byBpdCBkaXJlY3RseS4gSWYgdGhlIGZpcnN0IGhv
c3QgeW91IHVzZWQgdG8gZGVmaW5lIHRoZSBTRCBnb2VzIGRvd24sIGFsbCBvdGhlciBob3N0cyB3
aWxsIHN0aWxsIHJlbWFpbiBjb25uZWN0ZWQgYW5kIHdvcmsuIFNQTSBpcyBhbiBIQSBzZXJ2aWNl
LCBhbmQgaWYgdGhlIGN1cnJlbnQgU1BNIGhvc3QgZ29lcyBkb3duLCBTUE0gZ2V0cyBzdGFydGVk
IG9uIGFub3RoZXIgaG9zdCBpbiB0aGUgREMuIEluIHNob3J0LCB1bmxlc3MgeW91ciBhY3R1YWwg
TkZTIGV4cG9ydGluZyBob3N0IGdvZXMgZG93biwgdGhlcmUgaXMgbm8gb3V0YWdlLg0KDQpUaGVy
ZSBpcyBubyBzdG9yYWdlIG91dGFnZSwgYnV0IGlmIHlvdSBzaHV0ZG93biB0aGUgc3BtIGhvc3Qs
IHRoZSBzcG0gaG9zdA0Kd2lsbCBub3QgbW92ZSB0byBhIG5ldyBob3N0IHVudGlsIHRoZSBzcG0g
aG9zdCBpcyBvbmxpbmUgYWdhaW4sIG9yIHlvdSBjb25maXJtDQptYW51YWxseSB0aGF0IHRoZSBz
cG0gaG9zdCB3YXMgcmVib290ZWQuDQoNCkluIGEgcHJvcGVybHkgY29uZmlndXJlZCBzZXR1cCB0
aGUgU0JBIHNob3VsZCB0YWtlIGNhcmUgb2YgdGhhdC4gVGhhdCdzIHRoZSB3aG9sZSBwb2ludCBv
ZiBIQSBzZXJ2aWNlcw0KDQpJbiBzb21lIGNhc2VzIGxpa2UgcG93ZXIgbG9zcyBvciBoYXJkd2Fy
ZSBmYWlsdXJlLCB0aGVyZSBpcyBubyB3YXkgdG8gc3RhcnQNCnRoZSBzcG0gaG9zdCwgYW5kIHRo
ZSBzeXN0ZW0gY2Fubm90IHJlY292ZXIgYXV0b21hdGljYWxseS4NCg0KVGhlcmUgYXJlIGFsd2F5
cyBjb3JuZXIgY2FzZXMsIG5vIGRvdWJ0LiBCdXQgaW4gYSBub3JtYWwgc2l0dWF0aW9uLiB3aGVy
ZSBhbiBTUE0gaG9zdCBnb2VzIGRvd24gYmVjYXVzZSBvZiBhIGhhcmR3YXJlIGZhaWx1cmUsIGl0
IGdldHMgZmVuY2VkLCBvdGhlciBob3N0cyBjb250ZW5kIGZvciBTUE0gYW5kIHN0YXJ0IGl0LiBO
byBzdXJwcmlzZXMgdGhlcmUuDQoNCg0KTmlyDQoNCg0KDQpOaXINCg0KDQpPbiBTYXQsIEFwciAx
NSwgMjAxNyBhdCAxOjUzIFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdtYWls
LmNvbTxtYWlsdG86a29ucmFza29AZ21haWwuY29tPj4gd3JvdGU6DQpIaSBGZXJuYW5kbywNCkkg
c2VlIGVhY2ggaG9zdCBoYXMgZGlyZWN0IGNvbm5lY3Rpb24gbmZzIG1vdW50LCBidXQgeWVzLCBp
ZiBtYWluIGhvc3QgdG8gd2hpY2ggSSBjb25uZWN0ZWQgbmZzIHN0b3JhZ2UgZ29pbmcgZG93biB0
aGUgc3RvcmFnZSBiZWNvbWVzIHVuYXZhaWxhYmxlIGFuZCBhbGwgdm1zIGFyZSBkb3duDQoNCg0K
T24gU2F0LCBBcHIgMTUsIDIwMTcgYXQgMTA6MzcgQU0gRkVSTkFORE8gRlJFRElBTkkgPGZlcm5h
bmRvLmZyZWRpYW5pQHVweC5jb208bWFpbHRvOmZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb20+PiB3
cm90ZToNCkhlbGxvIEtvbnN0YW50aW4uDQpUaGF0IGRvZXNuYHQgbWFrZSBtdWNoIHNlbnNlIG1h
a2UgYSB3aG9sZSBjbHVzdGVyIGRlcGVuZCBvbiBhIHNpbmdsZSBob3N0LiBGcm9tIHdoYXQgSSBr
bm93IGFueSBob3N0IHRhbGsgZGlyZWN0bHkgdG8gTkZTIFN0b3JhZ2UgQXJyYXkgb3Igd2hhdGV2
ZXIgb3RoZXIgU2hhcmVkIFN0b3JhZ2UgeW91IGhhdmUuDQpIYXZlIHlvdSB0ZXN0ZWQgdGhhdCBo
b3N0IGdvaW5nIGRvd24gaWYgdGhhdCBhZmZlY3RzIHRoZSBvdGhlciB3aXRoIHRoZSBORlMgbW91
bnRlZCBkaXJlY3RsbHkgaW4gYSBORlMgU3RvcmFnZSBhcnJheSA/DQpGZXJuYW5kbw0KDQoyMDE3
LTA0LTE1IDEyOjQyIEdNVC0wMzowMCBLb25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdt
YWlsLmNvbTxtYWlsdG86a29ucmFza29AZ21haWwuY29tPj46DQpJbiBvdmlydCB5b3UgaGF2ZSB0
byBhdHRhY2ggc3RvcmFnZSB0aHJvdWdoIHNwZWNpZmljIGhvc3QuDQpJZiBob3N0IGdvZXMgZG93
biBzdG9yYWdlIGlzIG5vdCBhdmFpbGFibGUuDQoNCk9uIFNhdCwgQXByIDE1LCAyMDE3IGF0IDc6
MzEgQU0gRkVSTkFORE8gRlJFRElBTkkgPGZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb208bWFpbHRv
OmZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb20+PiB3cm90ZToNCldlbGwsIG1ha2UgaXQgbm90IGdv
IHRocm91Z2ggaG9zdDEgYW5kIGRlZGljYXRlIGEgc3RvcmFnZSBzZXJ2ZXIgZm9yIHJ1bm5pbmcg
TkZTIGFuZCBtYWtlIGJvdGggaG9zdHMgY29ubmVjdCB0byBpdC4NCkluIG15IHZpZXcgTkZTIGlz
IG11Y2ggZWFzaWVyIHRvIG1hbmFnZSB0aGFuIGFueSBvdGhlciB0eXBlIG9mIHN0b3JhZ2UsIHNw
ZWNpYWxseSBGQyBhbmQgaVNDU0kgYW5kIHBlcmZvcm1hbmNlIGlzIHByZXR0eSBtdWNoIHRoZSBz
YW1lLCBzbyB5b3Ugd29uYHQgZ2V0IGJldHRlciByZXN1bHRzIG90aGVyIHRoYW4gbWFuYWdlbWVu
dCBnb2luZyB0byBvdGhlciB0eXBlLg0KRmVybmFuZG8NCg0KMjAxNy0wNC0xNSA1OjI1IEdNVC0w
MzowMCBLb25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdtYWlsLmNvbTxtYWlsdG86a29u
cmFza29AZ21haWwuY29tPj46DQpIaSBndXlzLA0KSSBoYXZlIG9uZSBuZnMgc3RvcmFnZSwNCml0
J3MgY29ubmVjdGVkIHRocm91Z2ggaG9zdDEuDQpob3N0MiBhbHNvIGhhcyBhY2Nlc3MgdG8gaXQs
IEkgY2FuIGVhc2lseSBtaWdyYXRlIHZtcyBiZXR3ZWVuIHRoZW0uDQoNClRoZSBxdWVzdGlvbiBp
cyAtIGlmIGhvc3QxIGlzIGRvd24gLSBhbGwgaW5mcmFzdHJ1Y3R1cmUgaXMgZG93biwgc2luY2Ug
YWxsIHRyYWZmaWMgZ29lcyB0aHJvdWdoIGhvc3QxLA0KaXMgdGhlcmUgYW55IHdheSBpbiBvVmly
dCB0byB1c2UgcmVkdW5kYW50IHN0b3JhZ2U/DQoNCk9ubHkgZ2x1c3RlcmZzPw0KDQpUaGFua3MN
Cg0KDQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fXw0KVXNl
cnMgbWFpbGluZyBsaXN0DQpVc2Vyc0BvdmlydC5vcmc8bWFpbHRvOlVzZXJzQG92aXJ0Lm9yZz4N
Cmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2Vycw0KDQoNCl9fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpVc2VycyBtYWlsaW5n
IGxpc3QNClVzZXJzQG92aXJ0Lm9yZzxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPg0KaHR0cDovL2xp
c3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzDQoNCl9fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpVc2VycyBtYWlsaW5nIGxpc3QNClVzZXJz
QG92aXJ0Lm9yZzxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPg0KaHR0cDovL2xpc3RzLm92aXJ0Lm9y
Zy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fDQpVc2VycyBtYWlsaW5nIGxpc3QNClVzZXJzQG92aXJ0Lm9yZzxt
YWlsdG86VXNlcnNAb3ZpcnQub3JnPg0KaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xp
c3RpbmZvL3VzZXJzDQoNCg0K
--_000_EF8A6CD043B349ADB14F59EB42E5B511acroniscom_
Content-Type: text/html; charset="utf-8"
Content-ID: <351727B7CD9E1A4AA911DECC8CBB0E8A(a)acronis.com>
Content-Transfer-Encoding: base64
PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo
dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo
dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l
dGEgbmFtZT0iVGl0bGUiIGNvbnRlbnQ9IiI+DQo8bWV0YSBuYW1lPSJLZXl3b3JkcyIgY29udGVu
dD0iIj4NCjxtZXRhIG5hbWU9IkdlbmVyYXRvciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUg
KGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxlPjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8N
CkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0
IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJ
cGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8N
CnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsN
CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWls
eToiVGltZXMgTmV3IFJvbWFuIjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1z
dHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxp
bmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1w
cmlvcml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9
DQpzcGFuLm0tNzA0OTEzMTYxOTY3MzQ1NzA3N20tMzQ0NzQ3MzU3Nzk3OTk3MDUxM20xNzc0MTA4
MDMxOTMyNTkyODg2bTM4NDY1MjU1MTA3OTE2MTY1MjhtMTEyMDM1NzYwODA1NjY4NDc0M20zNjIz
NzUwMzA3MTY0OTY2NjJtLTc5NTQxMzA3MjIyMzU0MTUyMTBtMjE3MzkyNTI0MTU3NzMzNjkzNW0t
NjQzNDg0Njg3ODMxOTM3MTE5MWhvZW56Yg0KCXttc28tc3R5bGUtbmFtZTptXy03MDQ5MTMxNjE5
NjczNDU3MDc3bV8tMzQ0NzQ3MzU3Nzk3OTk3MDUxM21fMTc3NDEwODAzMTkzMjU5Mjg4Nm1fMzg0
NjUyNTUxMDc5MTYxNjUyOG1fMTEyMDM1NzYwODA1NjY4NDc0M21fMzYyMzc1MDMwNzE2NDk2NjYy
bV8tNzk1NDEzMDcyMjIzNTQxNTIxMG1fMjE3MzkyNTI0MTU3NzMzNjkzNW1fLTY0MzQ4NDY4Nzgz
MTkzNzExOTFob2VuemI7fQ0Kc3Bhbi5FbWFpbFN0eWxlMTgNCgl7bXNvLXN0eWxlLXR5cGU6cGVy
c29uYWwtcmVwbHk7DQoJZm9udC1mYW1pbHk6Q2FsaWJyaTsNCgljb2xvcjp3aW5kb3d0ZXh0O30N
CnNwYW4ubXNvSW5zDQoJe21zby1zdHlsZS10eXBlOmV4cG9ydC1vbmx5Ow0KCW1zby1zdHlsZS1u
YW1lOiIiOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxpbmU7DQoJY29sb3I6dGVhbDt9DQouTXNv
Q2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCglmb250LXNpemU6MTAu
MHB0O30NCkBwYWdlIFdvcmRTZWN0aW9uMQ0KCXtzaXplOjU5NS4wcHQgODQyLjBwdDsNCgltYXJn
aW46Mi4wY20gNDIuNXB0IDIuMGNtIDMuMGNtO30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpX
b3JkU2VjdGlvbjE7fQ0KLS0+PC9zdHlsZT4NCjwvaGVhZD4NCjxib2R5IGJnY29sb3I9IndoaXRl
IiBsYW5nPSJFTi1HQiIgbGluaz0iYmx1ZSIgdmxpbms9InB1cnBsZSI+DQo8ZGl2IGNsYXNzPSJX
b3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6
ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaTttc28tZmFyZWFzdC1sYW5ndWFnZTpFTi1VUyI+
TmlyLDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0
eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OkNhbGlicmk7bXNvLWZhcmVhc3QtbGFu
Z3VhZ2U6RU4tVVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OkNhbGlicmk7
bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPklzbuKAmXQgU1BNIG1hbmFnZWQgdmlhIFNhbmxv
Y2s/IEkgYmVsaWV2ZSB0aGVyZSBpcyBubyBuZWVkIHRvIGZlbmNlIFNQTSBob3N0LiBFc3BlY2lh
bGx5IGlmIHRoZXJlIGFyZSBubyBTUE0gdGFza3MgcnVubmluZy48bzpwPjwvbzpwPjwvc3Bhbj48
L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjExLjBwdDtm
b250LWZhbWlseTpDYWxpYnJpO21zby1mYXJlYXN0LWxhbmd1YWdlOkVOLVVTIj48bzpwPiZuYnNw
OzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9u
dC1zaXplOjExLjBwdDtmb250LWZhbWlseTpDYWxpYnJpO21zby1mYXJlYXN0LWxhbmd1YWdlOkVO
LVVTIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8ZGl2IHN0eWxlPSJib3JkZXI6bm9u
ZTtib3JkZXItdG9wOnNvbGlkICNCNUM0REYgMS4wcHQ7cGFkZGluZzozLjBwdCAwY20gMGNtIDBj
bSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48Yj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2Fs
aWJyaTtjb2xvcjpibGFjayI+RnJvbTogPC9zcGFuPg0KPC9iPjxzcGFuIHN0eWxlPSJmb250LWZh
bWlseTpDYWxpYnJpO2NvbG9yOmJsYWNrIj4mbHQ7dXNlcnMtYm91bmNlc0BvdmlydC5vcmcmZ3Q7
IG9uIGJlaGFsZiBvZiBOaXIgU29mZmVyICZsdDtuc29mZmVyQHJlZGhhdC5jb20mZ3Q7PGJyPg0K
PGI+RGF0ZTogPC9iPk1vbmRheSwgMTcgQXByaWwgMjAxNyBhdCAxNjowNjxicj4NCjxiPlRvOiA8
L2I+S29uc3RhbnRpbiBSYXNrb3NobnlpICZsdDtrb25yYXNrb0BnbWFpbC5jb20mZ3Q7LCBEYW4g
WWFzbnkgJmx0O2R5YXNueUBnbWFpbC5jb20mZ3Q7PGJyPg0KPGI+Q2M6IDwvYj51c2VycyAmbHQ7
dXNlcnNAb3ZpcnQub3JnJmd0OywgRkVSTkFORE8gRlJFRElBTkkgJmx0O2Zlcm5hbmRvLmZyZWRp
YW5pQHVweC5jb20mZ3Q7PGJyPg0KPGI+U3ViamVjdDogPC9iPlJlOiBbb3ZpcnQtdXNlcnNdIHN0
b3JhZ2UgcmVkdW5kYW5jeSBpbiBPdmlydDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+
DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBNb24sIEFwciAx
NywgMjAxNyBhdCA4OjI0IEFNIEtvbnN0YW50aW4gUmFza29zaG55aSAmbHQ7PGEgaHJlZj0ibWFp
bHRvOmtvbnJhc2tvQGdtYWlsLmNvbSI+a29ucmFza29AZ21haWwuY29tPC9hPiZndDsgd3JvdGU6
PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTti
b3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7
bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCI+QnV0IGFjdHVhbGx5LCBpdCBkaWRuJ3Qgd29yayB3ZWxsLiBBZnRlciBtYWluIFNQ
TSBob3N0IHdlbnQgZG93biBJIHNlZSB0aGlzDQo8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjxicj4NCjIwMTctMDQtMTcgMDU6MjM6MTUsNTU0WiBFUlJPUiBb
b3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5pcnNicm9rZXIuSXJzUHJveHldIChEZWZh
dWx0UXVhcnR6U2NoZWR1bGVyNSkgWzRkY2MwMzNkLTI2YmYtNDliYi1iZmFhLTAzYTk3MGRiYmVj
MV0gU1BNIEluaXQ6IGNvdWxkIG5vdCBmaW5kIHJlcG9ydGVkIHZkcyBvciBub3QgdXAgLSBwb29s
OiAnU1RHJyB2ZHNfc3BtX2lkOiAnMSc8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjIwMTctMDQtMTcgMDU6MjM6MTUsNTY3WiBJTkZPICZuYnNwO1tv
cmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmlyc2Jyb2tlci5JcnNQcm94eV0gKERlZmF1
bHRRdWFydHpTY2hlZHVsZXI1KSBbNGRjYzAzM2QtMjZiZi00OWJiLWJmYWEtMDNhOTcwZGJiZWMx
XSBTUE0gc2VsZWN0aW9uIC0gdmRzIHNlZW1zIGFzIHNwbSAndGFuazUnPG86cD48L286cD48L3A+
DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4yMDE3LTA0LTE3IDA1OjIzOjE1
LDU2N1ogV0FSTiAmbmJzcDtbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5pcnNicm9r
ZXIuSXJzUHJveHldIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyNSkgWzRkY2MwMzNkLTI2YmYtNDli
Yi1iZmFhLTAzYTk3MGRiYmVjMV0gc3BtIHZkcyBpcyBub24gcmVzcG9uc2l2ZSwgc3RvcHBpbmcg
c3BtIHNlbGVjdGlvbi48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+U28gdGhhdCBtZWFucyBvbmx5IGlmIEJNQyBpcyB1cCBpdCdzIHBvc3NpYmxl
IHRvIGF1dG9tYXRpY2FsbHkgc3dpdGNoICZuYnNwO1NQTSBob3N0PzxvOnA+PC9vOnA+PC9wPg0K
PC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiPkJNQz88bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+SWYgeW91ciBTUE0gaXMgbm8gcmVzcG9uc2l2ZSwgdGhlIHN5c3RlbSB3aWxsIHRyeSB0
byBmZW5jZSBpdC4gRGlkIHlvdSZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+Y29uZmlndXJlIHBvd2VyIG1hbmFnZW1lbnQgZm9yIGFsbCBo
b3N0cz8gZGlkIHlvdSBjaGVjayB0aGF0IGl0Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj53b3JrPyBIb3cgZGlkIHlvdSBzaW11bGF0ZSBu
b24tcmVzcG9uc2l2ZSBob3N0PyZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5JZiBwb3dlciBtYW5hZ2VtZW50IGlzIG5vdCBjb25maWd1
cmVkIG9yIGZhaWwsIHRoZSBzeXN0ZW0gY2Fubm90PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxk
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5tb3ZlIHRoZSBzcG0gdG8gYW5vdGhlciBob3N0LCB1
bmxlc3MgeW91IG1hbnVhbGx5IGNvbmZpcm0gdGhhdCB0aGU8bzpwPjwvbzpwPjwvcD4NCjwvZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPlNQTSBob3N0IHdhcyByZWJvb3RlZC48bzpw
PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5i
c3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+TmlyPG86
cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4mbmJzcDs8
bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2Jv
cmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDtt
YXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj5UaGFua3M8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8
ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+T24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgODoyOSBQTSwgS29u
c3RhbnRpbiBSYXNrb3NobnlpICZsdDs8YSBocmVmPSJtYWlsdG86a29ucmFza29AZ21haWwuY29t
IiB0YXJnZXQ9Il9ibGFuayI+a29ucmFza29AZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PG86cD48
L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29s
aWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQu
OHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9oLCBm
ZW5jZSBhZ2VudCB3b3JrcyBmaW5lIGlmIEkgc2VsZWN0IGlsbzQsPG86cD48L286cD48L3A+DQo8
L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5UaGFuayB5b3UgZm9yIHlvdXIgaGVs
cCE8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6MjIgUE0gRGFuIFlhc255ICZs
dDs8YSBocmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmR5YXNu
eUBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2Nr
cXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7
cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6
MGNtIj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIFN1biwg
QXByIDE2LCAyMDE3IGF0IDExOjE5IFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhy
ZWY9Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0Bn
bWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjxibG9ja3F1b3RlIHN0eWxl
PSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNt
IDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+TWFrZXMgc2Vuc2UuJm5ic3A7PG86cD48L286cD48L3A+
DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5JIHdhcyB0cnlpbmcgdG8gc2V0
IGl0IHVwLCBidXQgZG9lc24ndCB3b3JrIHdpdGggb3VyIHN0YWdpbmcgaGFyZHdhcmUuJm5ic3A7
PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5XZSBo
YXZlIG9sZCBpbG8xMDAsIEknbGwgdHJ5IGFnYWluLiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9k
aXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+VGhhbmtzISA8bzpwPjwvbzpwPjwvcD4N
CjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+
DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+
DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij5JdCBpcyBhYnNvbHV0ZWx5IG5lY2Vzc2FyeSBmb3IgYW55IEhBIHRvIHdvcmsgcHJvcGVybHku
IFRoZXJlJ3Mgb2YgY291cnNlIHRoZSAmcXVvdDtjb25maXJtIGhvc3QgaGFzIGJlZW4gc2h1dGRv
d24mcXVvdDsgb3B0aW9uLCB3aGljaCBzZXJ2ZXMgYXMgYW4gb3ZlcnJpZGUgZm9yIHRoZSBmZW5j
ZSBjb21tYW5kLCBidXQgaXQncyBtYW51YWw8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+
DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHls
ZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBj
bSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRp
dj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBT
dW4sIEFwciAxNiwgMjAxNyBhdCA4OjE4IFBNIERhbiBZYXNueSAmbHQ7PGEgaHJlZj0ibWFpbHRv
OmR5YXNueUBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5keWFzbnlAZ21haWwuY29tPC9hPiZn
dDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3Jk
ZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAw
Y20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPGRp
dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBTdW4sIEFwciAxNiwgMjAxNyBhdCAx
MToxNSBQTSwgS29uc3RhbnRpbiBSYXNrb3NobnlpICZsdDs8YSBocmVmPSJtYWlsdG86a29ucmFz
a29AZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+a29ucmFza29AZ21haWwuY29tPC9hPiZndDsg
d3JvdGU6PG86cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9y
ZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21h
cmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPkZlbmNlIGFnZW50IHVuZGVyIGVhY2ggbm9kZT88bzpwPjwvbzpwPjwvcD4NCjwvZGl2
Pg0KPC9ibG9ja3F1b3RlPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7
PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4N
CjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+V2hlbiB5b3UgY29uZmlndXJlIGEg
aG9zdCwgdGhlcmUncyB0aGUgcG93ZXIgbWFuYWdlbWVudCB0YWIsIHdoZXJlIHlvdSBuZWVkIHRv
IGVudGVyIHRoZSBibWMgZGV0YWlscyBmb3IgdGhlIGhvc3QuIElmIHlvdSBkb24ndCBoYXZlIGZl
bmNpbmcgZW5hYmxlZCwgaG93IGRvIHlvdSBleHBlY3QgdGhlIHN5c3RlbSB0byBtYWtlIHN1cmUg
YSBob3N0IHJ1bm5pbmcgYSBzZXJ2aWNlIGlzIGFjdHVhbGx5IGRvd24gKGFuZA0KIGl0IGlzIHNh
ZmUgdG8gc3RhcnQgSEEgc2VydmljZXMgZWxzZXdoZXJlKSwgYW5kIG5vdCwgZm9yIGV4YW1wbGUs
IGp1c3QgdW5yZWFjaGFibGUgYnkgdGhlIGVuZ2luZT8gSG93IGRvIHlvdSBhdm9pZCBhIHNwbGl0
YnJhaWQgLSZndDsgU0JBID88bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4N
CjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
PiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVy
Om5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNt
IDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+
DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6
MTQgUE0gRGFuIFlhc255ICZsdDs8YSBocmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFy
Z2V0PSJfYmxhbmsiPmR5YXNueUBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwv
cD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNv
bGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0
LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDExOjEzIFBNLCBLb25zdGFudGluIFJh
c2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20iIHRhcmdldD0i
X2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4N
CjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0ND
IDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2lu
LXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+JnF1b3Q7Q29ybmVyIGNh
c2VzJnF1b3Q7PyZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+SSB0cmllZCB0byBzaW11bGF0ZSBjcmFzaCBvZiBTUE0gc2VydmVyIGFuZCBv
dmlydCBrZXB0IHRyeWluZyB0byByZWlzdGFibGlzaGVkIGNvbm5lY3Rpb24gdG8gdGhlIGZhaWxl
ZCBub2RlLiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjwv
ZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj5EaWQgeW91IGNvbmZpZ3VyZSBmZW5jaW5nPzxvOnA+PC9vOnA+PC9wPg0K
PC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxi
bG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEu
MHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJp
Z2h0OjBjbSI+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpw
PiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxv
OnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5P
biBTdW4sIEFwciAxNiwgMjAxNyBhdCA4OjEwIFBNIERhbiBZYXNueSAmbHQ7PGEgaHJlZj0ibWFp
bHRvOmR5YXNueUBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5keWFzbnlAZ21haWwuY29tPC9h
PiZndDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJi
b3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBj
bSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0K
PGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBTdW4sIEFwciAxNiwgMjAxNyBh
dCA3OjI5IEFNLCBOaXIgU29mZmVyICZsdDs8YSBocmVmPSJtYWlsdG86bnNvZmZlckByZWRoYXQu
Y29tIiB0YXJnZXQ9Il9ibGFuayI+bnNvZmZlckByZWRoYXQuY29tPC9hPiZndDsgd3JvdGU6PG86
cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6
c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0
OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+T24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgMjowNSBQTSBEYW4gWWFzbnkgJmx0
OzxhIGhyZWY9Im1haWx0bzpkeWFzbnlAcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmR5YXNu
eUByZWRoYXQuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9j
a3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0
O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0
OjBjbSI+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9v
OnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9w
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIEFwciAxNiwgMjAxNyA3OjAxIEFNLCAm
cXVvdDtOaXIgU29mZmVyJnF1b3Q7ICZsdDs8YSBocmVmPSJtYWlsdG86bnNvZmZlckByZWRoYXQu
Y29tIiB0YXJnZXQ9Il9ibGFuayI+bnNvZmZlckByZWRoYXQuY29tPC9hPiZndDsgd3JvdGU6PG86
cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6
c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0
OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDQ6MTcgQU0gRGFuIFlh
c255ICZsdDs8YSBocmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsi
PmR5YXNueUBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K
PGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0Mg
MS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4t
cmlnaHQ6MGNtIj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5XaGVuIHlvdSBzZXQgdXAg
YSBzdG9yYWdlIGRvbWFpbiwgeW91IG5lZWQgdG8gc3BlY2lmeSBhIGhvc3QgdG8gcGVyZm9ybSB0
aGUgaW5pdGlhbCBzdG9yYWdlIG9wZXJhdGlvbnMsIGJ1dCBvbmNlIHRoZSBTRCBpcyBkZWZpbmVk
LCBpdCdzIGRldGFpbHMgYXJlIGluIHRoZSBlbmdpbmUgZGF0YWJhc2UsIGFuZCBhbGwgdGhlIGhv
c3RzIGdldCBjb25uZWN0ZWQgdG8gaXQgZGlyZWN0bHkuIElmIHRoZSBmaXJzdCBob3N0DQogeW91
IHVzZWQgdG8gZGVmaW5lIHRoZSBTRCBnb2VzIGRvd24sIGFsbCBvdGhlciBob3N0cyB3aWxsIHN0
aWxsIHJlbWFpbiBjb25uZWN0ZWQgYW5kIHdvcmsuIFNQTSBpcyBhbiBIQSBzZXJ2aWNlLCBhbmQg
aWYgdGhlIGN1cnJlbnQgU1BNIGhvc3QgZ29lcyBkb3duLCBTUE0gZ2V0cyBzdGFydGVkIG9uIGFu
b3RoZXIgaG9zdCBpbiB0aGUgREMuIEluIHNob3J0LCB1bmxlc3MgeW91ciBhY3R1YWwgTkZTIGV4
cG9ydGluZyBob3N0IGdvZXMgZG93biwgdGhlcmUNCiBpcyBubyBvdXRhZ2UuJm5ic3A7PG86cD48
L286cD48L3A+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+VGhlcmUgaXMgbm8gc3RvcmFnZSBvdXRhZ2UsIGJ1dCBpZiB5b3Ugc2h1
dGRvd24gdGhlIHNwbSBob3N0LCB0aGUgc3BtIGhvc3Q8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPndpbGwgbm90IG1vdmUgdG8gYSBuZXcgaG9zdCB1
bnRpbCB0aGUgc3BtIGhvc3QgaXMgb25saW5lIGFnYWluLCBvciB5b3UgY29uZmlybTxvOnA+PC9v
OnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+bWFudWFsbHkgdGhh
dCB0aGUgc3BtIGhvc3Qgd2FzIHJlYm9vdGVkLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rp
dj4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rp
dj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SW4gYSBwcm9wZXJseSBjb25m
aWd1cmVkIHNldHVwIHRoZSBTQkEgc2hvdWxkIHRha2UgY2FyZSBvZiB0aGF0LiBUaGF0J3MgdGhl
IHdob2xlIHBvaW50IG9mIEhBIHNlcnZpY2VzJm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4N
CjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SW4g
c29tZSBjYXNlcyBsaWtlIHBvd2VyIGxvc3Mgb3IgaGFyZHdhcmUgZmFpbHVyZSwgdGhlcmUgaXMg
bm8gd2F5IHRvIHN0YXJ0PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj50aGUgc3BtIGhvc3QsIGFuZCB0aGUgc3lzdGVtIGNhbm5vdCByZWNvdmVyIGF1
dG9tYXRpY2FsbHkuPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Js
b2NrcXVvdGU+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48
L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5UaGVyZSBhcmUgYWx3YXlzIGNvcm5lciBjYXNl
cywgbm8gZG91YnQuIEJ1dCBpbiBhIG5vcm1hbCBzaXR1YXRpb24uIHdoZXJlIGFuIFNQTSBob3N0
IGdvZXMgZG93biBiZWNhdXNlIG9mIGEgaGFyZHdhcmUgZmFpbHVyZSwgaXQgZ2V0cyBmZW5jZWQs
IG90aGVyIGhvc3RzIGNvbnRlbmQgZm9yIFNQTSBhbmQgc3RhcnQgaXQuIE5vIHN1cnByaXNlcyB0
aGVyZS4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2
Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNw
OzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7
Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0
O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImNvbG9yOiM4ODg4ODgiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiPjxzcGFuIHN0eWxlPSJjb2xvcjojODg4ODg4Ij5OaXI8bzpwPjwvbzpwPjwvc3Bhbj48L3A+
DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNw
OzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7
Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0
O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxk
aXY+DQo8ZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNv
bGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0
LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJjb2xvcjojODg4ODg4Ij48bzpwPiZuYnNwOzwvbzpwPjwv
c3Bhbj48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iY29sb3I6Izg4ODg4OCI+TmlyPG86cD48L286cD48L3NwYW4+PC9wPg0KPC9kaXY+DQo8ZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9k
aXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0ND
Q0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21h
cmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7
PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIFNhdCwgQXByIDE1LCAy
MDE3IGF0IDE6NTMgUE0sIEtvbnN0YW50aW4gUmFza29zaG55aSAmbHQ7PGEgaHJlZj0ibWFpbHRv
OmtvbnJhc2tvQGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmtvbnJhc2tvQGdtYWlsLmNvbTwv
YT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpu
b25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2
LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj5IaSBGZXJuYW5kbyw8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPkkgc2VlIGVhY2ggaG9zdCBoYXMgZGlyZWN0IGNvbm5lY3Rp
b24gbmZzIG1vdW50LCBidXQgeWVzLCBpZiBtYWluIGhvc3QgdG8gd2hpY2ggSSBjb25uZWN0ZWQg
bmZzIHN0b3JhZ2UgZ29pbmcgZG93biB0aGUgc3RvcmFnZSBiZWNvbWVzIHVuYXZhaWxhYmxlIGFu
ZCBhbGwgdm1zIGFyZSBkb3duPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+
DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPk9uIFNhdCwgQXByIDE1LCAyMDE3IGF0IDEw
OjM3IEFNIEZFUk5BTkRPIEZSRURJQU5JICZsdDs8YSBocmVmPSJtYWlsdG86ZmVybmFuZG8uZnJl
ZGlhbmlAdXB4LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb208
L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9
ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20g
MGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tcmlnaHQ6MGNtIj4NCjxkaXY+
DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibWFyZ2lu
LWJvdHRvbToxMi4wcHQiPkhlbGxvIEtvbnN0YW50aW4uPG86cD48L286cD48L3A+DQo8L2Rpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPlRoYXQgZG9lc25gdCBtYWtlIG11Y2ggc2Vuc2UgbWFrZSBh
IHdob2xlIGNsdXN0ZXIgZGVwZW5kIG9uIGEgc2luZ2xlIGhvc3QuIEZyb20gd2hhdCBJIGtub3cg
YW55IGhvc3QgdGFsayBkaXJlY3RseSB0byBORlMgU3RvcmFnZSBBcnJheSBvciB3aGF0ZXZlciBv
dGhlciBTaGFyZWQgU3RvcmFnZSB5b3UgaGF2ZS48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1hcmdpbi1ib3R0b206MTIuMHB0Ij5IYXZlIHlvdSB0
ZXN0ZWQgdGhhdCBob3N0IGdvaW5nIGRvd24gaWYgdGhhdCBhZmZlY3RzIHRoZSBvdGhlciB3aXRo
IHRoZSBORlMgbW91bnRlZCBkaXJlY3RsbHkgaW4gYSBORlMgU3RvcmFnZSBhcnJheSA/PG86cD48
L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPkZl
cm5hbmRvPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj4yMDE3
LTA0LTE1IDEyOjQyIEdNVC0wMzowMCBLb25zdGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9
Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0BnbWFp
bC5jb208L2E+Jmd0Ozo8bzpwPjwvbzpwPjwvcD4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6
bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20g
Ni4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+SW4gb3ZpcnQgeW91IGhhdmUgdG8gYXR0YWNoIHN0b3JhZ2UgdGhyb3Vn
aCBzcGVjaWZpYyBob3N0LiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCI+SWYgaG9zdCBnb2VzIGRvd24gc3RvcmFnZSBpcyBub3QgYXZhaWxh
YmxlLiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPk9uIFNhdCwgQXByIDE1LCAyMDE3IGF0IDc6MzEgQU0gRkVSTkFORE8gRlJFRElBTkkg
Jmx0OzxhIGhyZWY9Im1haWx0bzpmZXJuYW5kby5mcmVkaWFuaUB1cHguY29tIiB0YXJnZXQ9Il9i
bGFuayI+ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbTwvYT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+
PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6
c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0
OjQuOHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+V2VsbCwgbWFrZSBpdCBub3QgZ28gdGhyb3VnaCBob3N0MSBhbmQgZGVkaWNh
dGUgYSBzdG9yYWdlIHNlcnZlciBmb3IgcnVubmluZyBORlMgYW5kIG1ha2UgYm90aCBob3N0cyBj
b25uZWN0IHRvIGl0LjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibWFyZ2luLWJvdHRvbToxMi4wcHQiPkluIG15IHZpZXcgTkZTIGlzIG11Y2ggZWFz
aWVyIHRvIG1hbmFnZSB0aGFuIGFueSBvdGhlciB0eXBlIG9mIHN0b3JhZ2UsIHNwZWNpYWxseSBG
QyBhbmQgaVNDU0kgYW5kIHBlcmZvcm1hbmNlIGlzIHByZXR0eSBtdWNoIHRoZSBzYW1lLCBzbyB5
b3Ugd29uYHQgZ2V0IGJldHRlciByZXN1bHRzIG90aGVyIHRoYW4gbWFuYWdlbWVudCBnb2luZyB0
byBvdGhlciB0eXBlLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij5GZXJuYW5kbzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+MjAxNy0wNC0xNSA1OjI1IEdNVC0wMzowMCBLb25zdGFudGluIFJhc2tv
c2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20iIHRhcmdldD0iX2Js
YW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0Ozo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K
PC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3Jk
ZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFy
Z2luLWxlZnQ6NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+SGkgZ3V5cywgPG86cD48L286cD48L3A+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+SSBoYXZlIG9uZSBuZnMmbmJzcDtzdG9yYWdlLDxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8
ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+aXQncyBjb25uZWN0ZWQgdGhyb3VnaCBob3N0MS48
bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPmhvc3Qy
IGFsc28gaGFzIGFjY2VzcyB0byBpdCwgSSBjYW4gZWFzaWx5IG1pZ3JhdGUgdm1zJm5ic3A7YmV0
d2VlbiB0aGVtLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIj5UaGUgcXVlc3Rpb24gaXMgLSBpZiBob3N0MSBpcyBkb3duIC0gYWxsIGluZnJhc3Ry
dWN0dXJlIGlzIGRvd24sIHNpbmNlIGFsbCB0cmFmZmljIGdvZXMgdGhyb3VnaCBob3N0MSw8bzpw
PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPmlzIHRoZXJl
IGFueSB3YXkgaW4gb1ZpcnQgdG8gdXNlIHJlZHVuZGFudCBzdG9yYWdlPzxvOnA+PC9vOnA+PC9w
Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48
L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5Pbmx5IGdsdXN0ZXJmcz88
bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+VGhh
bmtzPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48
bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8
ZGl2Pg0KPGRpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpz
b2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6
NC44cHQ7bWFyZ2luLXJpZ2h0OjBjbSI+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibWFy
Z2luLWJvdHRvbToxMi4wcHQiPl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fPGJyPg0KVXNlcnMgbWFpbGluZyBsaXN0PGJyPg0KPGEgaHJlZj0ibWFpbHRvOlVz
ZXJzQG92aXJ0Lm9yZyIgdGFyZ2V0PSJfYmxhbmsiPlVzZXJzQG92aXJ0Lm9yZzwvYT48YnI+DQo8
YSBocmVmPSJodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiIHRh
cmdldD0iX2JsYW5rIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNl
cnM8L2E+PG86cD48L286cD48L3A+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9i
bG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjwvYmxvY2txdW90
ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1hcmdpbi1ib3R0b206MTIuMHB0Ij48YnI+DQpfX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fXzxicj4NClVzZXJzIG1haWxpbmcgbGlzdDxicj4NCjxh
IGhyZWY9Im1haWx0bzpVc2Vyc0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIj5Vc2Vyc0Bvdmly
dC5vcmc8L2E+PGJyPg0KPGEgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xp
c3RpbmZvL3VzZXJzIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWls
bWFuL2xpc3RpbmZvL3VzZXJzPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9ibG9ja3F1b3RlPg0KPC9k
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0K
PHAgY2xhc3M9Ik1zb05vcm1hbCI+X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX188YnI+DQpVc2VycyBtYWlsaW5nIGxpc3Q8YnI+DQo8YSBocmVmPSJtYWlsdG86
VXNlcnNAb3ZpcnQub3JnIiB0YXJnZXQ9Il9ibGFuayI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4N
CjxhIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyIg
dGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91
c2VyczwvYT48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8
L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtYXJnaW4tYm90dG9tOjEyLjBwdCI+
PGJyPg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188YnI+
DQpVc2VycyBtYWlsaW5nIGxpc3Q8YnI+DQo8YSBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3Jn
IiB0YXJnZXQ9Il9ibGFuayI+VXNlcnNAb3ZpcnQub3JnPC9hPjxicj4NCjxhIGhyZWY9Imh0dHA6
Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VycyIgdGFyZ2V0PSJfYmxhbmsi
Pmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91c2VyczwvYT48bzpwPjwv
bzpwPjwvcD4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86
cD4mbmJzcDs8L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+
DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0K
PC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwv
ZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ibG9ja3F1b3Rl
Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4N
CjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8
L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90
ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9i
bG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K
--_000_EF8A6CD043B349ADB14F59EB42E5B511acroniscom_--
5:32 p.m.
On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
Nir,
Isn’t SPM managed via Sanlock? I believe there is no need to fence SPM
host. Especially if there are no SPM tasks running.
It's true that the exclusivity of the SPM role is enforced by Sanlock but
you always need to fence a non-responsive SPM because there is no way to
guarantee that the host is not still manipulating storage (eg. LV
extensions) and we must ensure that only one host has the masterfs on the
master storage domain mounted.
*From: *<users-bounces(a)ovirt.org> on behalf of Nir Soffer <
nsoffer(a)redhat.com>
*Date: *Monday, 17 April 2017 at 16:06
*To: *Konstantin Raskoshnyi <konrasko(a)gmail.com>, Dan Yasny <
dyasny(a)gmail.com>
*Cc: *users <users(a)ovirt.org>, FERNANDO FREDIANI <
fernando.frediani(a)upx.com>
*Subject: *Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
But actually, it didn't work well. After main SPM host went down I see
this
2017-04-17 05:23:15,554Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
Init: could not find reported vds or not up - pool: 'STG' vds_spm_id:
'1'
2017-04-17 05:23:15,567Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds
is non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically switch SPM
host?
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
Makes sense.
I was trying to set it up, but doesn't work with our staging hardware.
We have old ilo100, I'll try again.
Thanks!
It is absolutely necessary for any HA to work properly. There's of course
the "confirm host has been shutdown" option, which serves as an override
for the fence command, but it's manual
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
Fence agent under each node?
When you configure a host, there's the power management tab, where you
need to enter the bmc details for the host. If you don't have fencing
enabled, how do you expect the system to make sure a host running a service
is actually down (and it is safe to start HA services elsewhere), and not,
for example, just unreachable by the engine? How do you avoid a splitbraid
-> SBA ?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
"Corner cases"?
I tried to simulate crash of SPM server and ovirt kept trying to
reistablished connection to the failed node.
Did you configure fencing?
On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
When you set up a storage domain, you need to specify a host to perform
the initial storage operations, but once the SD is defined, it's details
are in the engine database, and all the hosts get connected to it directly.
If the first host you used to define the SD goes down, all other hosts will
still remain connected and work. SPM is an HA service, and if the current
SPM host goes down, SPM gets started on another host in the DC. In short,
unless your actual NFS exporting host goes down, there is no outage.
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's
the whole point of HA services
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
There are always corner cases, no doubt. But in a normal situation. where
an SPM host goes down because of a hardware failure, it gets fenced, other
hosts contend for SPM and start it. No surprises there.
Nir
Nir
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host to
which I connected nfs storage going down the storage becomes unavailable
and all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host.
From what I know any host talk directly to NFS Storage Array or whatever
other Shared Storage you have.
Have you tested that host going down if that affects the other with the
NFS mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Well, make it not go through host1 and dedicate a storage server for
running NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other type of storage,
specially FC and iSCSI and performance is pretty much the same, so you
won`t get better results other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Adam Litke
6:04 p.m.
--_000_87040B69038146B985204DA5C6360EEAacroniscom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
QWRhbSwNCg0KWW91IGtub3csIFNhbmxvY2sgaGFzIHJlY292ZXJ5IG1lY2hhbmlzbSB0aGF0IGtp
bGxzIFZEU00sIG9yIGV2ZW4gdHJpZ2dlcnMgV2F0Y2hkb2cgdG8gcmVib290IFNQTSBob3N0IGlu
IGNhc2UgaXQgaGFzIGxvc3QgdGhlIFNQTSBsb2NrLg0KSeKAmW0gYXNraW5nIGJlY2F1c2UgSSBo
YWQgaXNzdWVzIHdpdGggbXkgbWFzdGVyIHN0b3JhZ2UgdGhhdCBjYXVzZWQgU1BNIGhvc3QgdG8g
cmVib290IGJ5IFdhdGNoZG9nLiBBbmQgSSB3YXMgc3VyZSB0aGF0IGl04oCZcyBhbiBpbnRlbmRl
ZCBiZWhhdmlvdXIuIElzbuKAmXQgaXQ/DQoNCg0KRnJvbTogQWRhbSBMaXRrZSA8YWxpdGtlQHJl
ZGhhdC5jb20+DQpEYXRlOiBNb25kYXksIDE3IEFwcmlsIDIwMTcgYXQgMTc6MzINClRvOiBQYXZl
bCBHYXNoZXYgPFBheEBhY3JvbmlzLmNvbT4NCkNjOiBOaXIgU29mZmVyIDxuc29mZmVyQHJlZGhh
dC5jb20+LCB1c2VycyA8dXNlcnNAb3ZpcnQub3JnPg0KU3ViamVjdDogUmU6IFtvdmlydC11c2Vy
c10gc3RvcmFnZSByZWR1bmRhbmN5IGluIE92aXJ0DQoNCg0KDQpPbiBNb24sIEFwciAxNywgMjAx
NyBhdCA5OjI2IEFNLCBQYXZlbCBHYXNoZXYgPFBheEBhY3JvbmlzLmNvbTxtYWlsdG86UGF4QGFj
cm9uaXMuY29tPj4gd3JvdGU6DQpOaXIsDQoNCklzbuKAmXQgU1BNIG1hbmFnZWQgdmlhIFNhbmxv
Y2s/IEkgYmVsaWV2ZSB0aGVyZSBpcyBubyBuZWVkIHRvIGZlbmNlIFNQTSBob3N0LiBFc3BlY2lh
bGx5IGlmIHRoZXJlIGFyZSBubyBTUE0gdGFza3MgcnVubmluZy4NCg0KSXQncyB0cnVlIHRoYXQg
dGhlIGV4Y2x1c2l2aXR5IG9mIHRoZSBTUE0gcm9sZSBpcyBlbmZvcmNlZCBieSBTYW5sb2NrIGJ1
dCB5b3UgYWx3YXlzIG5lZWQgdG8gZmVuY2UgYSBub24tcmVzcG9uc2l2ZSBTUE0gYmVjYXVzZSB0
aGVyZSBpcyBubyB3YXkgdG8gZ3VhcmFudGVlIHRoYXQgdGhlIGhvc3QgaXMgbm90IHN0aWxsIG1h
bmlwdWxhdGluZyBzdG9yYWdlIChlZy4gTFYgZXh0ZW5zaW9ucykgYW5kIHdlIG11c3QgZW5zdXJl
IHRoYXQgb25seSBvbmUgaG9zdCBoYXMgdGhlIG1hc3RlcmZzIG9uIHRoZSBtYXN0ZXIgc3RvcmFn
ZSBkb21haW4gbW91bnRlZC4NCg0KDQoNCkZyb206IDx1c2Vycy1ib3VuY2VzQG92aXJ0Lm9yZzxt
YWlsdG86dXNlcnMtYm91bmNlc0BvdmlydC5vcmc+PiBvbiBiZWhhbGYgb2YgTmlyIFNvZmZlciA8
bnNvZmZlckByZWRoYXQuY29tPG1haWx0bzpuc29mZmVyQHJlZGhhdC5jb20+Pg0KRGF0ZTogTW9u
ZGF5LCAxNyBBcHJpbCAyMDE3IGF0IDE2OjA2DQpUbzogS29uc3RhbnRpbiBSYXNrb3NobnlpIDxr
b25yYXNrb0BnbWFpbC5jb208bWFpbHRvOmtvbnJhc2tvQGdtYWlsLmNvbT4+LCBEYW4gWWFzbnkg
PGR5YXNueUBnbWFpbC5jb208bWFpbHRvOmR5YXNueUBnbWFpbC5jb20+Pg0KQ2M6IHVzZXJzIDx1
c2Vyc0BvdmlydC5vcmc8bWFpbHRvOnVzZXJzQG92aXJ0Lm9yZz4+LCBGRVJOQU5ETyBGUkVESUFO
SSA8ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbTxtYWlsdG86ZmVybmFuZG8uZnJlZGlhbmlAdXB4
LmNvbT4+DQpTdWJqZWN0OiBSZTogW292aXJ0LXVzZXJzXSBzdG9yYWdlIHJlZHVuZGFuY3kgaW4g
T3ZpcnQNCg0KT24gTW9uLCBBcHIgMTcsIDIwMTcgYXQgODoyNCBBTSBLb25zdGFudGluIFJhc2tv
c2hueWkgPGtvbnJhc2tvQGdtYWlsLmNvbTxtYWlsdG86a29ucmFza29AZ21haWwuY29tPj4gd3Jv
dGU6DQpCdXQgYWN0dWFsbHksIGl0IGRpZG4ndCB3b3JrIHdlbGwuIEFmdGVyIG1haW4gU1BNIGhv
c3Qgd2VudCBkb3duIEkgc2VlIHRoaXMNCg0KDQoyMDE3LTA0LTE3IDA1OjIzOjE1LDU1NFogRVJS
T1IgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuaXJzYnJva2VyLklyc1Byb3h5XSAo
RGVmYXVsdFF1YXJ0elNjaGVkdWxlcjUpIFs0ZGNjMDMzZC0yNmJmLTQ5YmItYmZhYS0wM2E5NzBk
YmJlYzFdIFNQTSBJbml0OiBjb3VsZCBub3QgZmluZCByZXBvcnRlZCB2ZHMgb3Igbm90IHVwIC0g
cG9vbDogJ1NURycgdmRzX3NwbV9pZDogJzEnDQoyMDE3LTA0LTE3IDA1OjIzOjE1LDU2N1ogSU5G
TyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuaXJzYnJva2VyLklyc1Byb3h5XSAo
RGVmYXVsdFF1YXJ0elNjaGVkdWxlcjUpIFs0ZGNjMDMzZC0yNmJmLTQ5YmItYmZhYS0wM2E5NzBk
YmJlYzFdIFNQTSBzZWxlY3Rpb24gLSB2ZHMgc2VlbXMgYXMgc3BtICd0YW5rNScNCjIwMTctMDQt
MTcgMDU6MjM6MTUsNTY3WiBXQVJOICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5p
cnNicm9rZXIuSXJzUHJveHldIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyNSkgWzRkY2MwMzNkLTI2
YmYtNDliYi1iZmFhLTAzYTk3MGRiYmVjMV0gc3BtIHZkcyBpcyBub24gcmVzcG9uc2l2ZSwgc3Rv
cHBpbmcgc3BtIHNlbGVjdGlvbi4NCg0KU28gdGhhdCBtZWFucyBvbmx5IGlmIEJNQyBpcyB1cCBp
dCdzIHBvc3NpYmxlIHRvIGF1dG9tYXRpY2FsbHkgc3dpdGNoICBTUE0gaG9zdD8NCg0KQk1DPw0K
DQpJZiB5b3VyIFNQTSBpcyBubyByZXNwb25zaXZlLCB0aGUgc3lzdGVtIHdpbGwgdHJ5IHRvIGZl
bmNlIGl0LiBEaWQgeW91DQpjb25maWd1cmUgcG93ZXIgbWFuYWdlbWVudCBmb3IgYWxsIGhvc3Rz
PyBkaWQgeW91IGNoZWNrIHRoYXQgaXQNCndvcms/IEhvdyBkaWQgeW91IHNpbXVsYXRlIG5vbi1y
ZXNwb25zaXZlIGhvc3Q/DQoNCklmIHBvd2VyIG1hbmFnZW1lbnQgaXMgbm90IGNvbmZpZ3VyZWQg
b3IgZmFpbCwgdGhlIHN5c3RlbSBjYW5ub3QNCm1vdmUgdGhlIHNwbSB0byBhbm90aGVyIGhvc3Qs
IHVubGVzcyB5b3UgbWFudWFsbHkgY29uZmlybSB0aGF0IHRoZQ0KU1BNIGhvc3Qgd2FzIHJlYm9v
dGVkLg0KDQpOaXINCg0KDQpUaGFua3MNCg0KT24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgODoyOSBQ
TSwgS29uc3RhbnRpbiBSYXNrb3NobnlpIDxrb25yYXNrb0BnbWFpbC5jb208bWFpbHRvOmtvbnJh
c2tvQGdtYWlsLmNvbT4+IHdyb3RlOg0KT2gsIGZlbmNlIGFnZW50IHdvcmtzIGZpbmUgaWYgSSBz
ZWxlY3QgaWxvNCwNClRoYW5rIHlvdSBmb3IgeW91ciBoZWxwIQ0KDQpPbiBTdW4sIEFwciAxNiwg
MjAxNyBhdCA4OjIyIFBNIERhbiBZYXNueSA8ZHlhc255QGdtYWlsLmNvbTxtYWlsdG86ZHlhc255
QGdtYWlsLmNvbT4+IHdyb3RlOg0KT24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgMTE6MTkgUE0sIEtv
bnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21haWwuY29tPG1haWx0bzprb25yYXNrb0Bn
bWFpbC5jb20+PiB3cm90ZToNCk1ha2VzIHNlbnNlLg0KSSB3YXMgdHJ5aW5nIHRvIHNldCBpdCB1
cCwgYnV0IGRvZXNuJ3Qgd29yayB3aXRoIG91ciBzdGFnaW5nIGhhcmR3YXJlLg0KV2UgaGF2ZSBv
bGQgaWxvMTAwLCBJJ2xsIHRyeSBhZ2Fpbi4NClRoYW5rcyENCg0KDQpJdCBpcyBhYnNvbHV0ZWx5
IG5lY2Vzc2FyeSBmb3IgYW55IEhBIHRvIHdvcmsgcHJvcGVybHkuIFRoZXJlJ3Mgb2YgY291cnNl
IHRoZSAiY29uZmlybSBob3N0IGhhcyBiZWVuIHNodXRkb3duIiBvcHRpb24sIHdoaWNoIHNlcnZl
cyBhcyBhbiBvdmVycmlkZSBmb3IgdGhlIGZlbmNlIGNvbW1hbmQsIGJ1dCBpdCdzIG1hbnVhbA0K
DQpPbiBTdW4sIEFwciAxNiwgMjAxNyBhdCA4OjE4IFBNIERhbiBZYXNueSA8ZHlhc255QGdtYWls
LmNvbTxtYWlsdG86ZHlhc255QGdtYWlsLmNvbT4+IHdyb3RlOg0KT24gU3VuLCBBcHIgMTYsIDIw
MTcgYXQgMTE6MTUgUE0sIEtvbnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21haWwuY29t
PG1haWx0bzprb25yYXNrb0BnbWFpbC5jb20+PiB3cm90ZToNCkZlbmNlIGFnZW50IHVuZGVyIGVh
Y2ggbm9kZT8NCg0KV2hlbiB5b3UgY29uZmlndXJlIGEgaG9zdCwgdGhlcmUncyB0aGUgcG93ZXIg
bWFuYWdlbWVudCB0YWIsIHdoZXJlIHlvdSBuZWVkIHRvIGVudGVyIHRoZSBibWMgZGV0YWlscyBm
b3IgdGhlIGhvc3QuIElmIHlvdSBkb24ndCBoYXZlIGZlbmNpbmcgZW5hYmxlZCwgaG93IGRvIHlv
dSBleHBlY3QgdGhlIHN5c3RlbSB0byBtYWtlIHN1cmUgYSBob3N0IHJ1bm5pbmcgYSBzZXJ2aWNl
IGlzIGFjdHVhbGx5IGRvd24gKGFuZCBpdCBpcyBzYWZlIHRvIHN0YXJ0IEhBIHNlcnZpY2VzIGVs
c2V3aGVyZSksIGFuZCBub3QsIGZvciBleGFtcGxlLCBqdXN0IHVucmVhY2hhYmxlIGJ5IHRoZSBl
bmdpbmU/IEhvdyBkbyB5b3UgYXZvaWQgYSBzcGxpdGJyYWlkIC0+IFNCQSA/DQoNCg0KT24gU3Vu
LCBBcHIgMTYsIDIwMTcgYXQgODoxNCBQTSBEYW4gWWFzbnkgPGR5YXNueUBnbWFpbC5jb208bWFp
bHRvOmR5YXNueUBnbWFpbC5jb20+PiB3cm90ZToNCk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDEx
OjEzIFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgPGtvbnJhc2tvQGdtYWlsLmNvbTxtYWlsdG86
a29ucmFza29AZ21haWwuY29tPj4gd3JvdGU6DQoiQ29ybmVyIGNhc2VzIj8NCkkgdHJpZWQgdG8g
c2ltdWxhdGUgY3Jhc2ggb2YgU1BNIHNlcnZlciBhbmQgb3ZpcnQga2VwdCB0cnlpbmcgdG8gcmVp
c3RhYmxpc2hlZCBjb25uZWN0aW9uIHRvIHRoZSBmYWlsZWQgbm9kZS4NCg0KRGlkIHlvdSBjb25m
aWd1cmUgZmVuY2luZz8NCg0KDQoNCk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6MTAgUE0gRGFu
IFlhc255IDxkeWFzbnlAZ21haWwuY29tPG1haWx0bzpkeWFzbnlAZ21haWwuY29tPj4gd3JvdGU6
DQpPbiBTdW4sIEFwciAxNiwgMjAxNyBhdCA3OjI5IEFNLCBOaXIgU29mZmVyIDxuc29mZmVyQHJl
ZGhhdC5jb208bWFpbHRvOm5zb2ZmZXJAcmVkaGF0LmNvbT4+IHdyb3RlOg0KT24gU3VuLCBBcHIg
MTYsIDIwMTcgYXQgMjowNSBQTSBEYW4gWWFzbnkgPGR5YXNueUByZWRoYXQuY29tPG1haWx0bzpk
eWFzbnlAcmVkaGF0LmNvbT4+IHdyb3RlOg0KDQoNCk9uIEFwciAxNiwgMjAxNyA3OjAxIEFNLCAi
TmlyIFNvZmZlciIgPG5zb2ZmZXJAcmVkaGF0LmNvbTxtYWlsdG86bnNvZmZlckByZWRoYXQuY29t
Pj4gd3JvdGU6DQpPbiBTdW4sIEFwciAxNiwgMjAxNyBhdCA0OjE3IEFNIERhbiBZYXNueSA8ZHlh
c255QGdtYWlsLmNvbTxtYWlsdG86ZHlhc255QGdtYWlsLmNvbT4+IHdyb3RlOg0KV2hlbiB5b3Ug
c2V0IHVwIGEgc3RvcmFnZSBkb21haW4sIHlvdSBuZWVkIHRvIHNwZWNpZnkgYSBob3N0IHRvIHBl
cmZvcm0gdGhlIGluaXRpYWwgc3RvcmFnZSBvcGVyYXRpb25zLCBidXQgb25jZSB0aGUgU0QgaXMg
ZGVmaW5lZCwgaXQncyBkZXRhaWxzIGFyZSBpbiB0aGUgZW5naW5lIGRhdGFiYXNlLCBhbmQgYWxs
IHRoZSBob3N0cyBnZXQgY29ubmVjdGVkIHRvIGl0IGRpcmVjdGx5LiBJZiB0aGUgZmlyc3QgaG9z
dCB5b3UgdXNlZCB0byBkZWZpbmUgdGhlIFNEIGdvZXMgZG93biwgYWxsIG90aGVyIGhvc3RzIHdp
bGwgc3RpbGwgcmVtYWluIGNvbm5lY3RlZCBhbmQgd29yay4gU1BNIGlzIGFuIEhBIHNlcnZpY2Us
IGFuZCBpZiB0aGUgY3VycmVudCBTUE0gaG9zdCBnb2VzIGRvd24sIFNQTSBnZXRzIHN0YXJ0ZWQg
b24gYW5vdGhlciBob3N0IGluIHRoZSBEQy4gSW4gc2hvcnQsIHVubGVzcyB5b3VyIGFjdHVhbCBO
RlMgZXhwb3J0aW5nIGhvc3QgZ29lcyBkb3duLCB0aGVyZSBpcyBubyBvdXRhZ2UuDQoNClRoZXJl
IGlzIG5vIHN0b3JhZ2Ugb3V0YWdlLCBidXQgaWYgeW91IHNodXRkb3duIHRoZSBzcG0gaG9zdCwg
dGhlIHNwbSBob3N0DQp3aWxsIG5vdCBtb3ZlIHRvIGEgbmV3IGhvc3QgdW50aWwgdGhlIHNwbSBo
b3N0IGlzIG9ubGluZSBhZ2Fpbiwgb3IgeW91IGNvbmZpcm0NCm1hbnVhbGx5IHRoYXQgdGhlIHNw
bSBob3N0IHdhcyByZWJvb3RlZC4NCg0KSW4gYSBwcm9wZXJseSBjb25maWd1cmVkIHNldHVwIHRo
ZSBTQkEgc2hvdWxkIHRha2UgY2FyZSBvZiB0aGF0LiBUaGF0J3MgdGhlIHdob2xlIHBvaW50IG9m
IEhBIHNlcnZpY2VzDQoNCkluIHNvbWUgY2FzZXMgbGlrZSBwb3dlciBsb3NzIG9yIGhhcmR3YXJl
IGZhaWx1cmUsIHRoZXJlIGlzIG5vIHdheSB0byBzdGFydA0KdGhlIHNwbSBob3N0LCBhbmQgdGhl
IHN5c3RlbSBjYW5ub3QgcmVjb3ZlciBhdXRvbWF0aWNhbGx5Lg0KDQpUaGVyZSBhcmUgYWx3YXlz
IGNvcm5lciBjYXNlcywgbm8gZG91YnQuIEJ1dCBpbiBhIG5vcm1hbCBzaXR1YXRpb24uIHdoZXJl
IGFuIFNQTSBob3N0IGdvZXMgZG93biBiZWNhdXNlIG9mIGEgaGFyZHdhcmUgZmFpbHVyZSwgaXQg
Z2V0cyBmZW5jZWQsIG90aGVyIGhvc3RzIGNvbnRlbmQgZm9yIFNQTSBhbmQgc3RhcnQgaXQuIE5v
IHN1cnByaXNlcyB0aGVyZS4NCg0KDQpOaXINCg0KDQoNCk5pcg0KDQoNCk9uIFNhdCwgQXByIDE1
LCAyMDE3IGF0IDE6NTMgUE0sIEtvbnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21haWwu
Y29tPG1haWx0bzprb25yYXNrb0BnbWFpbC5jb20+PiB3cm90ZToNCkhpIEZlcm5hbmRvLA0KSSBz
ZWUgZWFjaCBob3N0IGhhcyBkaXJlY3QgY29ubmVjdGlvbiBuZnMgbW91bnQsIGJ1dCB5ZXMsIGlm
IG1haW4gaG9zdCB0byB3aGljaCBJIGNvbm5lY3RlZCBuZnMgc3RvcmFnZSBnb2luZyBkb3duIHRo
ZSBzdG9yYWdlIGJlY29tZXMgdW5hdmFpbGFibGUgYW5kIGFsbCB2bXMgYXJlIGRvd24NCg0KDQpP
biBTYXQsIEFwciAxNSwgMjAxNyBhdCAxMDozNyBBTSBGRVJOQU5ETyBGUkVESUFOSSA8ZmVybmFu
ZG8uZnJlZGlhbmlAdXB4LmNvbTxtYWlsdG86ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbT4+IHdy
b3RlOg0KSGVsbG8gS29uc3RhbnRpbi4NClRoYXQgZG9lc25gdCBtYWtlIG11Y2ggc2Vuc2UgbWFr
ZSBhIHdob2xlIGNsdXN0ZXIgZGVwZW5kIG9uIGEgc2luZ2xlIGhvc3QuIEZyb20gd2hhdCBJIGtu
b3cgYW55IGhvc3QgdGFsayBkaXJlY3RseSB0byBORlMgU3RvcmFnZSBBcnJheSBvciB3aGF0ZXZl
ciBvdGhlciBTaGFyZWQgU3RvcmFnZSB5b3UgaGF2ZS4NCkhhdmUgeW91IHRlc3RlZCB0aGF0IGhv
c3QgZ29pbmcgZG93biBpZiB0aGF0IGFmZmVjdHMgdGhlIG90aGVyIHdpdGggdGhlIE5GUyBtb3Vu
dGVkIGRpcmVjdGxseSBpbiBhIE5GUyBTdG9yYWdlIGFycmF5ID8NCkZlcm5hbmRvDQoNCjIwMTct
MDQtMTUgMTI6NDIgR01ULTAzOjAwIEtvbnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21h
aWwuY29tPG1haWx0bzprb25yYXNrb0BnbWFpbC5jb20+PjoNCkluIG92aXJ0IHlvdSBoYXZlIHRv
IGF0dGFjaCBzdG9yYWdlIHRocm91Z2ggc3BlY2lmaWMgaG9zdC4NCklmIGhvc3QgZ29lcyBkb3du
IHN0b3JhZ2UgaXMgbm90IGF2YWlsYWJsZS4NCg0KT24gU2F0LCBBcHIgMTUsIDIwMTcgYXQgNzoz
MSBBTSBGRVJOQU5ETyBGUkVESUFOSSA8ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbTxtYWlsdG86
ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbT4+IHdyb3RlOg0KV2VsbCwgbWFrZSBpdCBub3QgZ28g
dGhyb3VnaCBob3N0MSBhbmQgZGVkaWNhdGUgYSBzdG9yYWdlIHNlcnZlciBmb3IgcnVubmluZyBO
RlMgYW5kIG1ha2UgYm90aCBob3N0cyBjb25uZWN0IHRvIGl0Lg0KSW4gbXkgdmlldyBORlMgaXMg
bXVjaCBlYXNpZXIgdG8gbWFuYWdlIHRoYW4gYW55IG90aGVyIHR5cGUgb2Ygc3RvcmFnZSwgc3Bl
Y2lhbGx5IEZDIGFuZCBpU0NTSSBhbmQgcGVyZm9ybWFuY2UgaXMgcHJldHR5IG11Y2ggdGhlIHNh
bWUsIHNvIHlvdSB3b25gdCBnZXQgYmV0dGVyIHJlc3VsdHMgb3RoZXIgdGhhbiBtYW5hZ2VtZW50
IGdvaW5nIHRvIG90aGVyIHR5cGUuDQpGZXJuYW5kbw0KDQoyMDE3LTA0LTE1IDU6MjUgR01ULTAz
OjAwIEtvbnN0YW50aW4gUmFza29zaG55aSA8a29ucmFza29AZ21haWwuY29tPG1haWx0bzprb25y
YXNrb0BnbWFpbC5jb20+PjoNCkhpIGd1eXMsDQpJIGhhdmUgb25lIG5mcyBzdG9yYWdlLA0KaXQn
cyBjb25uZWN0ZWQgdGhyb3VnaCBob3N0MS4NCmhvc3QyIGFsc28gaGFzIGFjY2VzcyB0byBpdCwg
SSBjYW4gZWFzaWx5IG1pZ3JhdGUgdm1zIGJldHdlZW4gdGhlbS4NCg0KVGhlIHF1ZXN0aW9uIGlz
IC0gaWYgaG9zdDEgaXMgZG93biAtIGFsbCBpbmZyYXN0cnVjdHVyZSBpcyBkb3duLCBzaW5jZSBh
bGwgdHJhZmZpYyBnb2VzIHRocm91Z2ggaG9zdDEsDQppcyB0aGVyZSBhbnkgd2F5IGluIG9WaXJ0
IHRvIHVzZSByZWR1bmRhbnQgc3RvcmFnZT8NCg0KT25seSBnbHVzdGVyZnM/DQoNClRoYW5rcw0K
DQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fDQpVc2Vy
cyBtYWlsaW5nIGxpc3QNClVzZXJzQG92aXJ0Lm9yZzxtYWlsdG86VXNlcnNAb3ZpcnQub3JnPg0K
aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzDQoNCg0KX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX18NClVzZXJzIG1haWxpbmcg
bGlzdA0KVXNlcnNAb3ZpcnQub3JnPG1haWx0bzpVc2Vyc0BvdmlydC5vcmc+DQpodHRwOi8vbGlz
dHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMNCg0KX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX18NClVzZXJzIG1haWxpbmcgbGlzdA0KVXNlcnNA
b3ZpcnQub3JnPG1haWx0bzpVc2Vyc0BvdmlydC5vcmc+DQpodHRwOi8vbGlzdHMub3ZpcnQub3Jn
L21haWxtYW4vbGlzdGluZm8vdXNlcnMNCg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX18NClVzZXJzIG1haWxpbmcgbGlzdA0KVXNlcnNAb3ZpcnQub3JnPG1h
aWx0bzpVc2Vyc0BvdmlydC5vcmc+DQpodHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlz
dGluZm8vdXNlcnMNCg0KDQoNCl9fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fDQpVc2VycyBtYWlsaW5nIGxpc3QNClVzZXJzQG92aXJ0Lm9yZzxtYWlsdG86VXNl
cnNAb3ZpcnQub3JnPg0KaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3Vz
ZXJzDQoNCg0KDQotLQ0KQWRhbSBMaXRrZQ0K
--_000_87040B69038146B985204DA5C6360EEAacroniscom_
Content-Type: text/html; charset="utf-8"
Content-ID: <B553B5567B282947AF72F9C192AD4EE5(a)acronis.com>
Content-Transfer-Encoding: base64
PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo
dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo
dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l
dGEgbmFtZT0iVGl0bGUiIGNvbnRlbnQ9IiI+DQo8bWV0YSBuYW1lPSJLZXl3b3JkcyIgY29udGVu
dD0iIj4NCjxtZXRhIG5hbWU9IkdlbmVyYXRvciIgY29udGVudD0iTWljcm9zb2Z0IFdvcmQgMTUg
KGZpbHRlcmVkIG1lZGl1bSkiPg0KPHN0eWxlPjwhLS0NCi8qIEZvbnQgRGVmaW5pdGlvbnMgKi8N
CkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6IkNhbWJyaWEgTWF0aCI7DQoJcGFub3NlLTE6MiA0
IDUgMyA1IDQgNiAzIDIgNDt9DQpAZm9udC1mYWNlDQoJe2ZvbnQtZmFtaWx5OkNhbGlicmk7DQoJ
cGFub3NlLTE6MiAxNSA1IDIgMiAyIDQgMyAyIDQ7fQ0KLyogU3R5bGUgRGVmaW5pdGlvbnMgKi8N
CnAuTXNvTm9ybWFsLCBsaS5Nc29Ob3JtYWwsIGRpdi5Nc29Ob3JtYWwNCgl7bWFyZ2luOjBjbTsN
CgltYXJnaW4tYm90dG9tOi4wMDAxcHQ7DQoJZm9udC1zaXplOjEyLjBwdDsNCglmb250LWZhbWls
eToiVGltZXMgTmV3IFJvbWFuIjt9DQphOmxpbmssIHNwYW4uTXNvSHlwZXJsaW5rDQoJe21zby1z
dHlsZS1wcmlvcml0eTo5OTsNCgljb2xvcjpibHVlOw0KCXRleHQtZGVjb3JhdGlvbjp1bmRlcmxp
bmU7fQ0KYTp2aXNpdGVkLCBzcGFuLk1zb0h5cGVybGlua0ZvbGxvd2VkDQoJe21zby1zdHlsZS1w
cmlvcml0eTo5OTsNCgljb2xvcjpwdXJwbGU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9
DQpwDQoJe21zby1zdHlsZS1wcmlvcml0eTo5OTsNCgltc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzsN
CgltYXJnaW4tcmlnaHQ6MGNtOw0KCW1zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvOw0KCW1hcmdp
bi1sZWZ0OjBjbTsNCglmb250LXNpemU6MTIuMHB0Ow0KCWZvbnQtZmFtaWx5OiJUaW1lcyBOZXcg
Um9tYW4iO30NCnNwYW4uRW1haWxTdHlsZTE4DQoJe21zby1zdHlsZS10eXBlOnBlcnNvbmFsLXJl
cGx5Ow0KCWZvbnQtZmFtaWx5OkNhbGlicmk7DQoJY29sb3I6d2luZG93dGV4dDt9DQpzcGFuLm1z
b0lucw0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTsNCgltc28tc3R5bGUtbmFtZToiIjsN
Cgl0ZXh0LWRlY29yYXRpb246dW5kZXJsaW5lOw0KCWNvbG9yOnRlYWw7fQ0KLk1zb0NocERlZmF1
bHQNCgl7bXNvLXN0eWxlLXR5cGU6ZXhwb3J0LW9ubHk7DQoJZm9udC1zaXplOjEwLjBwdDt9DQpA
cGFnZSBXb3JkU2VjdGlvbjENCgl7c2l6ZTo1OTUuMHB0IDg0Mi4wcHQ7DQoJbWFyZ2luOjIuMGNt
IDQyLjVwdCAyLjBjbSAzLjBjbTt9DQpkaXYuV29yZFNlY3Rpb24xDQoJe3BhZ2U6V29yZFNlY3Rp
b24xO30NCi0tPjwvc3R5bGU+DQo8L2hlYWQ+DQo8Ym9keSBiZ2NvbG9yPSJ3aGl0ZSIgbGFuZz0i
RU4tR0IiIGxpbms9ImJsdWUiIHZsaW5rPSJwdXJwbGUiPg0KPGRpdiBjbGFzcz0iV29yZFNlY3Rp
b24xIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0
O2ZvbnQtZmFtaWx5OkNhbGlicmk7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPkFkYW0sPG86
cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaTttc28tZmFyZWFzdC1sYW5ndWFnZTpF
Ti1VUyI+PG86cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+
PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaTttc28tZmFy
ZWFzdC1sYW5ndWFnZTpFTi1VUyI+WW91IGtub3csIFNhbmxvY2sgaGFzIHJlY292ZXJ5IG1lY2hh
bmlzbSB0aGF0IGtpbGxzIFZEU00sIG9yIGV2ZW4gdHJpZ2dlcnMgV2F0Y2hkb2cgdG8gcmVib290
IFNQTSBob3N0IGluIGNhc2UgaXQgaGFzIGxvc3QgdGhlIFNQTSBsb2NrLjxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu
MHB0O2ZvbnQtZmFtaWx5OkNhbGlicmk7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPknigJlt
IGFza2luZyBiZWNhdXNlIEkgaGFkIGlzc3VlcyB3aXRoIG15IG1hc3RlciBzdG9yYWdlIHRoYXQg
Y2F1c2VkIFNQTSBob3N0IHRvIHJlYm9vdCBieSBXYXRjaGRvZy4gQW5kIEkgd2FzIHN1cmUgdGhh
dCBpdOKAmXMgYW4gaW50ZW5kZWQgYmVoYXZpb3VyLiBJc27igJl0IGl0PzxvOnA+PC9vOnA+PC9z
cGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEu
MHB0O2ZvbnQtZmFtaWx5OkNhbGlicmk7bXNvLWZhcmVhc3QtbGFuZ3VhZ2U6RU4tVVMiPjxvOnA+
Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxl
PSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OkNhbGlicmk7bXNvLWZhcmVhc3QtbGFuZ3Vh
Z2U6RU4tVVMiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxkaXYgc3R5bGU9ImJvcmRl
cjpub25lO2JvcmRlci10b3A6c29saWQgI0I1QzRERiAxLjBwdDtwYWRkaW5nOjMuMHB0IDBjbSAw
Y20gMGNtIj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxiPjxzcGFuIHN0eWxlPSJmb250LWZhbWls
eTpDYWxpYnJpO2NvbG9yOmJsYWNrIj5Gcm9tOiA8L3NwYW4+DQo8L2I+PHNwYW4gc3R5bGU9ImZv
bnQtZmFtaWx5OkNhbGlicmk7Y29sb3I6YmxhY2siPkFkYW0gTGl0a2UgJmx0O2FsaXRrZUByZWRo
YXQuY29tJmd0Ozxicj4NCjxiPkRhdGU6IDwvYj5Nb25kYXksIDE3IEFwcmlsIDIwMTcgYXQgMTc6
MzI8YnI+DQo8Yj5UbzogPC9iPlBhdmVsIEdhc2hldiAmbHQ7UGF4QGFjcm9uaXMuY29tJmd0Ozxi
cj4NCjxiPkNjOiA8L2I+TmlyIFNvZmZlciAmbHQ7bnNvZmZlckByZWRoYXQuY29tJmd0OywgdXNl
cnMgJmx0O3VzZXJzQG92aXJ0Lm9yZyZndDs8YnI+DQo8Yj5TdWJqZWN0OiA8L2I+UmU6IFtvdmly
dC11c2Vyc10gc3RvcmFnZSByZWR1bmRhbmN5IGluIE92aXJ0PG86cD48L286cD48L3NwYW4+PC9w
Pg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48
L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpw
PjwvcD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5PbiBNb24sIEFwciAxNywgMjAxNyBhdCA5OjI2
IEFNLCBQYXZlbCBHYXNoZXYgJmx0OzxhIGhyZWY9Im1haWx0bzpQYXhAYWNyb25pcy5jb20iIHRh
cmdldD0iX2JsYW5rIj5QYXhAYWNyb25pcy5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwv
cD4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0ND
Q0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFy
Z2luLXJpZ2h0OjBjbSI+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl
PSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNw
YW4gc3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaSI+TmlyLDwvc3Bh
bj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2lu
LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaSI+Jm5ic3A7PC9zcGFuPjxvOnA+PC9v
OnA+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDph
dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEx
LjBwdDtmb250LWZhbWlseTpDYWxpYnJpIj5Jc27igJl0IFNQTSBtYW5hZ2VkIHZpYSBTYW5sb2Nr
PyBJIGJlbGlldmUgdGhlcmUgaXMgbm8gbmVlZCB0byBmZW5jZSBTUE0gaG9zdC4gRXNwZWNpYWxs
eSBpZiB0aGVyZSBhcmUgbm8gU1BNIHRhc2tzIHJ1bm5pbmcuPC9zcGFuPjxvOnA+PC9vOnA+PC9w
Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48bzpwPiZuYnNwOzwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiPkl0J3MgdHJ1ZSB0aGF0IHRoZSBleGNsdXNpdml0eSBvZiB0aGUgU1BNIHJvbGUgaXMg
ZW5mb3JjZWQgYnkgU2FubG9jayBidXQgeW91IGFsd2F5cyBuZWVkIHRvIGZlbmNlIGEgbm9uLXJl
c3BvbnNpdmUgU1BNIGJlY2F1c2UgdGhlcmUgaXMgbm8gd2F5IHRvIGd1YXJhbnRlZSB0aGF0IHRo
ZSBob3N0IGlzIG5vdCBzdGlsbCBtYW5pcHVsYXRpbmcgc3RvcmFnZSAoZWcuIExWIGV4dGVuc2lv
bnMpIGFuZCB3ZSBtdXN0DQogZW5zdXJlIHRoYXQgb25seSBvbmUgaG9zdCBoYXMgdGhlIG1hc3Rl
cmZzIG9uIHRoZSBtYXN0ZXIgc3RvcmFnZSBkb21haW4gbW91bnRlZC48bzpwPjwvbzpwPjwvcD4N
CjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPiZuYnNwOzxvOnA+PC9vOnA+PC9w
Pg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29s
aWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQu
OHB0O21hcmdpbi1yaWdodDowY20iPg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTEuMHB0O2ZvbnQtZmFtaWx5OkNhbGlicmkiPiZu
YnNwOzwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJt
c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4g
c3R5bGU9ImZvbnQtc2l6ZToxMS4wcHQ7Zm9udC1mYW1pbHk6Q2FsaWJyaSI+Jm5ic3A7PC9zcGFu
PjxvOnA+PC9vOnA+PC9wPg0KPGRpdiBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLXRvcDpzb2xp
ZCAjQjVDNERGIDEuMHB0O3BhZGRpbmc6My4wcHQgMGNtIDBjbSAwY20iPg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9t
LWFsdDphdXRvIj48Yj48c3BhbiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaTtjb2xvcjpibGFj
ayI+RnJvbToNCjwvc3Bhbj48L2I+PHNwYW4gc3R5bGU9ImZvbnQtZmFtaWx5OkNhbGlicmk7Y29s
b3I6YmxhY2siPiZsdDs8YSBocmVmPSJtYWlsdG86dXNlcnMtYm91bmNlc0BvdmlydC5vcmciIHRh
cmdldD0iX2JsYW5rIj51c2Vycy1ib3VuY2VzQG92aXJ0Lm9yZzwvYT4mZ3Q7IG9uIGJlaGFsZiBv
ZiBOaXIgU29mZmVyICZsdDs8YSBocmVmPSJtYWlsdG86bnNvZmZlckByZWRoYXQuY29tIiB0YXJn
ZXQ9Il9ibGFuayI+bnNvZmZlckByZWRoYXQuY29tPC9hPiZndDs8YnI+DQo8Yj5EYXRlOiA8L2I+
TW9uZGF5LCAxNyBBcHJpbCAyMDE3IGF0IDE2OjA2PGJyPg0KPGI+VG86IDwvYj5Lb25zdGFudGlu
IFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20iIHRhcmdl
dD0iX2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0OywgRGFuIFlhc255ICZsdDs8YSBo
cmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmR5YXNueUBnbWFp
bC5jb208L2E+Jmd0Ozxicj4NCjxiPkNjOiA8L2I+dXNlcnMgJmx0OzxhIGhyZWY9Im1haWx0bzp1
c2Vyc0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIj51c2Vyc0BvdmlydC5vcmc8L2E+Jmd0Oywg
RkVSTkFORE8gRlJFRElBTkkgJmx0OzxhIGhyZWY9Im1haWx0bzpmZXJuYW5kby5mcmVkaWFuaUB1
cHguY29tIiB0YXJnZXQ9Il9ibGFuayI+ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbTwvYT4mZ3Q7
PGJyPg0KPGI+U3ViamVjdDogPC9iPlJlOiBbb3ZpcnQtdXNlcnNdIHN0b3JhZ2UgcmVkdW5kYW5j
eSBpbiBPdmlydDwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJv
dHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6
YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+T24gTW9uLCBBcHIgMTcsIDIwMTcgYXQg
ODoyNCBBTSBLb25zdGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25yYXNr
b0BnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0OyB3
cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpu
b25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2
LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tdG9wOjUuMHB0O21hcmdpbi1yaWdodDowY207
bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9
Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5CdXQg
YWN0dWFsbHksIGl0IGRpZG4ndCB3b3JrIHdlbGwuIEFmdGVyIG1haW4gU1BNIGhvc3Qgd2VudCBk
b3duIEkgc2VlIHRoaXMNCjxvOnA+PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6
YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20t
YWx0OmF1dG8iPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj4yMDE3LTA0LTE3IDA1OjIzOjE1LDU1NFogRVJST1IgW29yZy5vdmlydC5lbmdp
bmUuY29yZS52ZHNicm9rZXIuaXJzYnJva2VyLklyc1Byb3h5XSAoRGVmYXVsdFF1YXJ0elNjaGVk
dWxlcjUpIFs0ZGNjMDMzZC0yNmJmLTQ5YmItYmZhYS0wM2E5NzBkYmJlYzFdIFNQTSBJbml0OiBj
b3VsZCBub3QgZmluZCByZXBvcnRlZCB2ZHMgb3Igbm90IHVwIC0gcG9vbDogJ1NURycgdmRzX3Nw
bV9pZDogJzEnPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0K
PGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3At
YWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjIwMTctMDQtMTcgMDU6MjM6MTUs
NTY3WiBJTkZPICZuYnNwO1tvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmlyc2Jyb2tl
ci5JcnNQcm94eV0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXI1KSBbNGRjYzAzM2QtMjZiZi00OWJi
LWJmYWEtMDNhOTcwZGJiZWMxXSBTUE0gc2VsZWN0aW9uIC0gdmRzIHNlZW1zIGFzDQogc3BtICd0
YW5rNSc8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
IHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0
byI+MjAxNy0wNC0xNyAwNToyMzoxNSw1NjdaIFdBUk4gJm5ic3A7W29yZy5vdmlydC5lbmdpbmUu
Y29yZS52ZHNicm9rZXIuaXJzYnJva2VyLklyc1Byb3h5XSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxl
cjUpIFs0ZGNjMDMzZC0yNmJmLTQ5YmItYmZhYS0wM2E5NzBkYmJlYzFdIHNwbSB2ZHMgaXMgbm9u
IHJlc3BvbnNpdmUsIHN0b3BwaW5nDQogc3BtIHNlbGVjdGlvbi48bzpwPjwvbzpwPjwvcD4NCjwv
ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1h
bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+
DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPlNvIHRoYXQgbWVhbnMgb25s
eSBpZiBCTUMgaXMgdXAgaXQncyBwb3NzaWJsZSB0byBhdXRvbWF0aWNhbGx5IHN3aXRjaCAmbmJz
cDtTUE0gaG9zdD88bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2
Pg0KPC9ibG9ja3F1b3RlPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph
dXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1h
bHQ6YXV0byI+Qk1DPzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1z
b05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9t
LWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJv
dHRvbS1hbHQ6YXV0byI+SWYgeW91ciBTUE0gaXMgbm8gcmVzcG9uc2l2ZSwgdGhlIHN5c3RlbSB3
aWxsIHRyeSB0byBmZW5jZSBpdC4gRGlkIHlvdSZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+
DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDph
dXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5jb25maWd1cmUgcG93ZXIgbWFuYWdlbWVu
dCBmb3IgYWxsIGhvc3RzPyBkaWQgeW91IGNoZWNrIHRoYXQgaXQmbmJzcDs8bzpwPjwvbzpwPjwv
cD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2lu
LXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+d29yaz8gSG93IGRpZCB5
b3Ugc2ltdWxhdGUgbm9uLXJlc3BvbnNpdmUgaG9zdD8mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwv
ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1h
bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+
DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPklmIHBvd2VyIG1hbmFnZW1l
bnQgaXMgbm90IGNvbmZpZ3VyZWQgb3IgZmFpbCwgdGhlIHN5c3RlbSBjYW5ub3Q8bzpwPjwvbzpw
PjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFy
Z2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+bW92ZSB0aGUgc3Bt
IHRvIGFub3RoZXIgaG9zdCwgdW5sZXNzIHlvdSBtYW51YWxseSBjb25maXJtIHRoYXQgdGhlPG86
cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPlNQTSBo
b3N0IHdhcyByZWJvb3RlZC48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNz
PSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJv
dHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp
bi1ib3R0b20tYWx0OmF1dG8iPk5pcjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJn
aW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJsb2Nr
cXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7
cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tdG9wOjUu
MHB0O21hcmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8ZGl2Pg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28t
bWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxk
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87
bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPlRoYW5rczxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+
DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+
PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1h
bHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+T24gU3VuLCBBcHIgMTYsIDIwMTcg
YXQgODoyOSBQTSwgS29uc3RhbnRpbiBSYXNrb3NobnlpICZsdDs8YSBocmVmPSJtYWlsdG86a29u
cmFza29AZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+a29ucmFza29AZ21haWwuY29tPC9hPiZn
dDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7
Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0
O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJn
aW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPk9oLCBmZW5j
ZSBhZ2VudCB3b3JrcyBmaW5lIGlmIEkgc2VsZWN0IGlsbzQsPG86cD48L286cD48L3A+DQo8L2Rp
dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPlRoYW5rIHlvdSBmb3IgeW91ciBoZWxw
ITxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0
b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i
b3R0b20tYWx0OmF1dG8iPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6MjIgUE0gRGFuIFlhc255
ICZsdDs8YSBocmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsiPmR5
YXNueUBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGJs
b2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4w
cHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tdG9w
OjUuMHB0O21hcmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8ZGl2Pg0KPGRp
dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0
IDExOjE5IFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25y
YXNrb0BnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0
OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTti
b3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7
bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdp
bi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28t
bWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+TWFrZXMgc2Vu
c2UuJm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0
OmF1dG8iPkkgd2FzIHRyeWluZyB0byBzZXQgaXQgdXAsIGJ1dCBkb2Vzbid0IHdvcmsgd2l0aCBv
dXIgc3RhZ2luZyBoYXJkd2FyZS4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28t
bWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+V2UgaGF2ZSBvbGQgaWxvMTAwLCBJJ2xsIHRyeSBhZ2Fp
bi4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6
YXV0byI+VGhhbmtzIQ0KPG86cD48L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+
DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1t
YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpw
PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+
DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRv
cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+SXQgaXMgYWJzb2x1dGVseSBu
ZWNlc3NhcnkgZm9yIGFueSBIQSB0byB3b3JrIHByb3Blcmx5LiBUaGVyZSdzIG9mIGNvdXJzZSB0
aGUgJnF1b3Q7Y29uZmlybSBob3N0IGhhcyBiZWVuIHNodXRkb3duJnF1b3Q7IG9wdGlvbiwgd2hp
Y2ggc2VydmVzIGFzIGFuIG92ZXJyaWRlIGZvciB0aGUgZmVuY2UgY29tbWFuZCwgYnV0IGl0J3MN
CiBtYW51YWw8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0K
PGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJt
c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7
PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTti
b3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7
bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdp
bi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp
bi1ib3R0b20tYWx0OmF1dG8iPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDg6MTggUE0gRGFuIFlh
c255ICZsdDs8YSBocmVmPSJtYWlsdG86ZHlhc255QGdtYWlsLmNvbSIgdGFyZ2V0PSJfYmxhbmsi
PmR5YXNueUBnbWFpbC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K
PGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0Mg
MS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4t
dG9wOjUuMHB0O21hcmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8ZGl2Pg0K
PGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3At
YWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPk9uIFN1biwgQXByIDE2LCAyMDE3
IGF0IDExOjE1IFBNLCBLb25zdGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzpr
b25yYXNrb0BnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+
Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwvcD4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9u
ZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4w
cHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21h
cmdpbi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJt
c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+RmVuY2Ug
YWdlbnQgdW5kZXIgZWFjaCBub2RlPzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Jsb2NrcXVv
dGU+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFs
dDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4N
CjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+V2hlbiB5b3UgY29uZmlndXJlIGEgaG9zdCwgdGhl
cmUncyB0aGUgcG93ZXIgbWFuYWdlbWVudCB0YWIsIHdoZXJlIHlvdSBuZWVkIHRvIGVudGVyIHRo
ZSBibWMgZGV0YWlscyBmb3IgdGhlIGhvc3QuIElmIHlvdSBkb24ndCBoYXZlIGZlbmNpbmcgZW5h
YmxlZCwgaG93IGRvIHlvdSBleHBlY3QgdGhlIHN5c3RlbQ0KIHRvIG1ha2Ugc3VyZSBhIGhvc3Qg
cnVubmluZyBhIHNlcnZpY2UgaXMgYWN0dWFsbHkgZG93biAoYW5kIGl0IGlzIHNhZmUgdG8gc3Rh
cnQgSEEgc2VydmljZXMgZWxzZXdoZXJlKSwgYW5kIG5vdCwgZm9yIGV4YW1wbGUsIGp1c3QgdW5y
ZWFjaGFibGUgYnkgdGhlIGVuZ2luZT8gSG93IGRvIHlvdSBhdm9pZCBhIHNwbGl0YnJhaWQgLSZn
dDsgU0JBID88bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0K
PGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJt
c28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7
PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTti
b3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7
bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdp
bi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph
dXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs
dDphdXRvIj5PbiBTdW4sIEFwciAxNiwgMjAxNyBhdCA4OjE0IFBNIERhbiBZYXNueSAmbHQ7PGEg
aHJlZj0ibWFpbHRvOmR5YXNueUBnbWFpbC5jb20iIHRhcmdldD0iX2JsYW5rIj5keWFzbnlAZ21h
aWwuY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3Rl
IHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRp
bmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDtt
YXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxkaXY+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z
by1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5PbiBTdW4sIEFwciAxNiwgMjAxNyBhdCAxMToxMyBQ
TSwgS29uc3RhbnRpbiBSYXNrb3NobnlpICZsdDs8YSBocmVmPSJtYWlsdG86a29ucmFza29AZ21h
aWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+a29ucmFza29AZ21haWwuY29tPC9hPiZndDsgd3JvdGU6
PG86cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxl
ZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1s
ZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJnaW4tYm90dG9t
OjUuMHB0Ij4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZxdW90O0Nvcm5lciBjYXNl
cyZxdW90Oz8mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+SSB0cmllZCB0byBzaW11bGF0ZSBjcmFzaCBvZiBTUE0gc2VydmVyIGFuZCBv
dmlydCBrZXB0IHRyeWluZyB0byByZWlzdGFibGlzaGVkIGNvbm5lY3Rpb24gdG8gdGhlIGZhaWxl
ZCBub2RlLiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z
by1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0K
PC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNs
YXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2lu
LWJvdHRvbS1hbHQ6YXV0byI+RGlkIHlvdSBjb25maWd1cmUgZmVuY2luZz88bzpwPjwvbzpwPjwv
cD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0
bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rp
dj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0ND
Q0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFy
Z2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4wcHQiPg0KPGRp
dj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4t
dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpw
PjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFy
Z2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48
L286cD48L3A+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28t
bWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+T24gU3VuLCBB
cHIgMTYsIDIwMTcgYXQgODoxMCBQTSBEYW4gWWFzbnkgJmx0OzxhIGhyZWY9Im1haWx0bzpkeWFz
bnlAZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+ZHlhc255QGdtYWlsLmNvbTwvYT4mZ3Q7IHdy
b3RlOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5v
bmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYu
MHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTtt
YXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1h
bHQ6YXV0byI+T24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgNzoyOSBBTSwgTmlyIFNvZmZlciAmbHQ7
PGEgaHJlZj0ibWFpbHRvOm5zb2ZmZXJAcmVkaGF0LmNvbSIgdGFyZ2V0PSJfYmxhbmsiPm5zb2Zm
ZXJAcmVkaGF0LmNvbTwvYT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPGJsb2NrcXVvdGUg
c3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGlu
ZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0LjhwdDttYXJnaW4tdG9wOjUuMHB0O21h
cmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1LjBwdCI+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+
DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNv
LW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPk9uIFN1biwgQXByIDE2LCAyMDE3IGF0IDI6MDUgUE0g
RGFuIFlhc255ICZsdDs8YSBocmVmPSJtYWlsdG86ZHlhc255QHJlZGhhdC5jb20iIHRhcmdldD0i
X2JsYW5rIj5keWFzbnlAcmVkaGF0LmNvbTwvYT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0K
PC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQg
I0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0
O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Ij4N
CjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9w
LWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwv
cD4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0K
PGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0
bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+T24gQXByIDE2LCAyMDE3IDc6MDEgQU0sICZx
dW90O05pciBTb2ZmZXImcXVvdDsgJmx0OzxhIGhyZWY9Im1haWx0bzpuc29mZmVyQHJlZGhhdC5j
b20iIHRhcmdldD0iX2JsYW5rIj5uc29mZmVyQHJlZGhhdC5jb208L2E+Jmd0OyB3cm90ZTo8bzpw
PjwvbzpwPjwvcD4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpz
b2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6
NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4w
cHQiPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0
eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+
T24gU3VuLCBBcHIgMTYsIDIwMTcgYXQgNDoxNyBBTSBEYW4gWWFzbnkgJmx0OzxhIGhyZWY9Im1h
aWx0bzpkeWFzbnlAZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+ZHlhc255QGdtYWlsLmNvbTwv
YT4mZ3Q7IHdyb3RlOjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0i
Ym9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAw
Y20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJp
Z2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPldoZW4geW91IHNldCB1cCBhIHN0b3JhZ2UgZG9tYWluLCB5b3UgbmVlZCB0byBzcGVjaWZ5
IGEgaG9zdCB0byBwZXJmb3JtIHRoZSBpbml0aWFsIHN0b3JhZ2Ugb3BlcmF0aW9ucywgYnV0IG9u
Y2UgdGhlIFNEIGlzIGRlZmluZWQsIGl0J3MgZGV0YWlscyBhcmUgaW4gdGhlIGVuZ2luZSBkYXRh
YmFzZSwgYW5kDQogYWxsIHRoZSBob3N0cyBnZXQgY29ubmVjdGVkIHRvIGl0IGRpcmVjdGx5LiBJ
ZiB0aGUgZmlyc3QgaG9zdCB5b3UgdXNlZCB0byBkZWZpbmUgdGhlIFNEIGdvZXMgZG93biwgYWxs
IG90aGVyIGhvc3RzIHdpbGwgc3RpbGwgcmVtYWluIGNvbm5lY3RlZCBhbmQgd29yay4gU1BNIGlz
IGFuIEhBIHNlcnZpY2UsIGFuZCBpZiB0aGUgY3VycmVudCBTUE0gaG9zdCBnb2VzIGRvd24sIFNQ
TSBnZXRzIHN0YXJ0ZWQgb24gYW5vdGhlciBob3N0IGluIHRoZSBEQy4NCiBJbiBzaG9ydCwgdW5s
ZXNzIHlvdXIgYWN0dWFsIE5GUyBleHBvcnRpbmcgaG9zdCBnb2VzIGRvd24sIHRoZXJlIGlzIG5v
IG91dGFnZS4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4N
CjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRv
cC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+VGhlcmUgaXMgbm8gc3RvcmFn
ZSBvdXRhZ2UsIGJ1dCBpZiB5b3Ugc2h1dGRvd24gdGhlIHNwbSBob3N0LCB0aGUgc3BtIGhvc3Q8
bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxl
PSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+d2ls
bCBub3QgbW92ZSB0byBhIG5ldyBob3N0IHVudGlsIHRoZSBzcG0gaG9zdCBpcyBvbmxpbmUgYWdh
aW4sIG9yIHlvdSBjb25maXJtPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i
b3R0b20tYWx0OmF1dG8iPm1hbnVhbGx5IHRoYXQgdGhlIHNwbSBob3N0IHdhcyByZWJvb3RlZC48
bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwv
ZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNw
OzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90
dG9tLWFsdDphdXRvIj5JbiBhIHByb3Blcmx5IGNvbmZpZ3VyZWQgc2V0dXAgdGhlIFNCQSBzaG91
bGQgdGFrZSBjYXJlIG9mIHRoYXQuIFRoYXQncyB0aGUgd2hvbGUgcG9pbnQgb2YgSEEgc2Vydmlj
ZXMmbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8
ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRv
O21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2
Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6
YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+SW4gc29tZSBjYXNlcyBsaWtlIHBvd2Vy
IGxvc3Mgb3IgaGFyZHdhcmUgZmFpbHVyZSwgdGhlcmUgaXMgbm8gd2F5IHRvIHN0YXJ0PG86cD48
L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPnRoZSBzcG0g
aG9zdCwgYW5kIHRoZSBzeXN0ZW0gY2Fubm90IHJlY292ZXIgYXV0b21hdGljYWxseS48bzpwPjwv
bzpwPjwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjxkaXY+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1h
cmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rp
dj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90
dG9tLWFsdDphdXRvIj5UaGVyZSBhcmUgYWx3YXlzIGNvcm5lciBjYXNlcywgbm8gZG91YnQuIEJ1
dCBpbiBhIG5vcm1hbCBzaXR1YXRpb24uIHdoZXJlIGFuIFNQTSBob3N0IGdvZXMgZG93biBiZWNh
dXNlIG9mIGEgaGFyZHdhcmUgZmFpbHVyZSwgaXQgZ2V0cyBmZW5jZWQsIG90aGVyIGhvc3RzIGNv
bnRlbmQgZm9yIFNQTSBhbmQgc3RhcnQNCiBpdC4gTm8gc3VycHJpc2VzIHRoZXJlLiZuYnNwOzxv
OnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRp
dj4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4t
dG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpw
PjwvcD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0
OnNvbGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVm
dDo0LjhwdDttYXJnaW4tdG9wOjUuMHB0O21hcmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1
LjBwdCI+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFu
IHN0eWxlPSJjb2xvcjojODg4ODg4Ij4mbmJzcDs8L3NwYW4+PG86cD48L286cD48L3A+DQo8L2Rp
dj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0
OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPjxzcGFuIHN0eWxlPSJjb2xvcjojODg4
ODg4Ij5OaXI8L3NwYW4+PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4N
CjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0ND
IDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2lu
LXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4wcHQiPg0KPGRpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1
dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9k
aXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7
Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0
O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJn
aW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6
YXV0byI+PHNwYW4gc3R5bGU9ImNvbG9yOiM4ODg4ODgiPiZuYnNwOzwvc3Bhbj48bzpwPjwvbzpw
PjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFy
Z2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+PHNwYW4gc3R5bGU9
ImNvbG9yOiM4ODg4ODgiPk5pcjwvc3Bhbj48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1
dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9k
aXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0ND
Q0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21h
cmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxk
aXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87
bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPGRpdj4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28t
bWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+T24gU2F0LCBBcHIgMTUsIDIwMTcgYXQgMTo1MyBQTSwg
S29uc3RhbnRpbiBSYXNrb3NobnlpICZsdDs8YSBocmVmPSJtYWlsdG86a29ucmFza29AZ21haWwu
Y29tIiB0YXJnZXQ9Il9ibGFuayI+a29ucmFza29AZ21haWwuY29tPC9hPiZndDsgd3JvdGU6PG86
cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6
c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0
OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBjbTttYXJnaW4tYm90dG9tOjUu
MHB0Ij4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3At
YWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPkhpIEZlcm5hbmRvLDxvOnA+PC9v
OnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1t
YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5JIHNlZSBlYWNo
IGhvc3QgaGFzIGRpcmVjdCBjb25uZWN0aW9uIG5mcyBtb3VudCwgYnV0IHllcywgaWYgbWFpbiBo
b3N0IHRvIHdoaWNoIEkgY29ubmVjdGVkIG5mcyBzdG9yYWdlIGdvaW5nIGRvd24gdGhlIHN0b3Jh
Z2UgYmVjb21lcyB1bmF2YWlsYWJsZSBhbmQgYWxsIHZtcyBhcmUgZG93bjxvOnA+PC9vOnA+PC9w
Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHls
ZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZu
YnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv
Ij4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph
dXRvIj5PbiBTYXQsIEFwciAxNSwgMjAxNyBhdCAxMDozNyBBTSBGRVJOQU5ETyBGUkVESUFOSSAm
bHQ7PGEgaHJlZj0ibWFpbHRvOmZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb20iIHRhcmdldD0iX2Js
YW5rIj5mZXJuYW5kby5mcmVkaWFuaUB1cHguY29tPC9hPiZndDsgd3JvdGU6PG86cD48L286cD48
L3A+DQo8L2Rpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3JkZXI6bm9uZTtib3JkZXItbGVmdDpz
b2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAwY20gNi4wcHQ7bWFyZ2luLWxlZnQ6
NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6MGNtO21hcmdpbi1ib3R0b206NS4w
cHQiPg0KPGRpdj4NCjxkaXY+DQo8ZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0
eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJnaW4tYm90dG9tOjEyLjBwdCI+SGVsbG8g
S29uc3RhbnRpbi48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv
Ij5UaGF0IGRvZXNuYHQgbWFrZSBtdWNoIHNlbnNlIG1ha2UgYSB3aG9sZSBjbHVzdGVyIGRlcGVu
ZCBvbiBhIHNpbmdsZSBob3N0LiBGcm9tIHdoYXQgSSBrbm93IGFueSBob3N0IHRhbGsgZGlyZWN0
bHkgdG8gTkZTIFN0b3JhZ2UgQXJyYXkgb3Igd2hhdGV2ZXIgb3RoZXIgU2hhcmVkIFN0b3JhZ2Ug
eW91IGhhdmUuPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0
eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJnaW4tYm90dG9tOjEyLjBwdCI+SGF2ZSB5
b3UgdGVzdGVkIHRoYXQgaG9zdCBnb2luZyBkb3duIGlmIHRoYXQgYWZmZWN0cyB0aGUgb3RoZXIg
d2l0aCB0aGUgTkZTIG1vdW50ZWQgZGlyZWN0bGx5IGluIGEgTkZTIFN0b3JhZ2UgYXJyYXkgPzxv
OnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
IiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1
dG8iPkZlcm5hbmRvPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20t
YWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3Jt
YWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6
YXV0byI+MjAxNy0wNC0xNSAxMjo0MiBHTVQtMDM6MDAgS29uc3RhbnRpbiBSYXNrb3NobnlpICZs
dDs8YSBocmVmPSJtYWlsdG86a29ucmFza29AZ21haWwuY29tIiB0YXJnZXQ9Il9ibGFuayI+a29u
cmFza29AZ21haWwuY29tPC9hPiZndDs6PG86cD48L286cD48L3A+DQo8YmxvY2txdW90ZSBzdHls
ZT0iYm9yZGVyOm5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBj
bSAwY20gMGNtIDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2lu
LXJpZ2h0OjBjbTttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0
OmF1dG8iPkluIG92aXJ0IHlvdSBoYXZlIHRvIGF0dGFjaCBzdG9yYWdlIHRocm91Z2ggc3BlY2lm
aWMgaG9zdC4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+SWYgaG9zdCBnb2VzIGRvd24gc3RvcmFnZSBpcyBub3QgYXZhaWxhYmxlLiZu
YnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIg
c3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRv
Ij4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjxkaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDph
dXRvIj5PbiBTYXQsIEFwciAxNSwgMjAxNyBhdCA3OjMxIEFNIEZFUk5BTkRPIEZSRURJQU5JICZs
dDs8YSBocmVmPSJtYWlsdG86ZmVybmFuZG8uZnJlZGlhbmlAdXB4LmNvbSIgdGFyZ2V0PSJfYmxh
bmsiPmZlcm5hbmRvLmZyZWRpYW5pQHVweC5jb208L2E+Jmd0OyB3cm90ZTo8bzpwPjwvbzpwPjwv
cD4NCjwvZGl2Pg0KPGJsb2NrcXVvdGUgc3R5bGU9ImJvcmRlcjpub25lO2JvcmRlci1sZWZ0OnNv
bGlkICNDQ0NDQ0MgMS4wcHQ7cGFkZGluZzowY20gMGNtIDBjbSA2LjBwdDttYXJnaW4tbGVmdDo0
LjhwdDttYXJnaW4tdG9wOjUuMHB0O21hcmdpbi1yaWdodDowY207bWFyZ2luLWJvdHRvbTo1LjBw
dCI+DQo8ZGl2Pg0KPGRpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNv
LW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPldlbGwsIG1h
a2UgaXQgbm90IGdvIHRocm91Z2ggaG9zdDEgYW5kIGRlZGljYXRlIGEgc3RvcmFnZSBzZXJ2ZXIg
Zm9yIHJ1bm5pbmcgTkZTIGFuZCBtYWtlIGJvdGggaG9zdHMgY29ubmVjdCB0byBpdC48bzpwPjwv
bzpwPjwvcD4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4t
dG9wLWFsdDphdXRvO21hcmdpbi1ib3R0b206MTIuMHB0Ij5JbiBteSB2aWV3IE5GUyBpcyBtdWNo
IGVhc2llciB0byBtYW5hZ2UgdGhhbiBhbnkgb3RoZXIgdHlwZSBvZiBzdG9yYWdlLCBzcGVjaWFs
bHkgRkMgYW5kIGlTQ1NJIGFuZCBwZXJmb3JtYW5jZSBpcyBwcmV0dHkgbXVjaCB0aGUgc2FtZSwg
c28geW91IHdvbmB0IGdldCBiZXR0ZXIgcmVzdWx0cyBvdGhlciB0aGFuIG1hbmFnZW1lbnQNCiBn
b2luZyB0byBvdGhlciB0eXBlLjxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20t
YWx0OmF1dG8iPkZlcm5hbmRvPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i
b3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPGRp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
c28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+MjAxNy0wNC0xNSA1OjI1IEdNVC0wMzowMCBLb25z
dGFudGluIFJhc2tvc2hueWkgJmx0OzxhIGhyZWY9Im1haWx0bzprb25yYXNrb0BnbWFpbC5jb20i
IHRhcmdldD0iX2JsYW5rIj5rb25yYXNrb0BnbWFpbC5jb208L2E+Jmd0Ozo8bzpwPjwvbzpwPjwv
cD4NCjwvZGl2Pg0KPC9kaXY+DQo8ZGl2Pg0KPGRpdj4NCjxibG9ja3F1b3RlIHN0eWxlPSJib3Jk
ZXI6bm9uZTtib3JkZXItbGVmdDpzb2xpZCAjQ0NDQ0NDIDEuMHB0O3BhZGRpbmc6MGNtIDBjbSAw
Y20gNi4wcHQ7bWFyZ2luLWxlZnQ6NC44cHQ7bWFyZ2luLXRvcDo1LjBwdDttYXJnaW4tcmlnaHQ6
MGNtO21hcmdpbi1ib3R0b206NS4wcHQiPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0
eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+
SGkgZ3V5cywNCjxvOnA+PC9vOnA+PC9wPg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0
eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+
SSBoYXZlIG9uZSBuZnMmbmJzcDtzdG9yYWdlLDxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21z
by1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5pdCdzIGNvbm5lY3RlZCB0aHJvdWdoIGhvc3QxLjxv
OnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9
Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5ob3N0
MiBhbHNvIGhhcyBhY2Nlc3MgdG8gaXQsIEkgY2FuIGVhc2lseSBtaWdyYXRlIHZtcyZuYnNwO2Jl
dHdlZW4gdGhlbS48bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29O
b3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1h
bHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0
b20tYWx0OmF1dG8iPlRoZSBxdWVzdGlvbiBpcyAtIGlmIGhvc3QxIGlzIGRvd24gLSBhbGwgaW5m
cmFzdHJ1Y3R1cmUgaXMgZG93biwgc2luY2UgYWxsIHRyYWZmaWMgZ29lcyB0aHJvdWdoIGhvc3Qx
LDxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5
bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj5p
cyB0aGVyZSBhbnkgd2F5IGluIG9WaXJ0IHRvIHVzZSByZWR1bmRhbnQgc3RvcmFnZT88bzpwPjwv
bzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28t
bWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86
cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0i
bXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPk9ubHkg
Z2x1c3RlcmZzPzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8ZGl2Pg0KPHAgY2xhc3M9Ik1zb05v
cm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFs
dDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPGRpdj4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFyZ2luLWJvdHRv
bS1hbHQ6YXV0byI+VGhhbmtzPG86cD48L286cD48L3A+DQo8L2Rpdj4NCjxkaXY+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1i
b3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bzttc28tbWFy
Z2luLWJvdHRvbS1hbHQ6YXV0byI+Jm5ic3A7PG86cD48L286cD48L3A+DQo8L2Jsb2NrcXVvdGU+
DQo8L2Rpdj4NCjwvZGl2Pg0KPGRpdj4NCjxkaXY+DQo8YmxvY2txdW90ZSBzdHlsZT0iYm9yZGVy
Om5vbmU7Ym9yZGVyLWxlZnQ6c29saWQgI0NDQ0NDQyAxLjBwdDtwYWRkaW5nOjBjbSAwY20gMGNt
IDYuMHB0O21hcmdpbi1sZWZ0OjQuOHB0O21hcmdpbi10b3A6NS4wcHQ7bWFyZ2luLXJpZ2h0OjBj
bTttYXJnaW4tYm90dG9tOjUuMHB0Ij4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28t
bWFyZ2luLXRvcC1hbHQ6YXV0bzttYXJnaW4tYm90dG9tOjEyLjBwdCI+X19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX188YnI+DQpVc2VycyBtYWlsaW5nIGxpc3Q8
YnI+DQo8YSBocmVmPSJtYWlsdG86VXNlcnNAb3ZpcnQub3JnIiB0YXJnZXQ9Il9ibGFuayI+VXNl
cnNAb3ZpcnQub3JnPC9hPjxicj4NCjxhIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFp
bG1hbi9saXN0aW5mby91c2VycyIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9saXN0cy5vdmlydC5v
cmcvbWFpbG1hbi9saXN0aW5mby91c2VyczwvYT48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90
ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ibG9j
a3F1b3RlPg0KPC9kaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10
b3AtYWx0OmF1dG87bXNvLW1hcmdpbi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+
PC9wPg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rp
dj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiIHN0eWxlPSJtc28tbWFyZ2luLXRvcC1hbHQ6YXV0bztt
YXJnaW4tYm90dG9tOjEyLjBwdCI+PGJyPg0KX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fX188YnI+DQpVc2VycyBtYWlsaW5nIGxpc3Q8YnI+DQo8YSBocmVmPSJt
YWlsdG86VXNlcnNAb3ZpcnQub3JnIiB0YXJnZXQ9Il9ibGFuayI+VXNlcnNAb3ZpcnQub3JnPC9h
Pjxicj4NCjxhIGhyZWY9Imh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0aW5mby91
c2VycyIgdGFyZ2V0PSJfYmxhbmsiPmh0dHA6Ly9saXN0cy5vdmlydC5vcmcvbWFpbG1hbi9saXN0
aW5mby91c2VyczwvYT48bzpwPjwvbzpwPjwvcD4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPHAg
Y2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJn
aW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpwPjwvbzpwPjwvcD4NCjwvZGl2Pg0KPHAgY2xh
c3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4t
Ym90dG9tLWFsdDphdXRvIj5fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fXzxicj4NClVzZXJzIG1haWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1haWx0bzpVc2Vy
c0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIj5Vc2Vyc0BvdmlydC5vcmc8L2E+PGJyPg0KPGEg
aHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJzIiB0YXJn
ZXQ9Il9ibGFuayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3VzZXJz
PC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1tYXJnaW4tdG9wLWFsdDphdXRvO21h
cmdpbi1ib3R0b206MTIuMHB0Ij48YnI+DQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fXzxicj4NClVzZXJzIG1haWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1h
aWx0bzpVc2Vyc0BvdmlydC5vcmciIHRhcmdldD0iX2JsYW5rIj5Vc2Vyc0BvdmlydC5vcmc8L2E+
PGJyPg0KPGEgaHJlZj0iaHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3RpbmZvL3Vz
ZXJzIiB0YXJnZXQ9Il9ibGFuayI+aHR0cDovL2xpc3RzLm92aXJ0Lm9yZy9tYWlsbWFuL2xpc3Rp
bmZvL3VzZXJzPC9hPjxvOnA+PC9vOnA+PC9wPg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIiBzdHlsZT0ibXNvLW1hcmdpbi10b3AtYWx0OmF1dG87bXNvLW1hcmdp
bi1ib3R0b20tYWx0OmF1dG8iPiZuYnNwOzxvOnA+PC9vOnA+PC9wPg0KPC9kaXY+DQo8L2Rpdj4N
CjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8
L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rp
dj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+
DQo8L2Rpdj4NCjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0K
PC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9k
aXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2
Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4N
CjwvYmxvY2txdW90ZT4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1zby1t
YXJnaW4tdG9wLWFsdDphdXRvO21zby1tYXJnaW4tYm90dG9tLWFsdDphdXRvIj4mbmJzcDs8bzpw
PjwvbzpwPjwvcD4NCjwvZGl2Pg0KPC9ibG9ja3F1b3RlPg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2
Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCIgc3R5bGU9Im1h
cmdpbi1ib3R0b206MTIuMHB0Ij48YnI+DQpfX19fX19fX19fX19fX19fX19fX19fX19fX19fX19f
X19fX19fX19fX19fX19fXzxicj4NClVzZXJzIG1haWxpbmcgbGlzdDxicj4NCjxhIGhyZWY9Im1h
aWx0bzpVc2Vyc0BvdmlydC5vcmciPlVzZXJzQG92aXJ0Lm9yZzwvYT48YnI+DQo8YSBocmVmPSJo
dHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnMiIHRhcmdldD0iX2Js
YW5rIj5odHRwOi8vbGlzdHMub3ZpcnQub3JnL21haWxtYW4vbGlzdGluZm8vdXNlcnM8L2E+PG86
cD48L286cD48L3A+DQo8L2Jsb2NrcXVvdGU+DQo8L2Rpdj4NCjxwIGNsYXNzPSJNc29Ob3JtYWwi
Pjxicj4NCjxiciBjbGVhcj0iYWxsIj4NCjxicj4NCi0tIDxvOnA+PC9vOnA+PC9wPg0KPGRpdj4N
CjxkaXY+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj5BZGFtIExpdGtlPG86cD48L286cD48L3A+DQo8
L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvZGl2Pg0KPC9ib2R5Pg0KPC9odG1sPg0K
--_000_87040B69038146B985204DA5C6360EEAacroniscom_--
6:54 p.m.
On Mon, Apr 17, 2017 at 11:04 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
Adam,
You know, Sanlock has recovery mechanism that kills VDSM, or even triggers
Watchdog to reboot SPM host in case it has lost the SPM lock.
I’m asking because I had issues with my master storage that caused SPM
host to reboot by Watchdog. And I was sure that it’s an intended behaviour.
Isn’t it?
Yes of course. But an SPM host can fail but still maintain its connection
to the storage lease. In this case still you need classic fencing.
Something new we are investigating is the use of sanlock's request feature
which allows a new host to take the lease away from the current holder.
The current holder would be fenced by sanlock (watchdog if necessary) and
only once the lease is free would it be granted to the new requester.
*From: *Adam Litke <alitke(a)redhat.com>
*Date: *Monday, 17 April 2017 at 17:32
*To: *Pavel Gashev <Pax(a)acronis.com>
*Cc: *Nir Soffer <nsoffer(a)redhat.com>, users <users(a)ovirt.org>
*Subject: *Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
Nir,
Isn’t SPM managed via Sanlock? I believe there is no need to fence SPM
host. Especially if there are no SPM tasks running.
It's true that the exclusivity of the SPM role is enforced by Sanlock but
you always need to fence a non-responsive SPM because there is no way to
guarantee that the host is not still manipulating storage (eg. LV
extensions) and we must ensure that only one host has the masterfs on the
master storage domain mounted.
*From: *<users-bounces(a)ovirt.org> on behalf of Nir Soffer <
nsoffer(a)redhat.com>
*Date: *Monday, 17 April 2017 at 16:06
*To: *Konstantin Raskoshnyi <konrasko(a)gmail.com>, Dan Yasny <
dyasny(a)gmail.com>
*Cc: *users <users(a)ovirt.org>, FERNANDO FREDIANI <
fernando.frediani(a)upx.com>
*Subject: *Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
But actually, it didn't work well. After main SPM host went down I see
this
2017-04-17 05:23:15,554Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
Init: could not find reported vds or not up - pool: 'STG' vds_spm_id:
'1'
2017-04-17 05:23:15,567Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
(DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds
is non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically switch SPM
host?
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
Makes sense.
I was trying to set it up, but doesn't work with our staging hardware.
We have old ilo100, I'll try again.
Thanks!
It is absolutely necessary for any HA to work properly. There's of course
the "confirm host has been shutdown" option, which serves as an override
for the fence command, but it's manual
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
Fence agent under each node?
When you configure a host, there's the power management tab, where you
need to enter the bmc details for the host. If you don't have fencing
enabled, how do you expect the system to make sure a host running a service
is actually down (and it is safe to start HA services elsewhere), and not,
for example, just unreachable by the engine? How do you avoid a splitbraid
-> SBA ?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
konrasko(a)gmail.com> wrote:
"Corner cases"?
I tried to simulate crash of SPM server and ovirt kept trying to
reistablished connection to the failed node.
Did you configure fencing?
On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
When you set up a storage domain, you need to specify a host to perform
the initial storage operations, but once the SD is defined, it's details
are in the engine database, and all the hosts get connected to it directly.
If the first host you used to define the SD goes down, all other hosts will
still remain connected and work. SPM is an HA service, and if the current
SPM host goes down, SPM gets started on another host in the DC. In short,
unless your actual NFS exporting host goes down, there is no outage.
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's
the whole point of HA services
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
There are always corner cases, no doubt. But in a normal situation. where
an SPM host goes down because of a hardware failure, it gets fenced, other
hosts contend for SPM and start it. No surprises there.
Nir
Nir
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host to
which I connected nfs storage going down the storage becomes unavailable
and all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host.
From what I know any host talk directly to NFS Storage Array or whatever
other Shared Storage you have.
Have you tested that host going down if that affects the other with the
NFS mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
fernando.frediani(a)upx.com> wrote:
Well, make it not go through host1 and dedicate a storage server for
running NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other type of storage,
specially FC and iSCSI and performance is pretty much the same, so you
won`t get better results other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Adam Litke
--
Adam Litke
8:39 p.m.
On Mon, Apr 17, 2017 at 6:54 PM Adam Litke <alitke(a)redhat.com> wrote:
On Mon, Apr 17, 2017 at 11:04 AM, Pavel Gashev
<Pax(a)acronis.com> wrote:
> Adam,
>
>
>
> You know, Sanlock has recovery mechanism that kills VDSM, or even
> triggers Watchdog to reboot SPM host in case it has lost the SPM lock.
>
> I’m asking because I had issues with my master storage that caused SPM
> host to reboot by Watchdog. And I was sure that it’s an intended behaviour.
> Isn’t it?
>
Yes of course. But an SPM host can fail but still maintain its connection
to the storage lease. In this case still you need classic fencing.
Something new we are investigating is the use of sanlock's request feature
which allows a new host to take the lease away from the current holder.
The current holder would be fenced by sanlock (watchdog if necessary) and
only once the lease is free would it be granted to the new requester.
We can use the SPM lease to kill vdsm on the non-responsive SPM host,
and start the SPM on another host, similar to the way we handle vms with
a lease.
But this does not help with the masterfs mounted on the SPM host. if vdsm
is killed before it unmount it, strating the SPM on another host (and
mounting
the msasterfs on the new host) will corrupt the masterfs.
When using file based storage (nfs, glusterfs) we don't have a masterfs so
killing vdsm on the SPM should be good enough to start the SPM on another
host, even if fencing is not possible.
We can start with enabling sanlock based SPM fencing on file based storage.
Nir
>
>
>
>
> *From: *Adam Litke <alitke(a)redhat.com>
> *Date: *Monday, 17 April 2017 at 17:32
> *To: *Pavel Gashev <Pax(a)acronis.com>
> *Cc: *Nir Soffer <nsoffer(a)redhat.com>, users <users(a)ovirt.org>
>
> *Subject: *Re: [ovirt-users] storage redundancy in Ovirt
>
>
>
>
>
>
>
> On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
>
> Nir,
>
>
>
> Isn’t SPM managed via Sanlock? I believe there is no need to fence SPM
> host. Especially if there are no SPM tasks running.
>
>
>
> It's true that the exclusivity of the SPM role is enforced by Sanlock but
> you always need to fence a non-responsive SPM because there is no way to
> guarantee that the host is not still manipulating storage (eg. LV
> extensions) and we must ensure that only one host has the masterfs on the
> master storage domain mounted.
>
>
>
>
>
>
>
> *From: *<users-bounces(a)ovirt.org> on behalf of Nir Soffer <
> nsoffer(a)redhat.com>
> *Date: *Monday, 17 April 2017 at 16:06
> *To: *Konstantin Raskoshnyi <konrasko(a)gmail.com>, Dan Yasny <
> dyasny(a)gmail.com>
> *Cc: *users <users(a)ovirt.org>, FERNANDO FREDIANI <
> fernando.frediani(a)upx.com>
> *Subject: *Re: [ovirt-users] storage redundancy in Ovirt
>
>
>
> On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com>
> wrote:
>
> But actually, it didn't work well. After main SPM host went down I see
> this
>
>
>
>
>
> 2017-04-17 05:23:15,554Z ERROR
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init:
> could not find reported vds or not up - pool: 'STG' vds_spm_id: '1'
>
> 2017-04-17 05:23:15,567Z INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
> selection - vds seems as spm 'tank5'
>
> 2017-04-17 05:23:15,567Z WARN
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is
> non responsive, stopping spm selection.
>
>
>
> So that means only if BMC is up it's possible to automatically switch
> SPM host?
>
>
>
> BMC?
>
>
>
> If your SPM is no responsive, the system will try to fence it. Did you
>
> configure power management for all hosts? did you check that it
>
> work? How did you simulate non-responsive host?
>
>
>
> If power management is not configured or fail, the system cannot
>
> move the spm to another host, unless you manually confirm that the
>
> SPM host was rebooted.
>
>
>
> Nir
>
>
>
>
>
> Thanks
>
>
>
> On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
> Oh, fence agent works fine if I select ilo4,
>
> Thank you for your help!
>
>
>
> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
> Makes sense.
>
> I was trying to set it up, but doesn't work with our staging hardware.
>
> We have old ilo100, I'll try again.
>
> Thanks!
>
>
>
>
>
> It is absolutely necessary for any HA to work properly. There's of course
> the "confirm host has been shutdown" option, which serves as an override
> for the fence command, but it's manual
>
>
>
> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
> Fence agent under each node?
>
>
>
> When you configure a host, there's the power management tab, where you
> need to enter the bmc details for the host. If you don't have fencing
> enabled, how do you expect the system to make sure a host running a service
> is actually down (and it is safe to start HA services elsewhere), and not,
> for example, just unreachable by the engine? How do you avoid a splitbraid
> -> SBA ?
>
>
>
>
>
> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
> "Corner cases"?
>
> I tried to simulate crash of SPM server and ovirt kept trying to
> reistablished connection to the failed node.
>
>
>
> Did you configure fencing?
>
>
>
>
>
>
>
> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>
> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>
> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
>
>
>
>
>
> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
>
> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>
> When you set up a storage domain, you need to specify a host to perform
> the initial storage operations, but once the SD is defined, it's details
> are in the engine database, and all the hosts get connected to it directly.
> If the first host you used to define the SD goes down, all other hosts will
> still remain connected and work. SPM is an HA service, and if the current
> SPM host goes down, SPM gets started on another host in the DC. In short,
> unless your actual NFS exporting host goes down, there is no outage.
>
>
>
> There is no storage outage, but if you shutdown the spm host, the spm host
>
> will not move to a new host until the spm host is online again, or you
> confirm
>
> manually that the spm host was rebooted.
>
>
>
> In a properly configured setup the SBA should take care of that. That's
> the whole point of HA services
>
>
>
> In some cases like power loss or hardware failure, there is no way to
> start
>
> the spm host, and the system cannot recover automatically.
>
>
>
> There are always corner cases, no doubt. But in a normal situation. where
> an SPM host goes down because of a hardware failure, it gets fenced, other
> hosts contend for SPM and start it. No surprises there.
>
>
>
>
>
> Nir
>
>
>
>
>
>
>
> Nir
>
>
>
>
>
> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
> Hi Fernando,
>
> I see each host has direct connection nfs mount, but yes, if main host to
> which I connected nfs storage going down the storage becomes unavailable
> and all vms are down
>
>
>
>
>
> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
> Hello Konstantin.
>
> That doesn`t make much sense make a whole cluster depend on a single
> host. From what I know any host talk directly to NFS Storage Array or
> whatever other Shared Storage you have.
>
> Have you tested that host going down if that affects the other with the
> NFS mounted directlly in a NFS Storage array ?
>
> Fernando
>
>
>
> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>
> In ovirt you have to attach storage through specific host.
>
> If host goes down storage is not available.
>
>
>
> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
> fernando.frediani(a)upx.com> wrote:
>
> Well, make it not go through host1 and dedicate a storage server for
> running NFS and make both hosts connect to it.
>
> In my view NFS is much easier to manage than any other type of storage,
> specially FC and iSCSI and performance is pretty much the same, so you
> won`t get better results other than management going to other type.
>
> Fernando
>
>
>
> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>
> Hi guys,
>
> I have one nfs storage,
>
> it's connected through host1.
>
> host2 also has access to it, I can easily migrate vms between them.
>
>
>
> The question is - if host1 is down - all infrastructure is down, since
> all traffic goes through host1,
>
> is there any way in oVirt to use redundant storage?
>
>
>
> Only glusterfs?
>
>
>
> Thanks
>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
>
> --
>
> Adam Litke
>
--
Adam Litke
12:23 a.m.
--_000_3162c49395404987a301406953a54eb4acroniscom_
Content-Type: text/plain; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
Nir,
A process can chdir into mount point and then lazy umount it. Filesystem re=
mains mounted while the process exists and current directory is on mounted =
filesystem.
# struncate -s 1g masterfs.img
# mkfs.ext4 masterfs.img
# mkdir masterfs
# mount -o loop masterfs.img masterfs
# cd masterfs
# umount -l .
# touch file
# ls
# cd ..
# ls masterfs
________________________________
From: Nir Soffer <nsoffer(a)redhat.com Sent: Apr
17, 2017 8:40 PM
To: Adam Litke; Pavel Gashev
Cc: users
Subject: Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 6:54 PM Adam Litke <alitke@redhat.com<mailto:alitke=
@redhat.com>> wrote:
On Mon, Apr 17, 2017 at 11:04 AM, Pavel Gashev <Pax@acronis.com<mailto:Pax@=
acronis.com>> wrote:
Adam,
You know, Sanlock has recovery mechanism that kills VDSM, or even triggers =
Watchdog to reboot SPM host in case it has lost the SPM lock.
I=92m asking because I had issues with my master storage that caused SPM ho=
st to reboot by Watchdog. And I was sure that it=92s an intended behaviour.=
Isn=92t it?
Yes of course. But an SPM host can fail but still maintain its connection =
to the storage lease. In this case still you need classic fencing.
Something new we are investigating is the use of sanlock's request feature =
which allows a new host to take the lease away from the current holder. Th=
e current holder would be fenced by sanlock (watchdog if necessary) and onl=
y once the lease is free would it be granted to the new requester.
We can use the SPM lease to kill vdsm on the non-responsive SPM host,
and start the SPM on another host, similar to the way we handle vms with
a lease.
But this does not help with the masterfs mounted on the SPM host. if vdsm
is killed before it unmount it, strating the SPM on another host (and mount=
ing
the msasterfs on the new host) will corrupt the masterfs.
When using file based storage (nfs, glusterfs) we don't have a masterfs so
killing vdsm on the SPM should be good enough to start the SPM on another
host, even if fencing is not possible.
We can start with enabling sanlock based SPM fencing on file based storage.
Nir
From: Adam Litke <alitke@redhat.com<mailto:alitke@redhat.com> Date: Monday, 17 April 2017 at 17:32
To: Pavel Gashev <Pax@acronis.com<mailto:Pax@acronis.com> Cc: Nir Soffer
<nsoffer@redhat.com<mailto:nsoffer@redhat.com>>, users <user=
s@ovirt.org<mailto:users@ovirt.org>
Subject: Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev <Pax@acronis.com<mailto:Pax@a=
cronis.com>> wrote:
Nir,
Isn=92t SPM managed via Sanlock? I believe there is no need to fence SPM ho=
st. Especially if there are no SPM tasks running.
It's true that the exclusivity of the SPM role is enforced by Sanlock but y=
ou always need to fence a non-responsive SPM because there is no way to gua=
rantee that the host is not still manipulating storage (eg. LV extensions) =
and we must ensure that only one host has the masterfs on the master storag=
e domain mounted.
From: <users-bounces@ovirt.org<mailto:users-bounces@ovirt.org>> on behalf o=
f Nir Soffer <nsoffer@redhat.com<mailto:nsoffer@redhat.com> Date: Monday, 17 April 2017 at 16:06
To: Konstantin Raskoshnyi <konrasko@gmail.com<mailto:konrasko@gmail.com>>, =
Dan Yasny <dyasny@gmail.com<mailto:dyasny@gmail.com> Cc: users
<users@ovirt.org<mailto:users@ovirt.org>>, FERNANDO FREDIANI <fer=
nando.frediani@upx.com<mailto:fernando.frediani@upx.com> Subject: Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <konrasko(a)gmail.com<m=
ailto:konrasko@gmail.com>> wrote:
But actually, it didn't work well. After main SPM host went down I see this
2017-04-17 05:23:15,554Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.I=
rsProxy] (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] S=
PM Init: could not find reported vds or not up - pool: 'STG' vds_spm_id: '1=
'
2017-04-17 05:23:15,567Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.I=
rsProxy] (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] S=
PM selection - vds seems as spm 'tank5'
2017-04-17 05:23:15,567Z WARN [org.ovirt.engine.core.vdsbroker.irsbroker.I=
rsProxy] (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] s=
pm vds is non responsive, stopping spm selection.
So that means only if BMC is up it's possible to automatically switch SPM =
host?
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
Thanks
On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com<=
mailto:konrasko@gmail.com>> wrote:
Oh, fence agent works fine if I select ilo4,
Thank you for your help!
On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny@gmail.com<mailto:dyasny@g=
mail.com>> wrote:
On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com=
<mailto:konrasko@gmail.com>> wrote:
Makes sense.
I was trying to set it up, but doesn't work with our staging hardware.
We have old ilo100, I'll try again.
Thanks!
It is absolutely necessary for any HA to work properly. There's of course t=
he "confirm host has been shutdown" option, which serves as an override for=
the fence command, but it's manual
On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny@gmail.com<mailto:dyasny@g=
mail.com>> wrote:
On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com=
<mailto:konrasko@gmail.com>> wrote:
Fence agent under each node?
When you configure a host, there's the power management tab, where you need=
to enter the bmc details for the host. If you don't have fencing enabled, =
how do you expect the system to make sure a host running a service is actua=
lly down (and it is safe to start HA services elsewhere), and not, for exam=
ple, just unreachable by the engine? How do you avoid a splitbraid -> SBA ?
On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny@gmail.com<mailto:dyasny@g=
mail.com>> wrote:
On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com=
<mailto:konrasko@gmail.com>> wrote:
"Corner cases"?
I tried to simulate crash of SPM server and ovirt kept trying to reistablis=
hed connection to the failed node.
Did you configure fencing?
On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny@gmail.com<mailto:dyasny@g=
mail.com>> wrote:
On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer@redhat.com<mailto:nsof=
fer(a)redhat.com>> wrote:
On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny@redhat.com<mailto:dyasny@=
redhat.com>> wrote:
On Apr 16, 2017 7:01 AM, "Nir Soffer"
<nsoffer@redhat.com<mailto:nsoffer@re=
dhat.com>> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny@gmail.com<mailto:dyasny@g=
mail.com>> wrote:
When you set up a storage domain, you need to specify a host to perform the=
initial storage operations, but once the SD is defined, it's details are i=
n the engine database, and all the hosts get connected to it directly. If t=
he first host you used to define the SD goes down, all other hosts will sti=
ll remain connected and work. SPM is an HA service, and if the current SPM =
host goes down, SPM gets started on another host in the DC. In short, unles=
s your actual NFS exporting host goes down, there is no outage.
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you conf=
irm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's the=
whole point of HA services
In some cases like power loss or hardware failure, there is no way to start
the spm host, and the system cannot recover automatically.
There are always corner cases, no doubt. But in a normal situation. where a=
n SPM host goes down because of a hardware failure, it gets fenced, other h=
osts contend for SPM and start it. No surprises there.
Nir
Nir
On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com<=
mailto:konrasko@gmail.com>> wrote:
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host to w=
hich I connected nfs storage going down the storage becomes unavailable and=
all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <fernando.frediani(a)upx.c=
om<mailto:fernando.frediani@upx.com>> wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a single host. =
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--
Adam Litke
--
Adam Litke
--_000_3162c49395404987a301406953a54eb4acroniscom_
Content-Type: text/html; charset="Windows-1252"
Content-Transfer-Encoding: quoted-printable
<html <head <meta http-equiv=3D"Content-Type"
content=3D"text/html; charset=3DWindows-1=
252" <meta content=3D"text/html;
charset=3Dutf-8" </head <body <style
type=3D"text/css" <!--
div#d_1492464244266 p
{font-family:Calibri,Arial,Helvetica,sans-serif;
font-size:12.0pt;
color:#1F497D}
-- </style <div
id=3D"d_1492464244266" style=3D"font-family:Calibri,Arial,Helvetica,sa=
ns-serif; font-size:12.0pt; color:#1F497D" <p
dir=3D"ltr">Nir,</p <p
dir=3D"ltr">A process can chdir into mount point and then lazy umount it=
. Filesystem remains mounted while the process exists and current directory=
is on mounted filesystem.</p <p dir=3D"ltr">#
struncate -s 1g masterfs.img<br # mkfs.ext4 masterfs.img<br # mkdir masterfs<br # mount
-o loop masterfs.img masterfs<br # cd
masterfs<br # umount -l .<br # touch file<br #
ls<br # cd ..<br # ls masterfs</p <div
id=3D"signature-x" style=3D"font-family:Calibri,Arial,Helvetica,sans-s=
erif; font-size:12.0pt; color:#1F497D"
<br </div </div <div
id=3D"quoted_header" style=3D"clear:both" <hr
style=3D"border:none; height:1px; color:#E1E1E1; background-color:#E1E1=
E1" <div style=3D"border:none;
padding:3.0pt 0cm 0cm 0cm"><span style=3D"font-s=
ize:11.0pt;
font-family:'Calibri','sans-serif'"><b>From:</b> Nir
Soffer <=
;nsoffer(a)redhat.com&gt;<br <b>Sent:</b> Apr 17,
2017 8:40 PM<br <b>To:</b> Adam Litke;
Pavel Gashev<br <b>Cc:</b>
users<br <b>Subject:</b> Re:
[ovirt-users] storage redundancy in Ovirt<br
</span></div </div <br type=3D"attribution" <div <div dir=3D"ltr" <div class=3D"gmail_quote" <div dir=3D"ltr">On Mon, Apr 17, 2017 at 6:54
PM Adam Litke <<a href=3D"=
mailto:alitke@redhat.com">alitke@redhat.com</a>> wrote:<br </div
<blockquote class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex;
border-left:1=
px #ccc solid; padding-left:1ex" <div
dir=3D"ltr" <div
class=3D"gmail_extra" <div
class=3D"gmail_quote">On Mon, Apr 17, 2017 at 11:04 AM, Pavel Gashev <=
span dir=3D"ltr" <<a
href=3D"mailto:Pax@acronis.com"
target=3D"_blank">Pax(a)acronis.com</a=
<div
bgcolor=3D"white" lang=3D"EN-GB" <div
class=3D"m_1785069678384749853m_-3628883287347250028WordSection1" <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
">Adam,<u></u><u></u></span></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"><u></u> <u></u></span></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
">You know, Sanlock has recovery mechanism that kills VDSM, or even trigger=
s Watchdog to reboot SPM host in case it has lost the SPM
lock.<u></u><u></=
u></span></p <p
class=3D"MsoNormal"><span style=3D"font-size:11.0pt;
font-family:Calibri=
">I=92m asking because I had issues with my master storage that caused SPM =
host to reboot by Watchdog. And I was sure that it=92s an intended behaviou=
r. Isn=92t it?</span></p </div </div
</blockquote <div><br </div
</div </div </div <div
dir=3D"ltr" <div
class=3D"gmail_extra" <div
class=3D"gmail_quote" <div>Yes of course.
But an SPM host can fail but still maintain its c=
onnection to the storage lease. In this case still you need classic f=
encing.<br <br Something
new we are investigating is the use of sanlock's request feature =
which allows a new host to take the lease away from the current holder.&nbs=
p; The current holder would be fenced by sanlock (watchdog if necessary) an=
d only once the lease is free would it
be granted to the new requester. <br
</div </div </div
</div </blockquote <div><br
</div <div>We can use the SPM lease
to kill vdsm on the non-responsive SPM host,<=
/div <div>and start the SPM on another host, similar to the
way we handle vms wi=
th</div <div>a lease.</div <div><br
</div <div>But this does not help
with the masterfs mounted on the SPM host. if v=
dsm</div <div>is killed before it
unmount it, strating the SPM on another host (and =
mounting</div <div>the msasterfs on the new
host) will corrupt the masterfs.</div
<div><br </div <div>When using file based storage (nfs, glusterfs) we
don't have a masterf=
s so</div <div>killing vdsm on the SPM
should be good enough to start the SPM on anot=
her</div <div>host, even if fencing is
not possible.</div <div><br </div
<div>We can start with enabling sanlock based SPM fencing on file based sto=
rage.</div <div><br </div
<div>Nir</div <div> </div <blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 .8ex; border-left:1=
px #ccc solid; padding-left:1ex" <div
dir=3D"ltr" <div
class=3D"gmail_extra" <div
class=3D"gmail_quote" <div></div </div
</div </div <div dir=3D"ltr" <div
class=3D"gmail_extra" <div
class=3D"gmail_quote" <div> </div <blockquote class=3D"gmail_quote"
style=3D"margin:0 0 0 .8ex; border-left:1=
px #ccc solid; padding-left:1ex" <div
bgcolor=3D"white" lang=3D"EN-GB" <div
class=3D"m_1785069678384749853m_-3628883287347250028WordSection1" <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"><u></u><u></u></span></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"><u></u> <u></u></span></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"><u></u> <u></u></span></p <div style=3D"border:none; border-top:solid #b5c4df
1.0pt; padding:3.0pt 0c=
m 0cm 0cm" <p
class=3D"MsoNormal"><b><span style=3D"font-family:Calibri;
color:black">=
From: </span </b><span
style=3D"font-family:Calibri; color:black">Adam Litke <<a href=
=3D"mailto:alitke@redhat.com"
target=3D"_blank">alitke(a)redhat.com</a>&gt;<b=
r <b>Date: </b>Monday, 17 April 2017 at
17:32<br <b>To: </b>Pavel Gashev
<<a href=3D"mailto:Pax@acronis.com" target=3D"_b=
lank">Pax(a)acronis.com</a>&gt;<br
<b>Cc: </b>Nir Soffer <<a
href=3D"mailto:nsoffer@redhat.com" target=3D"_=
blank">nsoffer(a)redhat.com</a>&gt;, users <<a
href=3D"mailto:users@ovirt.=
org"
target=3D"_blank">users(a)ovirt.org</a>&gt;</span></p <div <div
class=3D"m_1785069678384749853h5"><br
<b>Subject: </b>Re: [ovirt-users] storage redundancy in
Ovirt<u></u><u></u>=
</div </div <p></p
</div <div <div
class=3D"m_1785069678384749853h5"
<div <p
class=3D"MsoNormal"><u></u> <u></u></p </div
<div <p
class=3D"MsoNormal"><u></u> <u></u></p <div <p
class=3D"MsoNormal"><u></u> <u></u></p <div <p
class=3D"MsoNormal">On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev
<<a=
href=3D"mailto:Pax@acronis.com"
target=3D"_blank">Pax(a)acronis.com</a>&gt; =
wrote:<u></u><u></u></p
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-right:0cm"
<div <div <p
class=3D"MsoNormal"><span style=3D"font-size:11.0pt;
font-family:Calibri=
">Nir,</span><u></u><u></u></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"> </span><u></u><u></u></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
">Isn=92t SPM managed via Sanlock? I believe there is no need to fence SPM =
host. Especially if there are no SPM tasks
running.</span><u></u><u></u></p=
</div
</div </blockquote <div <p
class=3D"MsoNormal"><u></u> <u></u></p </div
<div <p
class=3D"MsoNormal">It's true that the exclusivity of the SPM role is
en=
forced by Sanlock but you always need to fence a non-responsive SPM because=
there is no way to guarantee that the host is not still manipulating stora=
ge (eg. LV extensions) and we must
ensure that only one host has the masterfs on the master storage domain mo=
unted.<u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-right:0cm"
<div <div <p
class=3D"MsoNormal"><span style=3D"font-size:11.0pt;
font-family:Calibri=
"> </span><u></u><u></u></p <p class=3D"MsoNormal"><span
style=3D"font-size:11.0pt; font-family:Calibri=
"> </span><u></u><u></u></p <div style=3D"border:none; border-top:solid #b5c4df
1.0pt; padding:3.0pt 0c=
m 0cm 0cm" <p
class=3D"MsoNormal"><b><span style=3D"font-family:Calibri;
color:black">=
From: </span </b><span
style=3D"font-family:Calibri; color:black"><<a
href=3D"mailto:=
users-bounces(a)ovirt.org"
target=3D"_blank">users-bounces(a)ovirt.org</a>&gt; =
on behalf of Nir Soffer <<a href=3D"mailto:nsoffer@redhat.com"
target=3D=
"_blank">nsoffer(a)redhat.com</a>&gt;<br <b>Date: </b>Monday, 17 April 2017 at
16:06<br <b>To: </b>Konstantin
Raskoshnyi <<a href=3D"mailto:konrasko@gmail.com" =
target=3D"_blank">konrasko(a)gmail.com</a>&gt;, Dan Yasny
<<a href=3D"mail=
to:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt;<br <b>Cc: </b>users <<a
href=3D"mailto:users@ovirt.org" target=3D"_blank">u=
sers(a)ovirt.org</a>&gt;, FERNANDO FREDIANI <<a
href=3D"mailto:fernando.fr=
ediani(a)upx.com"
target=3D"_blank">fernando.frediani(a)upx.com</a>&gt;<br <b>Subject: </b>Re: [ovirt-users] storage
redundancy in Ovirt</span><u></u>=
<u></u></p </div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <div
<div <p
class=3D"MsoNormal">On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshny=
i <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gmai=
l.com</a>> wrote:<u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">But actually, it didn't work well. After main SPM
ho=
st went down I see this
<u></u><u></u></p
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal"><u></u> <u></u></p <div <div <p
class=3D"MsoNormal">2017-04-17 05:23:15,554Z ERROR [org.ovirt.engine.cor=
e.vdsbroker.irsbroker.IrsProxy] (DefaultQuartzScheduler5) [4dcc033d-26bf-49=
bb-bfaa-03a970dbbec1] SPM Init: could not find reported vds or not up - poo=
l: 'STG' vds_spm_id: '1'<u></u><u></u></p </div
</div </div <div <div
<div <p
class=3D"MsoNormal">2017-04-17 05:23:15,567Z INFO
[org.ovirt.engin=
e.core.vdsbroker.irsbroker.IrsProxy] (DefaultQuartzScheduler5) [4dcc033d-26=
bf-49bb-bfaa-03a970dbbec1] SPM selection - vds seems as spm
'tank5'<u></u><=
u></u></p </div <div <p
class=3D"MsoNormal">2017-04-17 05:23:15,567Z WARN
[org.ovirt.engin=
e.core.vdsbroker.irsbroker.IrsProxy] (DefaultQuartzScheduler5) [4dcc033d-26=
bf-49bb-bfaa-03a970dbbec1] spm vds is non responsive, stopping spm selectio=
n.<u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">So that means only if BMC is up it's possible to
aut=
omatically switch SPM host?<u></u><u></u></p </div
</div </div </div
</blockquote <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">BMC?<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">If your SPM is no responsive, the system will try to=
fence it. Did you <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">configure power management for all hosts? did you ch=
eck that it <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">work? How did you simulate non-responsive
host? =
;<u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">If power management is not configured or fail, the s=
ystem cannot<u></u><u></u></p
</div <div <p
class=3D"MsoNormal">move the spm to another host, unless you manually co=
nfirm that the<u></u><u></u></p
</div <div <p
class=3D"MsoNormal">SPM host was
rebooted.<u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">Nir<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">Thanks<u></u><u></u></p </div
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshn=
yi <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gma=
il.com</a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">Oh, fence agent works fine if I select
ilo4,<u></u><=
u></u></p </div <div <p
class=3D"MsoNormal">Thank you for your
help!<u></u><u></u></p
</div <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt; wr=
ote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskosh=
nyi <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gm=
ail.com</a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">Makes
sense. <u></u><u></u></p
</div <div <p
class=3D"MsoNormal">I was trying to set it up, but doesn't work with
our=
staging hardware. <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">We have old ilo100, I'll try
again. <u></u><u><=
/u></p </div <div <p
class=3D"MsoNormal">Thanks! <u></u><u></u></p <div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </blockquote
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </div
<div <div
<div <div <p
class=3D"MsoNormal">It is absolutely necessary for any HA to work proper=
ly. There's of course the "confirm host has been shutdown" option=
, which serves as an override for the fence command, but it's
manual<u></u>=
<u></u></p </div </div
</div </div <div <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt; wr=
ote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskosh=
nyi <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gm=
ail.com</a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">Fence agent under each
node?<u></u><u></u></p
</div </blockquote <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </div
<div <div
<div <div <p
class=3D"MsoNormal">When you configure a host, there's the power
managem=
ent tab, where you need to enter the bmc details for the host. If you don't=
have fencing enabled, how do you expect the system to make sure a host run=
ning a service is actually down (and
it is safe to start HA services elsewhere), and not, for example, just unr=
eachable by the engine? How do you avoid a splitbraid -> SBA
?<u></u><u>=
</u></p </div </div
</div </div <div <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt; wr=
ote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskosh=
nyi <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gm=
ail.com</a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">"Corner
cases"? <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">I tried to simulate crash of SPM server and ovirt ke=
pt trying to reistablished connection to the failed
node. <u></u><u></=
u></p </div </blockquote
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </div
<div <div
<div <div <p
class=3D"MsoNormal">Did you configure
fencing?<u></u><u></u></p
</div </div </div
</div <div
<div <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt; wr=
ote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <<a
h=
ref=3D"mailto:nsoffer@redhat.com"
target=3D"_blank">nsoffer(a)redhat.com</a>&=
gt; wrote:<u></u><u></u></p
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@redhat.com"
target=3D"_blank">dyasny(a)redhat.com</a>&gt; =
wrote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <p
class=3D"MsoNormal">On Apr 16, 2017 7:01 AM, "Nir Soffer"
<=
<a href=3D"mailto:nsoffer@redhat.com"
target=3D"_blank">nsoffer(a)redhat.com<=
/a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div
<div <p
class=3D"MsoNormal">On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <<a
hre=
f=3D"mailto:dyasny@gmail.com"
target=3D"_blank">dyasny(a)gmail.com</a>&gt; wr=
ote:<u></u><u></u></p
</div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">When you set up a storage domain, you need to specif=
y a host to perform the initial storage operations, but once the SD is defi=
ned, it's details are in the engine database, and all the hosts get connect=
ed to it directly. If the first host
you used to define the SD goes down, all other hosts will still remain con=
nected and work. SPM is an HA service, and if the current SPM host goes dow=
n, SPM gets started on another host in the DC. In short, unless your actual=
NFS exporting host goes down, there
is no outage. <u></u><u></u></p </div
</blockquote <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div <div <p
class=3D"MsoNormal">There is no storage outage, but if you shutdown the =
spm host, the spm host<u></u><u></u></p </div
<div <p
class=3D"MsoNormal">will not move to a new host until the spm host is on=
line again, or you confirm<u></u><u></u></p </div
<div <p
class=3D"MsoNormal">manually that the spm host was
rebooted.<u></u><u></=
u></p </div </div
</div </blockquote </div
</div </div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div <div
<div <p
class=3D"MsoNormal">In a properly configured setup the SBA should take c=
are of that. That's the whole point of HA
services <u></u><u></u></p </div
</div </blockquote <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">In some cases like power loss or hardware failure, t=
here is no way to start<u></u><u></u></p </div
<div <p
class=3D"MsoNormal">the spm host, and the system cannot recover automati=
cally.<u></u><u></u></p
</div </div </div
</blockquote <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </div
<div <div
<div <div <p
class=3D"MsoNormal">There are always corner cases, no doubt. But in a no=
rmal situation. where an SPM host goes down because of a hardware failure, =
it gets fenced, other hosts contend for SPM and start it. No surprises ther=
e. <u></u><u></u></p
</div </div </div
</div <div
<div <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal"><span
style=3D"color:#888888"> </span><u></u><u=
></u></p </div <div <p
class=3D"MsoNormal"><span
style=3D"color:#888888">Nir</span><u></u><u></=
u></p </div <div <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <div
<div <blockquote
style=3D"border:none; border-left:solid #cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal"><span
style=3D"color:#888888"> </span><u></u><u=
></u></p </div <div <p
class=3D"MsoNormal"><span
style=3D"color:#888888">Nir</span><u></u><u></=
u></p </div <div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <p
class=3D"MsoNormal">On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshn=
yi <<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko@gma=
il.com</a>> wrote:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">Hi
Fernando,<u></u><u></u></p
</div <div <p
class=3D"MsoNormal">I see each host has direct connection nfs mount, but=
yes, if main host to which I connected nfs storage going down the storage =
becomes unavailable and all vms are down<u></u><u></u></p </div
<div <div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <div <p
class=3D"MsoNormal">On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI
&=
lt;<a href=3D"mailto:fernando.frediani@upx.com"
target=3D"_blank">fernando.=
frediani(a)upx.com</a>&gt; wrote:<u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div
<div <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt">Hello Konstantin.<u><=
/u><u></u></p </div <p class=3D"MsoNormal">That doesn`t make much
sense make a whole cluster de=
pend on a single host. From what I know any host talk directly to NFS Stora=
ge Array or whatever other Shared Storage you
have.<u></u><u></u></p
</div <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt">Have you tested that =
host going down if that affects the other with the NFS mounted directlly in=
a NFS Storage array ?<u></u><u></u></p </div
</div <div <p
class=3D"MsoNormal">Fernando<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <p
class=3D"MsoNormal">2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<=
;<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko(a)gmail.com=
</a>>:<u></u><u></u></p <blockquote style=3D"border:none; border-left:solid
#cccccc 1.0pt; padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">In ovirt you have to attach storage through specific=
host. <u></u><u></u></p
</div <div <p
class=3D"MsoNormal">If host goes down storage is not
available. <u>=
</u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p <div <div <p
class=3D"MsoNormal">On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI
&l=
t;<a href=3D"mailto:fernando.frediani@upx.com"
target=3D"_blank">fernando.f=
rediani(a)upx.com</a>&gt; wrote:<u></u><u></u></p </div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div
<div <div <p
class=3D"MsoNormal">Well, make it not go through host1 and dedicate a st=
orage server for running NFS and make both hosts connect to
it.<u></u><u></=
u></p </div <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt">In my view NFS is muc=
h easier to manage than any other type of storage, specially FC and iSCSI a=
nd performance is pretty much the same, so you won`t get better results oth=
er than management going to other type.<u></u><u></u></p </div <p
class=3D"MsoNormal">Fernando<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <div <p
class=3D"MsoNormal">2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<=
<a href=3D"mailto:konrasko@gmail.com"
target=3D"_blank">konrasko(a)gmail.com<=
/a>>:<u></u><u></u></p
</div </div <div <div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <div <p
class=3D"MsoNormal">Hi guys, <u></u><u></u></p <div <p
class=3D"MsoNormal">I have one
nfs storage,<u></u><u></u></p </div
<div <p
class=3D"MsoNormal">it's connected through
host1.<u></u><u></u></p
</div <div <p
class=3D"MsoNormal">host2 also has access to it, I can easily migrate vm=
s between them.<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">The question is - if host1 is down - all infrastruct=
ure is down, since all traffic goes through
host1,<u></u><u></u></p
</div <div <p
class=3D"MsoNormal">is there any way in oVirt to use redundant storage?<=
u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">Only
glusterfs?<u></u><u></u></p
</div <div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
<div <p
class=3D"MsoNormal">Thanks<u></u><u></u></p </div
<div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div <p
class=3D"MsoNormal"> <u></u><u></u></p </blockquote
</div </div <div <div
<blockquote style=3D"border:none; border-left:solid #cccccc 1.0pt;
padding:=
0cm 0cm 0cm 6.0pt; margin-left:4.8pt; margin-top:5.0pt; margin-right:0cm; m=
argin-bottom:5.0pt" <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt">_____________________=
__________________________<br Users mailing list<br <a href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br=
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
target=3D"_blank"=
>http://lists.ovirt.org/mailman/listinfo/users</a><u></u><u></u></p </blockquote
</div </div </blockquote
</div </div </blockquote
</div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</blockquote </div </div
</div </div <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt"><br
_______________________________________________<br Users
mailing list<br <a
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br=
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
target=3D"_blank"=
>http://lists.ovirt.org/mailman/listinfo/users</a><u></u><u></u></p </blockquote
</div <p
class=3D"MsoNormal"> <u></u><u></u></p </div <p
class=3D"MsoNormal">_______________________________________________<br Users mailing list<br <a
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br=
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
target=3D"_blank"=
>http://lists.ovirt.org/mailman/listinfo/users</a><u></u><u></u></p </blockquote
</div </div </div <p
class=3D"MsoNormal" style=3D"margin-bottom:12.0pt"><br _______________________________________________<br Users mailing list<br <a
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br=
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
target=3D"_blank"=
>http://lists.ovirt.org/mailman/listinfo/users</a><u></u><u></u></p </blockquote
</div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</div </div </blockquote
</div </div </div
</div </blockquote </div
</div </div </blockquote
</div </div </div
</div </blockquote </div
</div </div </blockquote
</div </div </div
</div </blockquote </div
</div </div </blockquote
</div </div </div
</div </blockquote </div
</div </div </blockquote
</div </div </div
</div </blockquote </div <p
class=3D"MsoNormal"> <u></u><u></u></p </div
</blockquote </div </div
</div </div </div
</div <p class=3D"MsoNormal"
style=3D"margin-bottom:12.0pt"><br
_______________________________________________<br Users
mailing list<br <a
href=3D"mailto:Users@ovirt.org"
target=3D"_blank">Users(a)ovirt.org</a><br=
<a
href=3D"http://lists.ovirt.org/mailman/listinfo/users"
target=3D"_blank"=
>http://lists.ovirt.org/mailman/listinfo/users</a><u></u><u></u></p </blockquote
</div <p
class=3D"MsoNormal"><br <br
clear=3D"all" <br --
<u></u><u></u></p
<div <div <p
class=3D"MsoNormal">Adam
Litke<u></u><u></u></p
</div </div </div
</div </div </div
</div </div </blockquote
</div </div </div <div
dir=3D"ltr" <div
class=3D"gmail_extra"><br <br
clear=3D"all" <br --
<br <div
class=3D"m_1785069678384749853gmail_signature" <div
dir=3D"ltr">Adam Litke</div
</div </div </div
</blockquote </div </div
</div </body </html
--_000_3162c49395404987a301406953a54eb4acroniscom_--
From what I know any host talk directly to NFS Storage Array or
whatever ot=
her Shared Storage you have.
Have you tested that host going down if that affects the other with the NFS=
mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com<mailto=
:konrasko@gmail.com>>:
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <fernando.frediani(a)upx.co=
m<mailto:fernando.frediani@upx.com>> wrote:
Well, make it not go through host1 and dedicate a storage server for runnin=
g NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other type of storage, spe=
cially FC and iSCSI and performance is pretty much the same, so you won`t g=
et better results other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko@gmail.com<mailto:=
konrasko(a)gmail.com>>:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate vms between them.
The question is - if host1 is down - all infrastructure is down, since all =
traffic goes through host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users@ovirt.org<mailto:Users@ovirt.org></span> wrote:<br>
<blockquote
class=3D"gmail_quote" style=3D"margin:0 0 0 .8ex; border-left:1=
px #ccc solid; padding-left:1ex"
8:19 p.m.
On Tue, Apr 18, 2017 at 12:23 AM Pavel Gashev <Pax(a)acronis.com> wrote:
Nir,
A process can chdir into mount point and then lazy umount it. Filesystem
remains mounted while the process exists and current directory is on
mounted filesystem.
# struncate -s 1g masterfs.img
# mkfs.ext4 masterfs.img
# mkdir masterfs
# mount -o loop masterfs.img masterfs
# cd masterfs
# umount -l .
# touch file
# ls
# cd ..
# ls masterfs
Interesting idea!
The only issue I see if not having a way to tell if the file system was
actually
unmounted. Does process termination guarantee that the file system was
unmounted?
Do you know if the behaviour is documented somewhere?
Nir
------------------------------
*From:* Nir Soffer <nsoffer(a)redhat.com>
*Sent:* Apr 17, 2017 8:40 PM
*To:* Adam Litke; Pavel Gashev
*Cc:* users
*Subject:* Re: [ovirt-users] storage redundancy in Ovirt
On Mon, Apr 17, 2017 at 6:54 PM Adam Litke <alitke(a)redhat.com> wrote:
> On Mon, Apr 17, 2017 at 11:04 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
>
>> Adam,
>>
>>
>>
>> You know, Sanlock has recovery mechanism that kills VDSM, or even
>> triggers Watchdog to reboot SPM host in case it has lost the SPM lock.
>>
>> I’m asking because I had issues with my master storage that caused SPM
>> host to reboot by Watchdog. And I was sure that it’s an intended behaviour.
>> Isn’t it?
>>
>
> Yes of course. But an SPM host can fail but still maintain its
> connection to the storage lease. In this case still you need classic
> fencing.
>
> Something new we are investigating is the use of sanlock's request
> feature which allows a new host to take the lease away from the current
> holder. The current holder would be fenced by sanlock (watchdog if
> necessary) and only once the lease is free would it be granted to the new
> requester.
>
We can use the SPM lease to kill vdsm on the non-responsive SPM host,
and start the SPM on another host, similar to the way we handle vms with
a lease.
But this does not help with the masterfs mounted on the SPM host. if vdsm
is killed before it unmount it, strating the SPM on another host (and
mounting
the msasterfs on the new host) will corrupt the masterfs.
When using file based storage (nfs, glusterfs) we don't have a masterfs so
killing vdsm on the SPM should be good enough to start the SPM on another
host, even if fencing is not possible.
We can start with enabling sanlock based SPM fencing on file based storage.
Nir
>
>
>>
>>
>>
>>
>> *From: *Adam Litke <alitke(a)redhat.com>
>> *Date: *Monday, 17 April 2017 at 17:32
>> *To: *Pavel Gashev <Pax(a)acronis.com>
>> *Cc: *Nir Soffer <nsoffer(a)redhat.com>, users <users(a)ovirt.org>
>>
>> *Subject: *Re: [ovirt-users] storage redundancy in Ovirt
>>
>>
>>
>>
>>
>>
>>
>> On Mon, Apr 17, 2017 at 9:26 AM, Pavel Gashev <Pax(a)acronis.com> wrote:
>>
>> Nir,
>>
>>
>>
>> Isn’t SPM managed via Sanlock? I believe there is no need to fence SPM
>> host. Especially if there are no SPM tasks running.
>>
>>
>>
>> It's true that the exclusivity of the SPM role is enforced by Sanlock
>> but you always need to fence a non-responsive SPM because there is no way
>> to guarantee that the host is not still manipulating storage (eg. LV
>> extensions) and we must ensure that only one host has the masterfs on the
>> master storage domain mounted.
>>
>>
>>
>>
>>
>>
>>
>> *From: *<users-bounces(a)ovirt.org> on behalf of Nir Soffer <
>> nsoffer(a)redhat.com>
>> *Date: *Monday, 17 April 2017 at 16:06
>> *To: *Konstantin Raskoshnyi <konrasko(a)gmail.com>, Dan Yasny <
>> dyasny(a)gmail.com>
>> *Cc: *users <users(a)ovirt.org>, FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com>
>> *Subject: *Re: [ovirt-users] storage redundancy in Ovirt
>>
>>
>>
>> On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> But actually, it didn't work well. After main SPM host went down I see
>> this
>>
>>
>>
>>
>>
>> 2017-04-17 05:23:15,554Z ERROR
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
>> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init:
>> could not find reported vds or not up - pool: 'STG' vds_spm_id:
'1'
>>
>> 2017-04-17 05:23:15,567Z INFO
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
>> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
>> selection - vds seems as spm 'tank5'
>>
>> 2017-04-17 05:23:15,567Z WARN
>> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
>> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is
>> non responsive, stopping spm selection.
>>
>>
>>
>> So that means only if BMC is up it's possible to automatically switch
>> SPM host?
>>
>>
>>
>> BMC?
>>
>>
>>
>> If your SPM is no responsive, the system will try to fence it. Did you
>>
>> configure power management for all hosts? did you check that it
>>
>> work? How did you simulate non-responsive host?
>>
>>
>>
>> If power management is not configured or fail, the system cannot
>>
>> move the spm to another host, unless you manually confirm that the
>>
>> SPM host was rebooted.
>>
>>
>>
>> Nir
>>
>>
>>
>>
>>
>> Thanks
>>
>>
>>
>> On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> Oh, fence agent works fine if I select ilo4,
>>
>> Thank you for your help!
>>
>>
>>
>> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> Makes sense.
>>
>> I was trying to set it up, but doesn't work with our staging hardware.
>>
>> We have old ilo100, I'll try again.
>>
>> Thanks!
>>
>>
>>
>>
>>
>> It is absolutely necessary for any HA to work properly. There's of
>> course the "confirm host has been shutdown" option, which serves as an
>> override for the fence command, but it's manual
>>
>>
>>
>> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> Fence agent under each node?
>>
>>
>>
>> When you configure a host, there's the power management tab, where you
>> need to enter the bmc details for the host. If you don't have fencing
>> enabled, how do you expect the system to make sure a host running a service
>> is actually down (and it is safe to start HA services elsewhere), and not,
>> for example, just unreachable by the engine? How do you avoid a splitbraid
>> -> SBA ?
>>
>>
>>
>>
>>
>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> "Corner cases"?
>>
>> I tried to simulate crash of SPM server and ovirt kept trying to
>> reistablished connection to the failed node.
>>
>>
>>
>> Did you configure fencing?
>>
>>
>>
>>
>>
>>
>>
>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer <nsoffer(a)redhat.com> wrote:
>>
>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny <dyasny(a)redhat.com> wrote:
>>
>>
>>
>>
>>
>> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com>
wrote:
>>
>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>> When you set up a storage domain, you need to specify a host to perform
>> the initial storage operations, but once the SD is defined, it's details
>> are in the engine database, and all the hosts get connected to it directly.
>> If the first host you used to define the SD goes down, all other hosts will
>> still remain connected and work. SPM is an HA service, and if the current
>> SPM host goes down, SPM gets started on another host in the DC. In short,
>> unless your actual NFS exporting host goes down, there is no outage.
>>
>>
>>
>> There is no storage outage, but if you shutdown the spm host, the spm
>> host
>>
>> will not move to a new host until the spm host is online again, or you
>> confirm
>>
>> manually that the spm host was rebooted.
>>
>>
>>
>> In a properly configured setup the SBA should take care of that. That's
>> the whole point of HA services
>>
>>
>>
>> In some cases like power loss or hardware failure, there is no way to
>> start
>>
>> the spm host, and the system cannot recover automatically.
>>
>>
>>
>> There are always corner cases, no doubt. But in a normal situation.
>> where an SPM host goes down because of a hardware failure, it gets fenced,
>> other hosts contend for SPM and start it. No surprises there.
>>
>>
>>
>>
>>
>> Nir
>>
>>
>>
>>
>>
>>
>>
>> Nir
>>
>>
>>
>>
>>
>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>> Hi Fernando,
>>
>> I see each host has direct connection nfs mount, but yes, if main host
>> to which I connected nfs storage going down the storage becomes unavailable
>> and all vms are down
>>
>>
>>
>>
>>
>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com> wrote:
>>
>> Hello Konstantin.
>>
>> That doesn`t make much sense make a whole cluster depend on a single
>> host. From what I know any host talk directly to NFS Storage Array or
>> whatever other Shared Storage you have.
>>
>> Have you tested that host going down if that affects the other with the
>> NFS mounted directlly in a NFS Storage array ?
>>
>> Fernando
>>
>>
>>
>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>> In ovirt you have to attach storage through specific host.
>>
>> If host goes down storage is not available.
>>
>>
>>
>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com> wrote:
>>
>> Well, make it not go through host1 and dedicate a storage server for
>> running NFS and make both hosts connect to it.
>>
>> In my view NFS is much easier to manage than any other type of storage,
>> specially FC and iSCSI and performance is pretty much the same, so you
>> won`t get better results other than management going to other type.
>>
>> Fernando
>>
>>
>>
>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>
>> Hi guys,
>>
>> I have one nfs storage,
>>
>> it's connected through host1.
>>
>> host2 also has access to it, I can easily migrate vms between them.
>>
>>
>>
>> The question is - if host1 is down - all infrastructure is down, since
>> all traffic goes through host1,
>>
>> is there any way in oVirt to use redundant storage?
>>
>>
>>
>> Only glusterfs?
>>
>>
>>
>> Thanks
>>
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>>
>>
>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>>
>>
>> --
>>
>> Adam Litke
>>
>
>
>
> --
> Adam Litke
>
9:34 p.m.
--_000_14925401941105315camelacroniscom_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
T24gVHVlLCAyMDE3LTA0LTE4IGF0IDE3OjE5ICswMDAwLCBOaXIgU29mZmVyIHdyb3RlOg0KT24g
VHVlLCBBcHIgMTgsIDIwMTcgYXQgMTI6MjMgQU0gUGF2ZWwgR2FzaGV2IDxQYXhAYWNyb25pcy5j
b208bWFpbHRvOlBheEBhY3JvbmlzLmNvbT4+IHdyb3RlOg0KDQpOaXIsDQoNCkEgcHJvY2VzcyBj
YW4gY2hkaXIgaW50byBtb3VudCBwb2ludCBhbmQgdGhlbiBsYXp5IHVtb3VudCBpdC4gRmlsZXN5
c3RlbSByZW1haW5zIG1vdW50ZWQgd2hpbGUgdGhlIHByb2Nlc3MgZXhpc3RzIGFuZCBjdXJyZW50
IGRpcmVjdG9yeSBpcyBvbiBtb3VudGVkIGZpbGVzeXN0ZW0uDQoNCiMgc3RydW5jYXRlIC1zIDFn
IG1hc3RlcmZzLmltZw0KIyBta2ZzLmV4dDQgbWFzdGVyZnMuaW1nDQojIG1rZGlyIG1hc3RlcmZz
DQojIG1vdW50IC1vIGxvb3AgbWFzdGVyZnMuaW1nIG1hc3RlcmZzDQojIGNkIG1hc3RlcmZzDQoj
IHVtb3VudCAtbCAuDQojIHRvdWNoIGZpbGUNCiMgbHMNCiMgY2QgLi4NCiMgbHMgbWFzdGVyZnMN
Cg0KSW50ZXJlc3RpbmcgaWRlYSENCg0KVGhlIG9ubHkgaXNzdWUgSSBzZWUgaWYgbm90IGhhdmlu
ZyBhIHdheSB0byB0ZWxsIGlmIHRoZSBmaWxlIHN5c3RlbSB3YXMgYWN0dWFsbHkNCnVubW91bnRl
ZC4gRG9lcyBwcm9jZXNzIHRlcm1pbmF0aW9uIGd1YXJhbnRlZSB0aGF0IHRoZSBmaWxlIHN5c3Rl
bSB3YXMNCnVubW91bnRlZD8NCg0KWW91IGNhbiBkb3VibGUgY2hlY2sgdGhhdCBieSBkZWFjdGl2
YXRpbmcgdW5kZXJseWluZyBMVi4gSXQgc2hvdWxkIG5vdCBiZSBidXN5Lg0KDQpBcyBJIHVuZGVy
c3RhbmQgaXQgdW5tb3VudHMgRlMgYXMgc29vbiBhcyBhbGwgaW5vZGVzIGFyZSBjbG9zZWQuIFNp
bmNlIGtpbGxpbmcgb2YgcHJvY2VzcyBpcyBjbG9zaW5nIGFsbCBvcGVuIGlub2RlcywgeW91IGNh
biBiZSBzdXJlIHRoYXQgZmlsZSBzeXN0ZW0gaXMgdW5tb3VudGVkIGlmIHRoZXJlIGFyZSBubyBv
dGhlciBwcm9jZXNzZXMuIE9mIGNvdXJzZSB0aGlzIGlzIGEgcGxhY2UgZm9yIHJhY2UgY29uZGl0
aW9ucy4gRm9yIGV4YW1wbGUsIHVubW91bnRpbmcgY2FuIGJlIHN0dWNrIHNvbWUgdGltZSBpZiB1
bmRlcmx5aW5nIHN0b3JhZ2UgaXMgdW5yZXNwb25zaXZlLiBNdWx0aXBhdGggc2hvdWxkIGRyb3Ag
YWxsIElPcyB3aXRoIEVJTy4gVkRTTSBoYXMgdG8gd2FpdCBlbm91Z2ggdGltZSBhZnRlciBhY3F1
aXJpbmcgdGhlIGxvY2sgYmVmb3JlIG1vdW50aW5nIG1hc3RlcmZzLg0KDQpEbyB5b3Uga25vdyBp
ZiB0aGUgYmVoYXZpb3VyIGlzIGRvY3VtZW50ZWQgc29tZXdoZXJlPw0KDQp1bW91bnQoOCk6DQoN
Ci1sLCAtLWxhenkNCkxhenkgdW5tb3VudC4gIERldGFjaCB0aGUgZmlsZXN5c3RlbSBmcm9tIHRo
ZSBmaWxlc3lzdGVtIGhpZXJhcmNoeSBub3csIGFuZCBjbGVhbnVwIGFsbCByZWZlcmVuY2VzIHRv
IHRoZSBmaWxlc3lzdGVtIGFzIHNvb24gYXMgaXQgaXMgbm90IGJ1c3kgYW55bW9yZS4gIChSZXF1
aXJlcyBrZXJuZWwgMi40LjExIG9yIGxhdGVyLikNCg0KDQo=
--_000_14925401941105315camelacroniscom_
Content-Type: text/html; charset="utf-8"
Content-ID: <4AC96B6347714B4BB1DF97D7D6BB2F00(a)acronis.com>
Content-Transfer-Encoding: base64
PGh0bWw+DQo8aGVhZD4NCjxtZXRhIGh0dHAtZXF1aXY9IkNvbnRlbnQtVHlwZSIgY29udGVudD0i
dGV4dC9odG1sOyBjaGFyc2V0PXV0Zi04Ij4NCjwvaGVhZD4NCjxib2R5Pg0KPGRpdj5PbiBUdWUs
IDIwMTctMDQtMTggYXQgMTc6MTkgJiM0MzswMDAwLCBOaXIgU29mZmVyIHdyb3RlOjwvZGl2Pg0K
PGJsb2NrcXVvdGUgdHlwZT0iY2l0ZSI+DQo8ZGl2IGRpcj0ibHRyIj4NCjxkaXYgY2xhc3M9Imdt
YWlsX3F1b3RlIj4NCjxkaXYgZGlyPSJsdHIiPk9uIFR1ZSwgQXByIDE4LCAyMDE3IGF0IDEyOjIz
IEFNIFBhdmVsIEdhc2hldiAmbHQ7PGEgaHJlZj0ibWFpbHRvOlBheEBhY3JvbmlzLmNvbSI+UGF4
QGFjcm9uaXMuY29tPC9hPiZndDsgd3JvdGU6PGJyPg0KPC9kaXY+DQo8YmxvY2txdW90ZSB0eXBl
PSJjaXRlIj4NCjxkaXY+DQo8ZGl2IGlkPSJtXy01MjU5ODQwODU1NTYxOTUzNzU2ZF8xNDkyNDY0
MjQ0MjY2IiBzdHlsZT0iZm9udC1mYW1pbHk6Q2FsaWJyaSxBcmlhbCxIZWx2ZXRpY2Esc2Fucy1z
ZXJpZjtmb250LXNpemU6MTIuMHB0O2NvbG9yOiMxZjQ5N2QiPg0KPHAgZGlyPSJsdHIiPk5pciw8
L3A+DQo8cCBkaXI9Imx0ciI+QSBwcm9jZXNzIGNhbiBjaGRpciBpbnRvIG1vdW50IHBvaW50IGFu
ZCB0aGVuIGxhenkgdW1vdW50IGl0LiBGaWxlc3lzdGVtIHJlbWFpbnMgbW91bnRlZCB3aGlsZSB0
aGUgcHJvY2VzcyBleGlzdHMgYW5kIGN1cnJlbnQgZGlyZWN0b3J5IGlzIG9uIG1vdW50ZWQgZmls
ZXN5c3RlbS48L3A+DQo8cCBkaXI9Imx0ciI+IyBzdHJ1bmNhdGUgLXMgMWcgbWFzdGVyZnMuaW1n
PGJyPg0KIyBta2ZzLmV4dDQgbWFzdGVyZnMuaW1nPGJyPg0KIyBta2RpciBtYXN0ZXJmczxicj4N
CiMgbW91bnQgLW8gbG9vcCBtYXN0ZXJmcy5pbWcgbWFzdGVyZnM8YnI+DQojIGNkIG1hc3RlcmZz
PGJyPg0KIyB1bW91bnQgLWwgLjxicj4NCiMgdG91Y2ggZmlsZTxicj4NCiMgbHM8YnI+DQojIGNk
IC4uPGJyPg0KIyBscyBtYXN0ZXJmczwvcD4NCjwvZGl2Pg0KPC9kaXY+DQo8YnI+DQo8L2Jsb2Nr
cXVvdGU+DQo8ZGl2PkludGVyZXN0aW5nIGlkZWEhPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0K
PGRpdj5UaGUgb25seSBpc3N1ZSBJIHNlZSBpZiBub3QgaGF2aW5nIGEgd2F5IHRvIHRlbGwgaWYg
dGhlIGZpbGUgc3lzdGVtIHdhcyBhY3R1YWxseTwvZGl2Pg0KPGRpdj51bm1vdW50ZWQuIERvZXMg
cHJvY2VzcyB0ZXJtaW5hdGlvbiBndWFyYW50ZWUgdGhhdCB0aGUgZmlsZSBzeXN0ZW0gd2FzPC9k
aXY+DQo8ZGl2PnVubW91bnRlZD8mbmJzcDs8L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2Nr
cXVvdGU+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5Zb3UgY2FuIGRvdWJsZSBjaGVjayB0aGF0
IGJ5IGRlYWN0aXZhdGluZyB1bmRlcmx5aW5nIExWLiBJdCBzaG91bGQgbm90IGJlIGJ1c3kuPC9k
aXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj5BcyBJIHVuZGVyc3RhbmQgaXQgdW5tb3VudHMg
RlMgYXMgc29vbiBhcyBhbGwgaW5vZGVzIGFyZSBjbG9zZWQuIFNpbmNlIGtpbGxpbmcgb2YgcHJv
Y2VzcyBpcyBjbG9zaW5nIGFsbCBvcGVuIGlub2RlcywgeW91IGNhbiBiZSBzdXJlIHRoYXQgZmls
ZSBzeXN0ZW0gaXMgdW5tb3VudGVkIGlmIHRoZXJlIGFyZSBubyBvdGhlciBwcm9jZXNzZXMuIE9m
IGNvdXJzZSB0aGlzIGlzIGEgcGxhY2UgZm9yIHJhY2UgY29uZGl0aW9ucy4gRm9yIGV4YW1wbGUs
DQogdW5tb3VudGluZyBjYW4gYmUgc3R1Y2sgc29tZSB0aW1lIGlmIHVuZGVybHlpbmcgc3RvcmFn
ZSBpcyB1bnJlc3BvbnNpdmUuIE11bHRpcGF0aCBzaG91bGQgZHJvcCBhbGwgSU9zIHdpdGggRUlP
LiBWRFNNIGhhcyB0byB3YWl0IGVub3VnaCB0aW1lIGFmdGVyIGFjcXVpcmluZyB0aGUgbG9jayBi
ZWZvcmUgbW91bnRpbmcgbWFzdGVyZnMuPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGJsb2Nr
cXVvdGUgdHlwZT0iY2l0ZSI+DQo8ZGl2IGRpcj0ibHRyIj4NCjxkaXYgY2xhc3M9ImdtYWlsX3F1
b3RlIj4NCjxkaXY+RG8geW91IGtub3cgaWYgdGhlIGJlaGF2aW91ciBpcyBkb2N1bWVudGVkIHNv
bWV3aGVyZT88L2Rpdj4NCjwvZGl2Pg0KPC9kaXY+DQo8L2Jsb2NrcXVvdGU+DQo8ZGl2Pjxicj4N
CjwvZGl2Pg0KPGRpdj51bW91bnQoOCk6PC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2Pg0KPGRpdj4t
bCwgLS1sYXp5PC9kaXY+DQo8ZGl2PkxhenkgdW5tb3VudC4mbmJzcDsmbmJzcDtEZXRhY2ggdGhl
IGZpbGVzeXN0ZW0gZnJvbSB0aGUgZmlsZXN5c3RlbSBoaWVyYXJjaHkgbm93LCBhbmQgY2xlYW51
cCBhbGwgcmVmZXJlbmNlcyB0byB0aGUgZmlsZXN5c3RlbSBhcyBzb29uIGFzIGl0IGlzIG5vdCBi
dXN5IGFueW1vcmUuJm5ic3A7Jm5ic3A7KFJlcXVpcmVzIGtlcm5lbCAyLjQuMTEgb3IgbGF0ZXIu
KTwvZGl2Pg0KPGRpdj48L2Rpdj4NCjxkaXY+PGJyPg0KPC9kaXY+DQo8ZGl2Pjxicj4NCjwvZGl2
Pg0KPGRpdj4NCjxkaXYgZGlyPSJsdHIiPg0KPGRpdiBjbGFzcz0iZ21haWxfcXVvdGUiPjwvZGl2
Pg0KPC9kaXY+DQo8L2Rpdj4NCjwvYm9keT4NCjwvaHRtbD4NCg==
--_000_14925401941105315camelacroniscom_--
5:26 p.m.
Hi Nir,
BMC - board management controller, in my case I have ilo.
Yes I set up power management for all hosts - ovirt sees ilo status as ok.
I use remote pdu to shutdown the port, after that happens the picture I
attached.
After I switch power port on, ovirt is able to read ilo status, sees that
Linux is down and immediately switches the spm server.
On Mon, Apr 17, 2017 at 6:07 AM Nir Soffer <nsoffer(a)redhat.com> wrote:
On Mon, Apr 17, 2017 at 8:24 AM Konstantin Raskoshnyi
<konrasko(a)gmail.com>
wrote:
> But actually, it didn't work well. After main SPM host went down I see
> this
>
[image: Screen Shot 2017-04-16 at 10.22.00 PM.png]
>
> 2017-04-17 05:23:15,554Z ERROR
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM Init:
> could not find reported vds or not up - pool: 'STG' vds_spm_id: '1'
> 2017-04-17 05:23:15,567Z INFO
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] SPM
> selection - vds seems as spm 'tank5'
> 2017-04-17 05:23:15,567Z WARN
> [org.ovirt.engine.core.vdsbroker.irsbroker.IrsProxy]
> (DefaultQuartzScheduler5) [4dcc033d-26bf-49bb-bfaa-03a970dbbec1] spm vds is
> non responsive, stopping spm selection.
>
> So that means only if BMC is up it's possible to automatically switch
> SPM host?
>
BMC?
If your SPM is no responsive, the system will try to fence it. Did you
configure power management for all hosts? did you check that it
work? How did you simulate non-responsive host?
If power management is not configured or fail, the system cannot
move the spm to another host, unless you manually confirm that the
SPM host was rebooted.
Nir
>
> Thanks
>
> On Sun, Apr 16, 2017 at 8:29 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Oh, fence agent works fine if I select ilo4,
>> Thank you for your help!
>>
>> On Sun, Apr 16, 2017 at 8:22 PM Dan Yasny <dyasny(a)gmail.com> wrote:
>>
>>> On Sun, Apr 16, 2017 at 11:19 PM, Konstantin Raskoshnyi <
>>> konrasko(a)gmail.com> wrote:
>>>
>>>> Makes sense.
>>>> I was trying to set it up, but doesn't work with our staging
hardware.
>>>> We have old ilo100, I'll try again.
>>>> Thanks!
>>>>
>>>>
>>> It is absolutely necessary for any HA to work properly. There's of
>>> course the "confirm host has been shutdown" option, which serves as
an
>>> override for the fence command, but it's manual
>>>
>>>
>>>> On Sun, Apr 16, 2017 at 8:18 PM Dan Yasny <dyasny(a)gmail.com>
wrote:
>>>>
>>>>> On Sun, Apr 16, 2017 at 11:15 PM, Konstantin Raskoshnyi <
>>>>> konrasko(a)gmail.com> wrote:
>>>>>
>>>>>> Fence agent under each node?
>>>>>>
>>>>>
>>>>> When you configure a host, there's the power management tab,
where
>>>>> you need to enter the bmc details for the host. If you don't have
fencing
>>>>> enabled, how do you expect the system to make sure a host running a
service
>>>>> is actually down (and it is safe to start HA services elsewhere), and
not,
>>>>> for example, just unreachable by the engine? How do you avoid a
splitbraid
>>>>> -> SBA ?
>>>>>
>>>>>
>>>>>>
>>>>>> On Sun, Apr 16, 2017 at 8:14 PM Dan Yasny
<dyasny(a)gmail.com> wrote:
>>>>>>
>>>>>>> On Sun, Apr 16, 2017 at 11:13 PM, Konstantin Raskoshnyi <
>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>
>>>>>>>> "Corner cases"?
>>>>>>>> I tried to simulate crash of SPM server and ovirt kept
trying to
>>>>>>>> reistablished connection to the failed node.
>>>>>>>>
>>>>>>>
>>>>>>> Did you configure fencing?
>>>>>>>
>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On Sun, Apr 16, 2017 at 8:10 PM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>> wrote:
>>>>>>>>
>>>>>>>>> On Sun, Apr 16, 2017 at 7:29 AM, Nir Soffer
<nsoffer(a)redhat.com>
>>>>>>>>> wrote:
>>>>>>>>>
>>>>>>>>>> On Sun, Apr 16, 2017 at 2:05 PM Dan Yasny
<dyasny(a)redhat.com>
>>>>>>>>>> wrote:
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Apr 16, 2017 7:01 AM, "Nir
Soffer" <nsoffer(a)redhat.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny
<dyasny(a)gmail.com>
>>>>>>>>>>> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> When you set up a storage domain, you
need to specify a host
>>>>>>>>>>>> to perform the initial storage
operations, but once the SD is defined, it's
>>>>>>>>>>>> details are in the engine database, and
all the hosts get connected to it
>>>>>>>>>>>> directly. If the first host you used to
define the SD goes down, all other
>>>>>>>>>>>> hosts will still remain connected and
work. SPM is an HA service, and if
>>>>>>>>>>>> the current SPM host goes down, SPM gets
started on another host in the DC.
>>>>>>>>>>>> In short, unless your actual NFS
exporting host goes down, there is no
>>>>>>>>>>>> outage.
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> There is no storage outage, but if you
shutdown the spm host,
>>>>>>>>>>> the spm host
>>>>>>>>>>> will not move to a new host until the spm
host is online again,
>>>>>>>>>>> or you confirm
>>>>>>>>>>> manually that the spm host was rebooted.
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> In a properly configured setup the SBA should
take care of
>>>>>>>>>>> that. That's the whole point of HA
services
>>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> In some cases like power loss or hardware
failure, there is no
>>>>>>>>>> way to start
>>>>>>>>>> the spm host, and the system cannot recover
automatically.
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>> There are always corner cases, no doubt. But in a
normal
>>>>>>>>> situation. where an SPM host goes down because of a
hardware failure, it
>>>>>>>>> gets fenced, other hosts contend for SPM and start
it. No surprises there.
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> Nir
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> Nir
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>>
>>>>>>>>>>>> On Sat, Apr 15, 2017 at 1:53 PM,
Konstantin Raskoshnyi <
>>>>>>>>>>>> konrasko(a)gmail.com> wrote:
>>>>>>>>>>>>
>>>>>>>>>>>>> Hi Fernando,
>>>>>>>>>>>>> I see each host has direct connection
nfs mount, but yes, if
>>>>>>>>>>>>> main host to which I connected nfs
storage going down the storage becomes
>>>>>>>>>>>>> unavailable and all vms are down
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 10:37 AM
FERNANDO FREDIANI <
>>>>>>>>>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>>>>>>>>>
>>>>>>>>>>>>>> Hello Konstantin.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> That doesn`t make much sense make
a whole cluster depend on
>>>>>>>>>>>>>> a single host. From what I know
any host talk directly to NFS Storage Array
>>>>>>>>>>>>>> or whatever other Shared Storage
you have.
>>>>>>>>>>>>>> Have you tested that host going
down if that affects the
>>>>>>>>>>>>>> other with the NFS mounted
directlly in a NFS Storage array ?
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> 2017-04-15 12:42 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> In ovirt you have to attach
storage through specific host.
>>>>>>>>>>>>>>> If host goes down storage is
not available.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On Sat, Apr 15, 2017 at 7:31
AM FERNANDO FREDIANI <
>>>>>>>>>>>>>>> fernando.frediani(a)upx.com>
wrote:
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Well, make it not go
through host1 and dedicate a storage
>>>>>>>>>>>>>>>> server for running NFS
and make both hosts connect to it.
>>>>>>>>>>>>>>>> In my view NFS is much
easier to manage than any other
>>>>>>>>>>>>>>>> type of storage,
specially FC and iSCSI and performance is pretty much the
>>>>>>>>>>>>>>>> same, so you won`t get
better results other than management going to other
>>>>>>>>>>>>>>>> type.
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Fernando
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> 2017-04-15 5:25 GMT-03:00
Konstantin Raskoshnyi <
>>>>>>>>>>>>>>>> konrasko(a)gmail.com>:
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Hi guys,
>>>>>>>>>>>>>>>>> I have one nfs
storage,
>>>>>>>>>>>>>>>>> it's connected
through host1.
>>>>>>>>>>>>>>>>> host2 also has access
to it, I can easily migrate
>>>>>>>>>>>>>>>>> vms between them.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> The question is - if
host1 is down - all infrastructure
>>>>>>>>>>>>>>>>> is down, since all
traffic goes through host1,
>>>>>>>>>>>>>>>>> is there any way in
oVirt to use redundant storage?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Only glusterfs?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>
>>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>>> Users mailing list
>>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
_______________________________________________
>>>>>>>>>>> Users mailing list
>>>>>>>>>>> Users(a)ovirt.org
>>>>>>>>>>>
http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>
7:44 p.m.
What's SBA?
On Sun, Apr 16, 2017 at 4:05 AM Dan Yasny <dyasny(a)redhat.com> wrote:
On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
> When you set up a storage domain, you need to specify a host to perform
> the initial storage operations, but once the SD is defined, it's details
> are in the engine database, and all the hosts get connected to it directly.
> If the first host you used to define the SD goes down, all other hosts will
> still remain connected and work. SPM is an HA service, and if the current
> SPM host goes down, SPM gets started on another host in the DC. In short,
> unless your actual NFS exporting host goes down, there is no outage.
>
There is no storage outage, but if you shutdown the spm host, the spm host
will not move to a new host until the spm host is online again, or you
confirm
manually that the spm host was rebooted.
In a properly configured setup the SBA should take care of that. That's
the whole point of HA services
Nir
>
> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
> konrasko(a)gmail.com> wrote:
>
>> Hi Fernando,
>> I see each host has direct connection nfs mount, but yes, if main host
>> to which I connected nfs storage going down the storage becomes unavailable
>> and all vms are down
>>
>>
>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>> fernando.frediani(a)upx.com> wrote:
>>
>>> Hello Konstantin.
>>>
>>> That doesn`t make much sense make a whole cluster depend on a single
>>> host. From what I know any host talk directly to NFS Storage Array or
>>> whatever other Shared Storage you have.
>>> Have you tested that host going down if that affects the other with the
>>> NFS mounted directlly in a NFS Storage array ?
>>>
>>> Fernando
>>>
>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi <konrasko(a)gmail.com>:
>>>
>>>> In ovirt you have to attach storage through specific host.
>>>> If host goes down storage is not available.
>>>>
>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>> fernando.frediani(a)upx.com> wrote:
>>>>
>>>>> Well, make it not go through host1 and dedicate a storage server for
>>>>> running NFS and make both hosts connect to it.
>>>>> In my view NFS is much easier to manage than any other type of
>>>>> storage, specially FC and iSCSI and performance is pretty much the
same, so
>>>>> you won`t get better results other than management going to other
type.
>>>>>
>>>>> Fernando
>>>>>
>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>>
>>>>>> Hi guys,
>>>>>> I have one nfs storage,
>>>>>> it's connected through host1.
>>>>>> host2 also has access to it, I can easily migrate vms between
them.
>>>>>>
>>>>>> The question is - if host1 is down - all infrastructure is down,
>>>>>> since all traffic goes through host1,
>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>
>>>>>> Only glusterfs?
>>>>>>
>>>>>> Thanks
>>>>>>
>>>>>>
>>>>>> _______________________________________________
>>>>>> Users mailing list
>>>>>> Users(a)ovirt.org
>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>
>>>>>>
>>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:12 a.m.
On Sun, Apr 16, 2017 at 12:44 PM, Konstantin Raskoshnyi <konrasko(a)gmail.com>
wrote:
What's SBA?
split brain avoidance mechanism
On Sun, Apr 16, 2017 at 4:05 AM Dan Yasny <dyasny(a)redhat.com> wrote:
>
>
> On Apr 16, 2017 7:01 AM, "Nir Soffer" <nsoffer(a)redhat.com> wrote:
>
> On Sun, Apr 16, 2017 at 4:17 AM Dan Yasny <dyasny(a)gmail.com> wrote:
>
>> When you set up a storage domain, you need to specify a host to perform
>> the initial storage operations, but once the SD is defined, it's details
>> are in the engine database, and all the hosts get connected to it directly.
>> If the first host you used to define the SD goes down, all other hosts will
>> still remain connected and work. SPM is an HA service, and if the current
>> SPM host goes down, SPM gets started on another host in the DC. In short,
>> unless your actual NFS exporting host goes down, there is no outage.
>>
>
> There is no storage outage, but if you shutdown the spm host, the spm host
> will not move to a new host until the spm host is online again, or you
> confirm
> manually that the spm host was rebooted.
>
>
> In a properly configured setup the SBA should take care of that. That's
> the whole point of HA services
>
>
> Nir
>
>
>>
>> On Sat, Apr 15, 2017 at 1:53 PM, Konstantin Raskoshnyi <
>> konrasko(a)gmail.com> wrote:
>>
>>> Hi Fernando,
>>> I see each host has direct connection nfs mount, but yes, if main host
>>> to which I connected nfs storage going down the storage becomes unavailable
>>> and all vms are down
>>>
>>>
>>> On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <
>>> fernando.frediani(a)upx.com> wrote:
>>>
>>>> Hello Konstantin.
>>>>
>>>> That doesn`t make much sense make a whole cluster depend on a single
>>>> host. From what I know any host talk directly to NFS Storage Array or
>>>> whatever other Shared Storage you have.
>>>> Have you tested that host going down if that affects the other with
>>>> the NFS mounted directlly in a NFS Storage array ?
>>>>
>>>> Fernando
>>>>
>>>> 2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>:
>>>>
>>>>> In ovirt you have to attach storage through specific host.
>>>>> If host goes down storage is not available.
>>>>>
>>>>> On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI <
>>>>> fernando.frediani(a)upx.com> wrote:
>>>>>
>>>>>> Well, make it not go through host1 and dedicate a storage server
for
>>>>>> running NFS and make both hosts connect to it.
>>>>>> In my view NFS is much easier to manage than any other type of
>>>>>> storage, specially FC and iSCSI and performance is pretty much
the same, so
>>>>>> you won`t get better results other than management going to other
type.
>>>>>>
>>>>>> Fernando
>>>>>>
>>>>>> 2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com>
>>>>>> :
>>>>>>
>>>>>>> Hi guys,
>>>>>>> I have one nfs storage,
>>>>>>> it's connected through host1.
>>>>>>> host2 also has access to it, I can easily migrate vms between
them.
>>>>>>>
>>>>>>> The question is - if host1 is down - all infrastructure is
down,
>>>>>>> since all traffic goes through host1,
>>>>>>> is there any way in oVirt to use redundant storage?
>>>>>>>
>>>>>>> Only glusterfs?
>>>>>>>
>>>>>>> Thanks
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> Users mailing list
>>>>>>> Users(a)ovirt.org
>>>>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>>>>
>>>>>>>
>>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>>
>> _______________________________________________
>> Users mailing list
>> Users(a)ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
3:41 p.m.
This is a multi-part message in MIME format.
--------------C778CD5E4FC1747C2A9FE3B2
Content-Type: text/plain; charset=utf-8; format=flowed
Content-Transfer-Encoding: 7bit
If this works in this way then it's a huge downside in the architecture.
Perhaps someone can clarify in more details.
Fernando
On 15/04/2017 14:53, Konstantin Raskoshnyi wrote:
Hi Fernando,
I see each host has direct connection nfs mount, but yes, if main host
to which I connected nfs storage going down the storage becomes
unavailable and all vms are down
On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI
<fernando.frediani(a)upx.com <mailto:fernando.frediani@upx.com>> wrote:
Hello Konstantin.
That doesn`t make much sense make a whole cluster depend on a
single host. From what I know any host talk directly to NFS
Storage Array or whatever other Shared Storage you have.
Have you tested that host going down if that affects the other
with the NFS mounted directlly in a NFS Storage array ?
Fernando
2017-04-15 12:42 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com <mailto:konrasko@gmail.com>>:
In ovirt you have to attach storage through specific host.
If host goes down storage is not available.
On Sat, Apr 15, 2017 at 7:31 AM FERNANDO FREDIANI
<fernando.frediani(a)upx.com <mailto:fernando.frediani@upx.com>>
wrote:
Well, make it not go through host1 and dedicate a storage
server for running NFS and make both hosts connect to it.
In my view NFS is much easier to manage than any other
type of storage, specially FC and iSCSI and performance is
pretty much the same, so you won`t get better results
other than management going to other type.
Fernando
2017-04-15 5:25 GMT-03:00 Konstantin Raskoshnyi
<konrasko(a)gmail.com <mailto:konrasko@gmail.com>>:
Hi guys,
I have one nfs storage,
it's connected through host1.
host2 also has access to it, I can easily migrate
vms between them.
The question is - if host1 is down - all
infrastructure is down, since all traffic goes through
host1,
is there any way in oVirt to use redundant storage?
Only glusterfs?
Thanks
_______________________________________________
Users mailing list
Users(a)ovirt.org <mailto:Users@ovirt.org>
http://lists.ovirt.org/mailman/listinfo/users
--------------C778CD5E4FC1747C2A9FE3B2
Content-Type: text/html; charset=utf-8
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=utf-8"
http-equiv="Content-Type">
</head>
<body bgcolor="#FFFFFF" text="#000000">
<p>If this works in this way then it's a huge downside in the
architecture. Perhaps someone can clarify in more details.</p>
<p>Fernando<br>
</p>
<br>
<div class="moz-cite-prefix">On 15/04/2017 14:53, Konstantin
Raskoshnyi wrote:<br>
</div>
<blockquote
cite="mid:CAJU06NOAMePk_OaA1yAgeV0pHdb9qXR7Yh=Cui3Gixs=vtnT-Q@mail.gmail.com"
type="cite">
<div>Hi Fernando,</div>
<div>I see each host has direct connection nfs mount, but yes, if
main host to which I connected nfs storage going down the
storage becomes unavailable and all vms are down</div>
<div><br>
</div>
<div><br>
<div class="gmail_quote">
<div>On Sat, Apr 15, 2017 at 10:37 AM FERNANDO FREDIANI <<a
moz-do-not-send="true"
href="mailto:fernando.frediani@upx.com">fernando.frediani@upx.com</a>>
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>
<div>
<div>
<div>Hello Konstantin.<br>
<br>
</div>
That doesn`t make much sense make a whole cluster
depend on a single host. From what I know any host
talk directly to NFS Storage Array or whatever other
Shared Storage you have.<br>
</div>
Have you tested that host going down if that affects the
other with the NFS mounted directlly in a NFS Storage
array ?<br>
<br>
</div>
</div>
<div>Fernando<br>
</div>
<div class="gmail_extra"><br>
<div class="gmail_quote">2017-04-15 12:42 GMT-03:00
Konstantin Raskoshnyi <span><<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
<blockquote class="gmail_quote" style="margin:0 0 0
.8ex;border-left:1px #ccc solid;padding-left:1ex">
<div>In ovirt you have to attach storage through
specific host. </div>
<div>If host goes down storage is not available. </div>
<div><br>
<div class="gmail_quote">
<div>On Sat, Apr 15, 2017 at 7:31 AM FERNANDO
FREDIANI <<a moz-do-not-send="true"
href="mailto:fernando.frediani@upx.com"
target="_blank">fernando.frediani(a)upx.com</a>&gt;
wrote:<br>
</div>
<blockquote class="gmail_quote" style="margin:0 0
0 .8ex;border-left:1px #ccc
solid;padding-left:1ex">
<div>
<div>
<div>Well, make it not go through host1 and
dedicate a storage server for running NFS
and make both hosts connect to it.<br>
</div>
In my view NFS is much easier to manage than
any other type of storage, specially FC and
iSCSI and performance is pretty much the
same, so you won`t get better results other
than management going to other type.<br>
<br>
</div>
Fernando<br>
</div>
<div class="gmail_extra"><br>
</div>
<div class="gmail_extra">
<div class="gmail_quote">2017-04-15 5:25
GMT-03:00 Konstantin Raskoshnyi <span><<a
moz-do-not-send="true"
href="mailto:konrasko@gmail.com"
target="_blank">konrasko@gmail.com</a>></span>:<br>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc solid;padding-left:1ex">
<div>Hi guys,
<div>I have one nfs storage,</div>
<div>it's connected through host1.</div>
<div>host2 also has access to it, I can
easily migrate vms between them.</div>
<div><br>
</div>
<div>The question is - if host1 is down
- all infrastructure is down, since
all traffic goes through host1,</div>
<div>is there any way in oVirt to use
redundant storage?</div>
<div><br>
</div>
<div>Only glusterfs?</div>
<div><br>
</div>
<div>Thanks</div>
<div><br>
</div>
</div>
<br>
</blockquote>
</div>
</div>
<div class="gmail_extra">
<div class="gmail_quote">
<blockquote class="gmail_quote"
style="margin:0 0 0 .8ex;border-left:1px
#ccc
solid;padding-left:1ex">_______________________________________________<br>
Users mailing list<br>
<a moz-do-not-send="true"
href="mailto:Users@ovirt.org"
target="_blank">Users(a)ovirt.org</a><br>
<a moz-do-not-send="true"
href="http://lists.ovirt.org/mailman/listinfo/users"
rel="noreferrer"
target="_blank">http://lists.ovirt.org/mailman/listinfo/user...
<br>
</blockquote>
</div>
</div>
</blockquote>
</div>
</div>
</blockquote>
</div>
<br>
</div>
</blockquote>
</div>
</div>
</blockquote>
<br>
</body>
</html>
--------------C778CD5E4FC1747C2A9FE3B2--
2775
days inactive
2778
days old
36 comments
9 participants
participants (9)
-
Adam Litke -
Charles Kozler -
Dan Yasny -
Dan Yasny -
FERNANDO FREDIANI -
Konstantin Raskoshnyi -
Nir Soffer -
Niyazi Elvan -
Pavel Gashev