3:39 p.m.
--=-IhXY1VMfXvq07Otm13OK
Content-Type: text/plain; charset="UTF-8"
Content-Transfer-Encoding: quoted-printable
On Mi, 2014-04-09 at 14:42 +0200, Jiri Moskovcak wrote:
On 04/09/2014 02:32 PM, Daniel Helgenberger wrote:
> On Mi, 2014-04-09 at 09:18 +0200, Jiri Moskovcak wrote:
>> On 04/08/2014 06:09 PM, Daniel Helgenberger wrote:
>>> Hello,
>>>
>>> I have an oVirt 3.4 hosted engine lab setup witch I am evaluating for
>>> production use.
>>>
>>> I "simulated" an ungraceful shutdown of all HA nodes (powercut)
while
>>> the engine was running. After powering up, the system did not recover
>>> itself (it seemed).
>>> I had to restart the ovirt-hosted-ha service (witch was in a locked
>>> state) and then manually run 'hosted-engine --vm-start'.
>>>
>>> What is the supposed procedure after a shutdown (graceful / ungracefu=
l)
>>> of Hosted-Engine HA nodes? Should the engine recover by
itself? Shoul=
d
>>> the running VM's be restarted automatically?
>>
>> When this happens the agent should start the engine VM and the engine
>> should take care of restarting the VMs which were running on that
>> restarted host and are marked as HA. Can you please provide contents o=
v
>> /var/log/ovirt* from the host after the powercut when the
engine VM
>> doesn't come up?
>>
> Hello Jirka,
>
> I accidentally already send the message without pointing out the
> interesting part; this is:
>
> <<< start logging ha-agent after reboot:
> /var/log/ovirt-hosted-engine-ha/agent.log:MainTMainThread::INFO::2014-0=
4-08
15:53:33,862::agent::52::ovirt_hosted_engine_ha.agent.agent.Agent::(ru=
n) ovirt-hosted-engine-ha agent 1.1.2-1 started
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:33,936::hosted_engine::223::ovirt_hosted_engine_ha.agent.hosted_engin=
e.HostedEngine::(_get_hostname) Found certificate common name: 192.168.50.2=
01
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:33,937::hosted_engine::363::ovirt_hosted_engine_ha.agent.hosted_engin=
e.HostedEngine::(_initialize_broker) Initializing ha-broker connection
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:33,937::brokerlink::126::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Starting monitor ping, options {'addr': '192.168.50.1=
'}
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:33,939::brokerlink::137::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Success, id 139700911299600
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:33,939::brokerlink::126::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Starting monitor mgmt-bridge, options {'use_ssl': 'tr=
ue', 'bridge_name': 'ovirtmgmt', 'address': '0'}
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,013::brokerlink::137::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Success, id 139700911300304
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,013::brokerlink::126::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Starting monitor mem-free, options {'use_ssl':
'true'=
, 'address': '0'}
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,015::brokerlink::137::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Success, id 139700911300112
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,015::brokerlink::126::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Starting monitor cpu-load-no-engine, options {'use_ss=
l': 'true', 'vm_uuid': 'e68a11c8-1251-4c13-9e3b-3847bbb4fa3d',
'address': '=
0'}
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,018::brokerlink::137::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Success, id 139700911300240
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,018::brokerlink::126::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Starting monitor engine-health, options {'use_ssl': '=
true', 'vm_uuid': 'e68a11c8-1251-4c13-9e3b-3847bbb4fa3d',
'address': '0'}
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,024::brokerlink::137::ovirt_hosted_engine_ha.lib.brokerlink.Broker=
Link::(start_monitor) Success, id 139700723857104
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,024::hosted_engine::386::ovirt_hosted_engine_ha.agent.hosted_engin=
e.HostedEngine::(_initialize_broker) Broker initialized, all submonitors st=
arted
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:53:34,312::hosted_engine::430::ovirt_hosted_engine_ha.agent.hosted_engin=
e.HostedEngine::(_cond_start_service) Starting vdsmd
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::CRITICAL::2014-04=
-08
15:53:34,442::agent::103::ovirt_hosted_engine_ha.agent.agent.Agent::(ru=
n) Could not start ha-agent
> > (10 min nothing)
> > <<< here I did a 'service ovirt-hosted-ha start'
>
/var/log/ovirt-hosted-engine-ha/agent.log:MainThread::INFO::2014-04-08 =
15:59:16,698::agent::52::ovirt_hosted_engine_ha.agent.agent.Agent::(run) ov=
irt-hosted-engine-ha agent 1.1.2-1 started
> ....
>
> after this things went quite smoothly.
>
=20
Hi Daniel,
I noticed that in the log and I was just about to ask if that's when you=
=20
manually fixed it. Is there something else around that time in=20
/var/log/message which might be related to it?
No problem, here we go at the relevant time. I added some newlines for
the interesting parts - otherwise it is complete:
Apr 8 15:53:29 nodehv01 kernel: mboxdomain: port 1(eth1.1) entering forwar=
ding state
Apr 8 15:53:29 nodehv01 rpc.statd[1924]: Version 1.2.3 starting
Apr 8 15:53:30 nodehv01 sm-notify[1925]: Version 1.2.3 starting
Apr 8 15:53:34 nodehv01 wdmd[2315]: wdmd started S0 H1 G179
Apr 8 15:53:34 nodehv01 wdmd[2315]: /dev/watchdog armed with fire_timeout =
60
---
Apr 8 15:53:34 nodehv01 ovirt-ha-agent ovirt_hosted_engine_ha.agent.agent.=
Agent CRITICAL Could not start ha-agent#012Traceback (most recent call last=
):#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/agent=
/agent.py", line 97, in run#012 self._run_agent()#012 File "/usr/lib/py=
thon2.6/site-packages/ovirt_hosted_engine_ha/agent/agent.py", line 154, in =
_run_agent#012 hosted_engine.HostedEngine(self.shutdown_requested).start=
_monitoring()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engi=
ne_ha/agent/hosted_engine.py", line 299, in start_monitoring#012 self._i=
nitialize_vdsm()#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_e=
ngine_ha/agent/hosted_engine.py", line 390, in _initialize_vdsm#012 self=
._cond_start_service('vdsmd')#012 File "/usr/lib/python2.6/site-packages/o=
virt_hosted_engine_ha/agent/hosted_engine.py", line 439, in _cond_start_ser=
vice#012 .format(service_name, res[1]))#012Exception: Could not start vd=
smd:=20
---
Apr 8 15:53:34 nodehv01 kernel: device-mapper: table: 253:2: multipath: er=
ror getting device
Apr 8 15:53:34 nodehv01 kernel: device-mapper: ioctl: error adding target =
to table
Apr 8 15:53:34 nodehv01 multipathd: 1ATA_ST3500418AS_6VMNMNV0: ignoring ma=
p
Apr 8 15:53:34 nodehv01 multipathd: path checkers start up
Apr 8 15:53:35 nodehv01 ntpd[2463]: ntpd 4.2.6p5(a)1.2349-o Sat Nov 23 18:21=
:48 UTC 2013 (1)
Apr 8 15:53:35 nodehv01 ntpd[2464]: proto: precision =3D 0.062 usec
Apr 8 15:53:35 nodehv01 ntpd[2464]: 0.0.0.0 c01d 0d kern kernel time sync =
enabled
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen and drop on 0 v4wildcard 0.0.0.=
0 UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen and drop on 1 v6wildcard :: UDP=
123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 2 lo 127.0.0.1 UDP =
123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 3 ovirtmgmt 192.168=
.50.201 UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 4 lo ::1 UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 5 eth1.1 fe80::215:=
17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 6 eth1.100 fe80::21=
5:17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 7 eth1.1000 fe80::2=
15:17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 8 eth1 fe80::215:17=
ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 9 mboxsan fe80::215=
:17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 10 mboxdomain fe80:=
:215:17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 11 mboxdmz fe80::21=
5:17ff:fe21:111f UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 12 eth0 fe80::215:1=
7ff:fe21:111e UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listen normally on 13 ovirtmgmt fe80::=
215:17ff:fe21:111e UDP 123
Apr 8 15:53:35 nodehv01 ntpd[2464]: peers refreshed
Apr 8 15:53:35 nodehv01 ntpd[2464]: Listening on routing socket on fd #30 =
for interface updates
Apr 8 15:53:35 nodehv01 kernel: Loading iSCSI transport class v2.0-870.
Apr 8 15:53:35 nodehv01 ntpd[2464]: 0.0.0.0 c016 06 restart
Apr 8 15:53:35 nodehv01 ntpd[2464]: 0.0.0.0 c012 02 freq_set kernel 3.731 =
PPM
Apr 8 15:53:35 nodehv01 kernel: iscsi: registered transport (tcp)
Apr 8 15:53:35 nodehv01 kernel: iscsi: registered transport (iser)
Apr 8 15:53:35 nodehv01 kernel: libcxgbi:libcxgbi_init_module: tag itt 0x1=
fff, 13 bits, age 0xf, 4 bits.
Apr 8 15:53:35 nodehv01 kernel: libcxgbi:ddp_setup_host_page_size: system =
PAGE 4096, ddp idx 0.
Apr 8 15:53:35 nodehv01 kernel: Chelsio T3 iSCSI Driver cxgb3i v2.0.0 (Jun=
. 2010)
Apr 8 15:53:35 nodehv01 kernel: iscsi: registered transport (cxgb3i)
Apr 8 15:53:35 nodehv01 kernel: Chelsio T4 iSCSI Driver cxgb4i v0.9.1 (Aug=
. 2010)
Apr 8 15:53:35 nodehv01 kernel: iscsi: registered transport (cxgb4i)
Apr 8 15:53:35 nodehv01 kernel: cnic: Broadcom NetXtreme II CNIC Driver cn=
ic v2.5.16 (Dec 05, 2012)
Apr 8 15:53:36 nodehv01 kernel: Broadcom NetXtreme II iSCSI Driver bnx2i v=
2.7.6.2 (Jun 06, 2013)
Apr 8 15:53:36 nodehv01 kernel: iscsi: registered transport (bnx2i)
Apr 8 15:53:36 nodehv01 kernel: iscsi: registered transport (be2iscsi)
Apr 8 15:53:36 nodehv01 kernel: In beiscsi_module_init, tt=3Dffffffffa0699=
e80
Apr 8 15:53:36 nodehv01 iscsid: iSCSI logger with pid=3D2619 started!
Apr 8 15:53:36 nodehv01 kernel: Ebtables v2.0 registered
---
Apr 8 15:53:37 nodehv01 ovirt-ha-broker mem_free.MemFree ERROR Failed to g=
etVdsStats: VDSM initialization timeout
Apr 8 15:53:37 nodehv01 ovirt-ha-broker engine_health.CpuLoadNoEngine ERRO=
R Failed to getVmStats: VDSM initialization timeout
Apr 8 15:53:37 nodehv01 ovirt-ha-broker cpu_load_no_engine.EngineHealth ER=
ROR Failed to getVmStats: VDSM initialization timeout
Apr 8 15:53:37 nodehv01 ovirt-ha-broker mgmt_bridge.MgmtBridge ERROR Faile=
d to getVdsCapabilities: VDSM initialization timeout
---
Apr 8 15:53:37 nodehv01 iscsid: iSCSI daemon with pid=3D2620 started!
Apr 8 15:53:37 nodehv01 kernel: lo: Disabled Privacy Extensions
Apr 8 15:53:38 nodehv01 kernel: Ethernet Channel Bonding Driver: v3.6.0 (S=
eptember 26, 2009)
Apr 8 15:53:38 nodehv01 kernel: bonding: bond4 is being created...
Apr 8 15:53:38 nodehv01 kernel: bonding: bond1 is being created...
Apr 8 15:53:38 nodehv01 kernel: bonding: bond2 is being created...
Apr 8 15:53:38 nodehv01 kernel: bonding: bond3 is being created...
Apr 8 15:53:40 nodehv01 kernel: device-mapper: table: 253:2: multipath: er=
ror getting device
Apr 8 15:53:40 nodehv01 kernel: device-mapper: ioctl: error adding target =
to table
Apr 8 15:53:40 nodehv01 vdsm vds WARNING Unable to load the json rpc serve=
r module. Please make sure it is installed.
Apr 8 15:53:42 nodehv01 ntpd[2464]: 0.0.0.0 c61c 0c clock_step +0.171138 s
Apr 8 15:53:42 nodehv01 ntpd[2464]: 0.0.0.0 c614 04 freq_mode
Apr 8 15:53:43 nodehv01 ntpd[2464]: 0.0.0.0 c618 08 no_sys_peer
---
Apr 8 15:59:16 nodehv01 vdsm vds ERROR failed to retrieve Hosted Engine HA=
info#012Traceback (most recent call last):#012 File "/usr/share/vdsm/API.=
py", line 1603, in _getHaInfo#012 stats =3D instance.get_all_stats()#012=
File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/client/clie=
nt.py", line 85, in get_all_stats#012 path.get_metadata_path(self._confi=
g),#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_engine_ha/env/=
path.py", line 47, in get_metadata_path#012 return os.path.join(get_doma=
in_path(config_),#012 File "/usr/lib/python2.6/site-packages/ovirt_hosted_=
engine_ha/env/path.py", line 40, in get_domain_path#012 .format(sd_uuid,=
parent))#012Exception: path to storage domain bcfa7ec4-5278-44d8-9f31-682f=
2d9de91d not found in /rhev/data-center/mnt
---
Apr 8 15:59:17 nodehv01 kernel: RPC: Registered named UNIX socket transpor=
t module.
Apr 8 15:59:17 nodehv01 kernel: RPC: Registered udp transport module.
Apr 8 15:59:17 nodehv01 kernel: RPC: Registered tcp transport module.
Apr 8 15:59:17 nodehv01 kernel: RPC: Registered tcp NFSv4.1 backchannel tr=
ansport module.
Apr 8 15:59:17 nodehv01 kernel: FS-Cache: Loaded
Apr 8 15:59:17 nodehv01 kernel: NFS: Registering the id_resolver key type
Apr 8 15:59:17 nodehv01 kernel: FS-Cache: Netfs 'nfs' registered for cachi=
ng
=20
Thanks,
Jirka
=20
>> Thanks,
>> Jirka
>
>
>>
>>>
>>> Thanks,
>>> Daniel
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Users mailing list
>>> Users(a)ovirt.org
>>> http://lists.ovirt.org/mailman/listinfo/users
>>>
>>
>
=20
--=20
Daniel Helgenberger=20
m box bewegtbild GmbH=20
P: +49/30/2408781-22
F: +49/30/2408781-10
ACKERSTR. 19=20
D-10115 BERLIN=20
www.m-box.de www.monkeymen.tv=20
Gesch=C3=A4ftsf=C3=BChrer: Martin Retschitzegger / Michaela G=C3=B6llner
Handeslregister: Amtsgericht Charlottenburg / HRB 112767=20
--=-IhXY1VMfXvq07Otm13OK
Content-Type: application/x-pkcs7-signature; name="smime.p7s"
Content-Disposition: attachment; filename="smime.p7s"
Content-Transfer-Encoding: base64
MIAGCSqGSIb3DQEHAqCAMIACAQExCzAJBgUrDgMCGgUAMIAGCSqGSIb3DQEHAQAAoIIN9zCCBFcw
ggM/oAMCAQICCwQAAAAAAS9O4TFGMA0GCSqGSIb3DQEBBQUAMFcxCzAJBgNVBAYTAkJFMRkwFwYD
VQQKExBHbG9iYWxTaWduIG52LXNhMRAwDgYDVQQLEwdSb290IENBMRswGQYDVQQDExJHbG9iYWxT
aWduIFJvb3QgQ0EwHhcNMTEwNDEzMTAwMDAwWhcNMTkwNDEzMTAwMDAwWjBUMQswCQYDVQQGEwJC
RTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEqMCgGA1UEAxMhR2xvYmFsU2lnbiBQZXJzb25h
bFNpZ24gMiBDQSAtIEcyMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAwWtB+TXs+BJ9
3SJRaV+3uRNGJ3cUO+MTgW8+5HQXfgy19CzkDI1T1NwwICi/bo4R/mYR5FEWx91//eE0ElC/89iY
7GkL0tDasmVx4TOXnrqrsziUcxEPPqHRE8x4NhtBK7+8o0nsMIJMA1gyZ2FA5To2Ew1BBuvovvDJ
+Nua3qOCNBNu+8A+eNpJlVnlu/qB7+XWaPXtUMlsIikxD+gREFVUgYE4VzBuLa2kkg0VLd09XkE2
ceRDm6YgRATuDk6ogUyX4OLxCGIJF8yi6Z37M0wemDA6Uff0EuqdwDQd5HwG/rernUjt1grLdAxq
8BwywRRg0eFHmE+ShhpyO3Fi+wIDAQABo4IBJTCCASEwDgYDVR0PAQH/BAQDAgEGMBIGA1UdEwEB
/wQIMAYBAf8CAQAwHQYDVR0OBBYEFD8V0m18L+cxnkMKBqiUbCw7xe5lMEcGA1UdIARAMD4wPAYE
VR0gADA0MDIGCCsGAQUFBwIBFiZodHRwczovL3d3dy5nbG9iYWxzaWduLmNvbS9yZXBvc2l0b3J5
LzAzBgNVHR8ELDAqMCigJqAkhiJodHRwOi8vY3JsLmdsb2JhbHNpZ24ubmV0L3Jvb3QuY3JsMD0G
CCsGAQUFBwEBBDEwLzAtBggrBgEFBQcwAYYhaHR0cDovL29jc3AuZ2xvYmFsc2lnbi5jb20vcm9v
dHIxMB8GA1UdIwQYMBaAFGB7ZhpFDZfKiVAvfQTNNKj//P1LMA0GCSqGSIb3DQEBBQUAA4IBAQDI
WOF8oQHpI41wO21cUvjE819juuGa05F5yK/ESqW+9th9vfhG92eaBSLViTIJV7gfCFbt11WexfK/
44NeiJMfi5wX6sK7Xnt8QIK5lH7ZX1Wg/zK1cXjrgRaYUOX/MA+PmuRm4gWV0zFwYOK2uv4OFgaM
mVr+8en7K1aQY2ecI9YhEaDWOcSGj6SN8DvzPdE4G4tBk4/aIsUged9sGDqRYweKla3LTNjXPps1
Y+zsVbgHLtjdOIB0YZ1hrlAQcY2L/b+V+Yyoi7CMdOtmm1Rm6Jh5ILbwQTjlUCkgu5yVdfs9LDKc
M0SPeCldkjfaGVSd+nURMOUy3hfxsMVux9+FMIIEyjCCA7KgAwIBAgIRAJZpZsDepakv5CafojXo
PKcwDQYJKoZIhvcNAQEFBQAwVDELMAkGA1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYt
c2ExKjAoBgNVBAMTIUdsb2JhbFNpZ24gUGVyc29uYWxTaWduIDIgQ0EgLSBHMjAeFw0xMzA4Mjcx
NjU3NThaFw0xNjA4MjcxNjU3NThaMFgxCzAJBgNVBAYTAkRFMRwwGgYDVQQDExNEYW5pZWwgSGVs
Z2VuYmVyZ2VyMSswKQYJKoZIhvcNAQkBFhxkYW5pZWwuaGVsZ2VuYmVyZ2VyQG0tYm94LmRlMIIB
IjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEAzgFDm8+SeTU4Yt3WopJQgqZAuuNxyMlxiPuq
0C0D581goXz2nVVjhTCIVwX2MqWYD1Dyjy1hLHXothgWgZaiQ1EB4oVdmIFmIfIjR6SkR/Gjw3lx
MwJzEpxJhZXyyrOYE8Kgw2maJWgLx5zw2/lKpcffhVW0OY0t+JWWxPKiYFcAmQnb+fleonM8sUZZ
ZES08uRVVL67jbq+3+E2xCLlqQ2iJ1h5ej3wlyuZ4CkUnfMHYrG8zOIfHwsPirWACX026a1flgts
Kl1Yv0CRZ1c5qujcP3OPpDovIbBr9RBStl2DcFdzTuGMdmfp32963VLOlvKpClPMzrfJeJfWZ4Qy
UwIDAQABo4IBkTCCAY0wDgYDVR0PAQH/BAQDAgWgMEwGA1UdIARFMEMwQQYJKwYBBAGgMgEoMDQw
MgYIKwYBBQUHAgEWJmh0dHBzOi8vd3d3Lmdsb2JhbHNpZ24uY29tL3JlcG9zaXRvcnkvMCcGA1Ud
EQQgMB6BHGRhbmllbC5oZWxnZW5iZXJnZXJAbS1ib3guZGUwCQYDVR0TBAIwADAdBgNVHSUEFjAU
BggrBgEFBQcDAgYIKwYBBQUHAwQwQwYDVR0fBDwwOjA4oDagNIYyaHR0cDovL2NybC5nbG9iYWxz
aWduLmNvbS9ncy9nc3BlcnNvbmFsc2lnbjJnMi5jcmwwVQYIKwYBBQUHAQEESTBHMEUGCCsGAQUF
BzAChjlodHRwOi8vc2VjdXJlLmdsb2JhbHNpZ24uY29tL2NhY2VydC9nc3BlcnNvbmFsc2lnbjJn
Mi5jcnQwHQYDVR0OBBYEFLw0UD+6l35aKnDaePxEP8K35HYZMB8GA1UdIwQYMBaAFD8V0m18L+cx
nkMKBqiUbCw7xe5lMA0GCSqGSIb3DQEBBQUAA4IBAQBdVOm7h+E4sRMBbTN1tCIjAEgxmB5U0mdZ
XcawzEHLJxTrc/5YFBMGX2qPju8cuZV14XszMfRBJdlJz1Od+voJggianIhnFEAakCxaa1l/cmJ5
EDT6PgZAkXbMB5rU1dhegb35lJJkcFLEpR2tF1V0TfbSe5UZNPYeMQjYsRhs69pfKLoeGm4dSLK7
gsPT5EhPd+JPyNSIootOwClMP4CTxIsXQgRI5IDqG2Ku/r2YMMLsqWD11PtAE87t2mgohQ6V1XdW
FqGd1V+wN98oPumRRS8bld+1gRA7GVYMnO5MF6p//iHFcy3MVT05ojqgomMt+voH5cFzrHA61z80
xaZ6MIIEyjCCA7KgAwIBAgIRAJZpZsDepakv5CafojXoPKcwDQYJKoZIhvcNAQEFBQAwVDELMAkG
A1UEBhMCQkUxGTAXBgNVBAoTEEdsb2JhbFNpZ24gbnYtc2ExKjAoBgNVBAMTIUdsb2JhbFNpZ24g
UGVyc29uYWxTaWduIDIgQ0EgLSBHMjAeFw0xMzA4MjcxNjU3NThaFw0xNjA4MjcxNjU3NThaMFgx
CzAJBgNVBAYTAkRFMRwwGgYDVQQDExNEYW5pZWwgSGVsZ2VuYmVyZ2VyMSswKQYJKoZIhvcNAQkB
FhxkYW5pZWwuaGVsZ2VuYmVyZ2VyQG0tYm94LmRlMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIB
CgKCAQEAzgFDm8+SeTU4Yt3WopJQgqZAuuNxyMlxiPuq0C0D581goXz2nVVjhTCIVwX2MqWYD1Dy
jy1hLHXothgWgZaiQ1EB4oVdmIFmIfIjR6SkR/Gjw3lxMwJzEpxJhZXyyrOYE8Kgw2maJWgLx5zw
2/lKpcffhVW0OY0t+JWWxPKiYFcAmQnb+fleonM8sUZZZES08uRVVL67jbq+3+E2xCLlqQ2iJ1h5
ej3wlyuZ4CkUnfMHYrG8zOIfHwsPirWACX026a1flgtsKl1Yv0CRZ1c5qujcP3OPpDovIbBr9RBS
tl2DcFdzTuGMdmfp32963VLOlvKpClPMzrfJeJfWZ4QyUwIDAQABo4IBkTCCAY0wDgYDVR0PAQH/
BAQDAgWgMEwGA1UdIARFMEMwQQYJKwYBBAGgMgEoMDQwMgYIKwYBBQUHAgEWJmh0dHBzOi8vd3d3
Lmdsb2JhbHNpZ24uY29tL3JlcG9zaXRvcnkvMCcGA1UdEQQgMB6BHGRhbmllbC5oZWxnZW5iZXJn
ZXJAbS1ib3guZGUwCQYDVR0TBAIwADAdBgNVHSUEFjAUBggrBgEFBQcDAgYIKwYBBQUHAwQwQwYD
VR0fBDwwOjA4oDagNIYyaHR0cDovL2NybC5nbG9iYWxzaWduLmNvbS9ncy9nc3BlcnNvbmFsc2ln
bjJnMi5jcmwwVQYIKwYBBQUHAQEESTBHMEUGCCsGAQUFBzAChjlodHRwOi8vc2VjdXJlLmdsb2Jh
bHNpZ24uY29tL2NhY2VydC9nc3BlcnNvbmFsc2lnbjJnMi5jcnQwHQYDVR0OBBYEFLw0UD+6l35a
KnDaePxEP8K35HYZMB8GA1UdIwQYMBaAFD8V0m18L+cxnkMKBqiUbCw7xe5lMA0GCSqGSIb3DQEB
BQUAA4IBAQBdVOm7h+E4sRMBbTN1tCIjAEgxmB5U0mdZXcawzEHLJxTrc/5YFBMGX2qPju8cuZV1
4XszMfRBJdlJz1Od+voJggianIhnFEAakCxaa1l/cmJ5EDT6PgZAkXbMB5rU1dhegb35lJJkcFLE
pR2tF1V0TfbSe5UZNPYeMQjYsRhs69pfKLoeGm4dSLK7gsPT5EhPd+JPyNSIootOwClMP4CTxIsX
QgRI5IDqG2Ku/r2YMMLsqWD11PtAE87t2mgohQ6V1XdWFqGd1V+wN98oPumRRS8bld+1gRA7GVYM
nO5MF6p//iHFcy3MVT05ojqgomMt+voH5cFzrHA61z80xaZ6MYIC5zCCAuMCAQEwaTBUMQswCQYD
VQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEqMCgGA1UEAxMhR2xvYmFsU2lnbiBQ
ZXJzb25hbFNpZ24gMiBDQSAtIEcyAhEAlmlmwN6lqS/kJp+iNeg8pzAJBgUrDgMCGgUAoIIBUzAY
BgkqhkiG9w0BCQMxCwYJKoZIhvcNAQcBMBwGCSqGSIb3DQEJBTEPFw0xNDA0MDkxMzM5MzRaMCMG
CSqGSIb3DQEJBDEWBBSM8MmC8bIk4JEBE+f9yKmcngZRATB4BgkrBgEEAYI3EAQxazBpMFQxCzAJ
BgNVBAYTAkJFMRkwFwYDVQQKExBHbG9iYWxTaWduIG52LXNhMSowKAYDVQQDEyFHbG9iYWxTaWdu
IFBlcnNvbmFsU2lnbiAyIENBIC0gRzICEQCWaWbA3qWpL+Qmn6I16DynMHoGCyqGSIb3DQEJEAIL
MWugaTBUMQswCQYDVQQGEwJCRTEZMBcGA1UEChMQR2xvYmFsU2lnbiBudi1zYTEqMCgGA1UEAxMh
R2xvYmFsU2lnbiBQZXJzb25hbFNpZ24gMiBDQSAtIEcyAhEAlmlmwN6lqS/kJp+iNeg8pzANBgkq
hkiG9w0BAQEFAASCAQAK1UaePHOIDC/IJjczWNCn7do3p9Kb2QhjMR8vS5qYhLIPlQdnguLLDZA1
Qelw1DNWg6SfnHI1Senj4WGscoH7vPwopvBQVxbdNO4eWKm1UG59/UPxCA+KMNDEvTloGlbCKXuO
MCeURHLZF1SoH62wrZhF182YlWlyLO3KtBe6eA4qyH/1ZJiYN3w/GbCDHRZh0oSXOw8/FD/JFt0F
y1dbPUDyCLn5IYqiB+5TUs/wmOwD6A6xYqT8JgjbzBC721GT0ToN+Js094scov+apPsEkDrja/HZ
kGPtqNkau1pPptjyfP4izY4WztmB2IgTU6zRfvHRLYAaVBZvtFZXgN8zAAAAAAAA
--=-IhXY1VMfXvq07Otm13OK--