9:19 p.m.
--_000_CO2PR0801MB0743180FC142ED11D7D5D3E0A62E0CO2PR0801MB0743_
Content-Type: text/plain; charset="utf-8"
Content-Transfer-Encoding: base64
SGVsbG8sDQoNCkhhdmUgMyBub2RlcyBydW5uaW5nIGJvdGggb1ZpcnQgYW5kIEdsdXN0ZXIgb24g
NCBTU0TigJlzIGVhY2guIEF0IHRoZSBtb21lbnQsIHRoZXJlIGFyZSB0d28gcGh5c2ljYWwgbmlj
cywgb25lIGhhcyBwdWJsaWMgaW50ZXJuZXQgYWNjZXNzIGFuZCB0aGUgb3RoZXIgaXMgYSBub24t
cm91dGFibGUgbmV0d29yayB1c2VkIGZvciBvdmlydG1nbXQgJiBnbHVzdGVyLg0KDQpJbiB0aGUg
bG9naWNhbCBuZXR3b3JrcywgSSBoYXZlIHNlbGVjdGVkIGdsdXN0ZXIgZm9yIHRoZSBub25yb3V0
YWJsZSBuZXR3b3JrIHJ1bm5pbmcgb3ZpcnRtZ210IGFuZCBnbHVzdGVyIGhvd2V2ZXIsIHR3byBW
TeKAmXMgcmFuZG9tbHkgcGF1c2UgZm9yIHdoYXQgc2VlbXMgbGlrZSBubyByZWFzb24uIFRoZXkg
Y2FuIGJvdGggYmUgcmVzdW1lZCB3aXRob3V0IGlzc3VlLg0KDQpPbmUgdGVzdCBWTSBoYXMgNEdC
IG9mIG1lbW9yeSBhbmQgYSBzbWFsbCBkaXNrIOKAkyBubyBwcm9ibGVtcyB3aXRoIHRoaXMgb25l
LiBUd28gb3RoZXJzIGhhdmUgODAwR0IgZGlza3MgYW5kIDMyR0Igb2YgUkFNIOKAkyBib3RoIHZt
4oCZcyBleGhpYml0IHRoZSBzYW1lIGlzc3VlLg0KDQpJIGFsc28gc2VlIHRoZXNlIGluIHRoZSBv
VmlydCBkYXNoYm9hcmQ6DQoNCg0KRmFpbGVkIHRvIHVwZGF0ZSBPVkYgZGlza3MgOWU2MDMyOGQt
MjlhZi00NTMzLTg0ZjktNjMzZDg3ZjU0OGE3LCBPVkYgZGF0YSBpc24ndCB1cGRhdGVkIG9uIHRo
b3NlIE9WRiBzdG9yZXMgKERhdGEgQ2VudGVyIHh4eHh4LCBTdG9yYWdlIERvbWFpbiBzci12b2x1
bWUwMSkuDQoNCkp1biAyMywgMjAxNiA5OjU0OjAzIFBNDQoNClZEU00gY29tbWFuZCBmYWlsZWQ6
IENvdWxkIG5vdCBhY3F1aXJlIHJlc291cmNlLiBQcm9iYWJseSByZXNvdXJjZSBmYWN0b3J5IHRo
cmV3IGFuIGV4Y2VwdGlvbi46ICgpDQoNCi8vLy8vLy8vLy8vLy8vLw0KDQpWTSB4eHh4eCBoYXMg
YmVlbiBwYXVzZWQgZHVlIHRvIHVua25vd24gc3RvcmFnZSBlcnJvci4NCg0KLy8vLy8vLy8vLy8v
Ly8vDQoNCkluIHRoZSBlcnJvciBsb2cgb24gdGhlIGVuZ2luZSwgSSBzZWUgdGhlc2U6DQoNCkVS
Uk9SIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1ZGl0bG9naGFuZGxpbmcu
QXVkaXRMb2dEaXJlY3Rvcl0gKEZvcmtKb2luUG9vbC0xLXdvcmtlci03KSBbMTBjYWY5M2VdIENv
cnJlbGF0aW9uIElEOiBudWxsLCBDYWxsIFN0YWNrOiBudWxsLCBDdXN0b20gRXZlbnQgSUQ6IC0x
LCBNZXNzYWdlOiBWTSB4eHh4eHggaGFzIGJlZW4gcGF1c2VkIGR1ZSB0byB1bmtub3duIHN0b3Jh
Z2UgZXJyb3IuDQoNCklORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1
ZGl0bG9naGFuZGxpbmcuQXVkaXRMb2dEaXJlY3Rvcl0gKEZvcmtKb2luUG9vbC0xLXdvcmtlci0x
MSkgWzEwY2FmOTNlXSBDb3JyZWxhdGlvbiBJRDogbnVsbCwgQ2FsbCBTdGFjazogbnVsbCwgQ3Vz
dG9tIEV2ZW50IElEOiAtMSwgTWVzc2FnZTogVk0geHh4eHh4IGhhcyByZWNvdmVyZWQgZnJvbSBw
YXVzZWQgYmFjayB0byB1cC4NCg0KLy8vLy8vLy8vLy8vLy8vDQoNCkhvc3RuYW1lcyBhcmUgYWxs
IGxvY2FsIHRvIC9ldGMvaG9zdHMgb24gYWxsIHNlcnZlcnMg4oCTIHRoZXkgYWxzbyByZXNvbHZl
IHdpdGhvdXQgaXNzdWUgZnJvbSBlYWNoIGhvc3QuDQoNCi8vLy8vLy8vLy8vLy8vDQoNCjIwMTYt
MDYtMjMgMjI6MDg6NTksNjExIFdBUk4gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2Vy
LmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0
elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlIGJyaWNr
ICdvdmlydDM0MzU6L21udC9kYXRhL3NyLXZvbHVtZTAxJyBvZiB2b2x1bWUgJzkzZTM2Y2RjLWFi
MWItNDFlYy1hYzdmLTk2NmNmMzg1NmI1OScgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMgbm8gZ2x1
c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05Y2QwLWI2
M2U5MTliOWFjYScNCjIwMTYtMDYtMjMgMjI6MDg6NTksNjE0IFdBUk4gIFtvcmcub3ZpcnQuZW5n
aW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1s
UnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBu
b3QgYXNzb2NpYXRlIGJyaWNrICdvdmlydDM2Mzc6L21udC9kYXRhL3NyLXZvbHVtZTAxJyBvZiB2
b2x1bWUgJzkzZTM2Y2RjLWFiMWItNDFlYy1hYzdmLTk2NmNmMzg1NmI1OScgd2l0aCBjb3JyZWN0
IG5ldHdvcmsgYXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRl
LTA0YjItNGE5OS05Y2QwLWI2M2U5MTliOWFjYScNCjIwMTYtMDYtMjMgMjI6MDg6NTksNjE2IFdB
Uk4gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVt
ZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYp
IFsxYzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlIGJyaWNrICdvdmlydDM4Mzk6L21udC9kYXRh
L3NyLXZvbHVtZTAxJyBvZiB2b2x1bWUgJzkzZTM2Y2RjLWFiMWItNDFlYy1hYzdmLTk2NmNmMzg1
NmI1OScgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGlu
IGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05Y2QwLWI2M2U5MTliOWFjYScNCjIwMTYtMDYt
MjMgMjI6MDg6NTksNjE4IFdBUk4gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmds
dXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNj
aGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlIGJyaWNrICdv
dmlydDM0MzU6L21udC9kYXRhL2Rpc3RyaWJ1dGVkJyBvZiB2b2x1bWUgJ2I4ODdiMDVlLTJlYTYt
NDk2ZS05NTUyLTE1NWQ2NThlZWFhNicgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMgbm8gZ2x1c3Rl
ciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05Y2QwLWI2M2U5
MTliOWFjYScNCjIwMTYtMDYtMjMgMjI6MDg6NTksNjIwIFdBUk4gIFtvcmcub3ZpcnQuZW5naW5l
LmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBj
XSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3Qg
YXNzb2NpYXRlIGJyaWNrICdvdmlydDM2Mzc6L21udC9kYXRhL2Rpc3RyaWJ1dGVkJyBvZiB2b2x1
bWUgJ2I4ODdiMDVlLTJlYTYtNDk2ZS05NTUyLTE1NWQ2NThlZWFhNicgd2l0aCBjb3JyZWN0IG5l
dHdvcmsgYXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0
YjItNGE5OS05Y2QwLWI2M2U5MTliOWFjYScNCjIwMTYtMDYtMjMgMjI6MDg6NTksNjIyIFdBUk4g
IFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNM
aXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsx
YzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlIGJyaWNrICdvdmlydDM4Mzk6L21udC9kYXRhL2Rp
c3RyaWJ1dGVkJyBvZiB2b2x1bWUgJ2I4ODdiMDVlLTJlYTYtNDk2ZS05NTUyLTE1NWQ2NThlZWFh
Nicgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNs
dXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05Y2QwLWI2M2U5MTliOWFjYScNCjIwMTYtMDYtMjMg
MjI6MDg6NTksNjI0IFdBUk4gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0
ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVk
dWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlIGJyaWNrICdvdmly
dDM0MzU6L21udC9kYXRhL2lzbycgb2Ygdm9sdW1lICc4OWYzMjQ1Ny1jOGMzLTQ5MGUtYjQ5MS0x
NmRkMjdkZTAwNzMnIHdpdGggY29ycmVjdCBuZXR3b3JrIGFzIG5vIGdsdXN0ZXIgbmV0d29yayBm
b3VuZCBpbiBjbHVzdGVyICc3NWJkNjRkZS0wNGIyLTRhOTktOWNkMC1iNjNlOTE5YjlhY2EnDQoy
MDE2LTA2LTIzIDIyOjA4OjU5LDYyNiBXQVJOICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jy
b2tlci5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFJldHVybkZvclhtbFJwY10gKERlZmF1bHRR
dWFydHpTY2hlZHVsZXJfV29ya2VyLTc2KSBbMWMxY2Y0Zl0gQ291bGQgbm90IGFzc29jaWF0ZSBi
cmljayAnb3ZpcnQzNjM3Oi9tbnQvZGF0YS9pc28nIG9mIHZvbHVtZSAnODlmMzI0NTctYzhjMy00
OTBlLWI0OTEtMTZkZDI3ZGUwMDczJyB3aXRoIGNvcnJlY3QgbmV0d29yayBhcyBubyBnbHVzdGVy
IG5ldHdvcmsgZm91bmQgaW4gY2x1c3RlciAnNzViZDY0ZGUtMDRiMi00YTk5LTljZDAtYjYzZTkx
OWI5YWNhJw0KMjAxNi0wNi0yMyAyMjowODo1OSw2MjggV0FSTiAgW29yZy5vdmlydC5lbmdpbmUu
Y29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RSZXR1cm5Gb3JYbWxScGNd
IChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03NikgWzFjMWNmNGZdIENvdWxkIG5vdCBh
c3NvY2lhdGUgYnJpY2sgJ292aXJ0MzgzOTovbW50L2RhdGEvaXNvJyBvZiB2b2x1bWUgJzg5ZjMy
NDU3LWM4YzMtNDkwZS1iNDkxLTE2ZGQyN2RlMDA3Mycgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMg
bm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05
Y2QwLWI2M2U5MTliOWFjYScNCjIwMTYtMDYtMjMgMjI6MDg6NTksNjI5IElORk8gIFtvcmcub3Zp
cnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0VkRTQ29t
bWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTc2KSBbMWMxY2Y0Zl0gRklOSVNI
LCBHbHVzdGVyVm9sdW1lc0xpc3RWRFNDb21tYW5kLCByZXR1cm46IHtiODg3YjA1ZS0yZWE2LTQ5
NmUtOTU1Mi0xNTVkNjU4ZWVhYTY9b3JnLm92aXJ0LmVuZ2luZS5jb3JlLmNvbW1vbi5idXNpbmVz
c2VudGl0aWVzLmdsdXN0ZXIuR2x1c3RlclZvbHVtZUVudGl0eUBkOGM5MDM5YiwgOTNlMzZjZGMt
YWIxYi00MWVjLWFjN2YtOTY2Y2YzODU2YjU5PW9yZy5vdmlydC5lbmdpbmUuY29yZS5jb21tb24u
YnVzaW5lc3NlbnRpdGllcy5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVFbnRpdHlAZmU1YWIwMTksIDg5
ZjMyNDU3LWM4YzMtNDkwZS1iNDkxLTE2ZGQyN2RlMDA3Mz1vcmcub3ZpcnQuZW5naW5lLmNvcmUu
Y29tbW9uLmJ1c2luZXNzZW50aXRpZXMuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lRW50aXR5QDlhNTZk
NjMzfSwgbG9nIGlkOiA0ODVhMDYxMQ0KMjAxNi0wNi0yMyAyMjowOTowNCw2NDUgSU5GTyAgW29y
Zy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RW
RFNDb21tYW5kXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItODkpIFtdIFNUQVJULCBH
bHVzdGVyVm9sdW1lc0xpc3RWRFNDb21tYW5kKEhvc3ROYW1lID0gb3ZpcnQzODM5LCBHbHVzdGVy
Vm9sdW1lc0xpc3RWRFNQYXJhbWV0ZXJzOntydW5Bc3luYz0ndHJ1ZScsIGhvc3RJZD0nMzJjNTAw
ZTUtMjY4ZC00MjZhLTlhNGEtMTA4NTM1ZTY3NzIyJ30pLCBsb2cgaWQ6IDQxYjY0NzlkDQo=
--_000_CO2PR0801MB0743180FC142ED11D7D5D3E0A62E0CO2PR0801MB0743_
Content-Type: text/html; charset="utf-8"
Content-ID:
<D0C4FA3A5B1EE240A396E58D5F57A2E7(a)sct-15-1-485-2-msonline-outlook-1173a.templateTenant>
Content-Transfer-Encoding: base64
PGh0bWwgeG1sbnM6bz0idXJuOnNjaGVtYXMtbWljcm9zb2Z0LWNvbTpvZmZpY2U6b2ZmaWNlIiB4
bWxuczp3PSJ1cm46c2NoZW1hcy1taWNyb3NvZnQtY29tOm9mZmljZTp3b3JkIiB4bWxuczptPSJo
dHRwOi8vc2NoZW1hcy5taWNyb3NvZnQuY29tL29mZmljZS8yMDA0LzEyL29tbWwiIHhtbG5zPSJo
dHRwOi8vd3d3LnczLm9yZy9UUi9SRUMtaHRtbDQwIj4NCjxoZWFkPg0KPG1ldGEgaHR0cC1lcXVp
dj0iQ29udGVudC1UeXBlIiBjb250ZW50PSJ0ZXh0L2h0bWw7IGNoYXJzZXQ9dXRmLTgiPg0KPG1l
dGEgbmFtZT0iR2VuZXJhdG9yIiBjb250ZW50PSJNaWNyb3NvZnQgV29yZCAxNSAoZmlsdGVyZWQg
bWVkaXVtKSI+DQo8c3R5bGU+PCEtLQ0KLyogRm9udCBEZWZpbml0aW9ucyAqLw0KQGZvbnQtZmFj
ZQ0KCXtmb250LWZhbWlseToiQ2FtYnJpYSBNYXRoIjsNCglwYW5vc2UtMToyIDQgNSAzIDUgNCA2
IDMgMiA0O30NCkBmb250LWZhY2UNCgl7Zm9udC1mYW1pbHk6Q2FsaWJyaTsNCglwYW5vc2UtMToy
IDE1IDUgMiAyIDIgNCAzIDIgNDt9DQovKiBTdHlsZSBEZWZpbml0aW9ucyAqLw0KcC5Nc29Ob3Jt
YWwsIGxpLk1zb05vcm1hbCwgZGl2Lk1zb05vcm1hbA0KCXttYXJnaW46MGluOw0KCW1hcmdpbi1i
b3R0b206LjAwMDFwdDsNCglmb250LXNpemU6MTEuMHB0Ow0KCWZvbnQtZmFtaWx5OiJDYWxpYnJp
IixzYW5zLXNlcmlmO30NCmE6bGluaywgc3Bhbi5Nc29IeXBlcmxpbmsNCgl7bXNvLXN0eWxlLXBy
aW9yaXR5Ojk5Ow0KCWNvbG9yOmJsdWU7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQph
OnZpc2l0ZWQsIHNwYW4uTXNvSHlwZXJsaW5rRm9sbG93ZWQNCgl7bXNvLXN0eWxlLXByaW9yaXR5
Ojk5Ow0KCWNvbG9yOiM5NTRGNzI7DQoJdGV4dC1kZWNvcmF0aW9uOnVuZGVybGluZTt9DQouTXNv
Q2hwRGVmYXVsdA0KCXttc28tc3R5bGUtdHlwZTpleHBvcnQtb25seTt9DQpAcGFnZSBXb3JkU2Vj
dGlvbjENCgl7c2l6ZTo4LjVpbiAxMS4waW47DQoJbWFyZ2luOjEuMGluIDEuMGluIDEuMGluIDEu
MGluO30NCmRpdi5Xb3JkU2VjdGlvbjENCgl7cGFnZTpXb3JkU2VjdGlvbjE7fQ0KLS0+PC9zdHls
ZT4NCjwvaGVhZD4NCjxib2R5IGxhbmc9IkVOLVVTIiBsaW5rPSJibHVlIiB2bGluaz0iIzk1NEY3
MiI+DQo8ZGl2IGNsYXNzPSJXb3JkU2VjdGlvbjEiPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+SGVs
bG8sPC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj5IYXZlIDMgbm9kZXMgcnVubmluZyBib3RoIG9WaXJ0IGFuZCBHbHVz
dGVyIG9uIDQgU1NE4oCZcyBlYWNoLiBBdCB0aGUgbW9tZW50LCB0aGVyZSBhcmUgdHdvIHBoeXNp
Y2FsIG5pY3MsIG9uZSBoYXMgcHVibGljIGludGVybmV0IGFjY2VzcyBhbmQgdGhlIG90aGVyIGlz
IGEgbm9uLXJvdXRhYmxlIG5ldHdvcmsgdXNlZCBmb3Igb3ZpcnRtZ210ICZhbXA7IGdsdXN0ZXIu
PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PG86cD4mbmJzcDs8L286cD48L3A+DQo8cCBjbGFz
cz0iTXNvTm9ybWFsIj5JbiB0aGUgbG9naWNhbCBuZXR3b3JrcywgSSBoYXZlIHNlbGVjdGVkIGds
dXN0ZXIgZm9yIHRoZSBub25yb3V0YWJsZSBuZXR3b3JrIHJ1bm5pbmcgb3ZpcnRtZ210IGFuZCBn
bHVzdGVyIGhvd2V2ZXIsIHR3byBWTeKAmXMgcmFuZG9tbHkgcGF1c2UgZm9yIHdoYXQgc2VlbXMg
bGlrZSBubyByZWFzb24uIFRoZXkgY2FuIGJvdGggYmUgcmVzdW1lZCB3aXRob3V0IGlzc3VlLjwv
cD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxvOnA+Jm5ic3A7PC9vOnA+PC9wPg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+T25lIHRlc3QgVk0gaGFzIDRHQiBvZiBtZW1vcnkgYW5kIGEgc21hbGwgZGlz
ayDigJMgbm8gcHJvYmxlbXMgd2l0aCB0aGlzIG9uZS4gVHdvIG90aGVycyBoYXZlIDgwMEdCIGRp
c2tzIGFuZCAzMkdCIG9mIFJBTSDigJMgYm90aCB2beKAmXMgZXhoaWJpdCB0aGUgc2FtZSBpc3N1
ZS48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBw
dDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPjxvOnA+Jm5i
c3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90Oyxz
ZXJpZiI+SSBhbHNvIHNlZSB0aGVzZSBpbiB0aGUgb1ZpcnQgZGFzaGJvYXJkOjxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+PG86
cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1
b3Q7LHNlcmlmIj4mbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJzcDsmbmJz
cDsmbmJzcDsmbmJzcDsmbmJzcDsNCjxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90
O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+RmFpbGVkIHRvIHVwZGF0ZSBPVkYgZGlza3Mg
OWU2MDMyOGQtMjlhZi00NTMzLTg0ZjktNjMzZDg3ZjU0OGE3LCBPVkYgZGF0YSBpc24ndCB1cGRh
dGVkIG9uIHRob3NlIE9WRiBzdG9yZXMgKERhdGEgQ2VudGVyIHh4eHh4LCBTdG9yYWdlIERvbWFp
biBzci12b2x1bWUwMSkuPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1h
bCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMg
TmV3IFJvbWFuJnF1b3Q7LHNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWls
eTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPkp1biAyMywgMjAxNiA5OjU0OjAz
IFBNPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1
b3Q7LHNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9y
bWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1l
cyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPlZEU00gY29tbWFuZCBmYWlsZWQ6IENvdWxkIG5vdCBh
Y3F1aXJlIHJlc291cmNlLiBQcm9iYWJseSByZXNvdXJjZSBmYWN0b3J5IHRocmV3IGFuIGV4Y2Vw
dGlvbi46ICgpPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNw
YW4gc3R5bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJv
bWFuJnF1b3Q7LHNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0i
TXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVv
dDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPi8vLy8vLy8vLy8vLy8vLzxvOnA+PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+PG86
cD4mbmJzcDs8L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1
b3Q7LHNlcmlmIj5WTSB4eHh4eCBoYXMgYmVlbiBwYXVzZWQgZHVlIHRvIHVua25vd24gc3RvcmFn
ZSBlcnJvci48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh
biBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9t
YW4mcXVvdDssc2VyaWYiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90
O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+Ly8vLy8vLy8vLy8vLy8vPG86cD48L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox
Mi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNlcmlmIj48bzpw
PiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVv
dDssc2VyaWYiPkluIHRoZSBlcnJvciBsb2cgb24gdGhlIGVuZ2luZSwgSSBzZWUgdGhlc2U6PG86
cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNl
cmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48
c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcg
Um9tYW4mcXVvdDssc2VyaWYiPkVSUk9SIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuZGFsLmRiYnJv
a2VyLmF1ZGl0bG9naGFuZGxpbmcuQXVkaXRMb2dEaXJlY3Rvcl0gKEZvcmtKb2luUG9vbC0xLXdv
cmtlci03KSBbMTBjYWY5M2VdIENvcnJlbGF0aW9uIElEOiBudWxsLCBDYWxsIFN0YWNrOiBudWxs
LCBDdXN0b20gRXZlbnQgSUQ6DQogLTEsIE1lc3NhZ2U6IFZNIHh4eHh4eCBoYXMgYmVlbiBwYXVz
ZWQgZHVlIHRvIHVua25vd24gc3RvcmFnZSBlcnJvci48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8
cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZh
bWlseTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPjxvOnA+Jm5ic3A7PC9vOnA+
PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6
MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+SU5G
TyZuYnNwOyBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmRhbC5kYmJyb2tlci5hdWRpdGxvZ2hhbmRs
aW5nLkF1ZGl0TG9nRGlyZWN0b3JdIChGb3JrSm9pblBvb2wtMS13b3JrZXItMTEpIFsxMGNhZjkz
ZV0gQ29ycmVsYXRpb24gSUQ6IG51bGwsIENhbGwgU3RhY2s6IG51bGwsIEN1c3RvbSBFdmVudCBJ
RDoNCiAtMSwgTWVzc2FnZTogVk0geHh4eHh4IGhhcyByZWNvdmVyZWQgZnJvbSBwYXVzZWQgYmFj
ayB0byB1cC48bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh
biBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9t
YW4mcXVvdDssc2VyaWYiPjxvOnA+Jm5ic3A7PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJN
c29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90
O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+Ly8vLy8vLy8vLy8vLy8vPG86cD48L286cD48
L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZTox
Mi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNlcmlmIj48bzpw
PiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHls
ZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVv
dDssc2VyaWYiPkhvc3RuYW1lcyBhcmUgYWxsIGxvY2FsIHRvIC9ldGMvaG9zdHMgb24gYWxsIHNl
cnZlcnMg4oCTIHRoZXkgYWxzbyByZXNvbHZlIHdpdGhvdXQgaXNzdWUgZnJvbSBlYWNoIGhvc3Qu
PG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9
ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7
LHNlcmlmIj48bzpwPiZuYnNwOzwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFs
Ij48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBO
ZXcgUm9tYW4mcXVvdDssc2VyaWYiPi8vLy8vLy8vLy8vLy8vPG86cD48L286cD48L3NwYW4+PC9w
Pg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9u
dC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNlcmlmIj48bzpwPiZuYnNwOzwv
bzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1z
aXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYi
PjIwMTYtMDYtMjMgMjI6MDg6NTksNjExIFdBUk4mbmJzcDsgW29yZy5vdmlydC5lbmdpbmUuY29y
ZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RSZXR1cm5Gb3JYbWxScGNdIChE
ZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03NikgWzFjMWNmNGZdIENvdWxkIG5vdCBhc3Nv
Y2lhdGUNCiBicmljayAnb3ZpcnQzNDM1Oi9tbnQvZGF0YS9zci12b2x1bWUwMScgb2Ygdm9sdW1l
ICc5M2UzNmNkYy1hYjFiLTQxZWMtYWM3Zi05NjZjZjM4NTZiNTknIHdpdGggY29ycmVjdCBuZXR3
b3JrIGFzIG5vIGdsdXN0ZXIgbmV0d29yayBmb3VuZCBpbiBjbHVzdGVyICc3NWJkNjRkZS0wNGIy
LTRhOTktOWNkMC1iNjNlOTE5YjlhY2EnPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9
Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1
b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNlcmlmIj4yMDE2LTA2LTIzIDIyOjA4OjU5LDYxNCBX
QVJOJm5ic3A7IFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3Rl
clZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3Jr
ZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3QgYXNzb2NpYXRlDQogYnJpY2sgJ292aXJ0MzYzNzov
bW50L2RhdGEvc3Itdm9sdW1lMDEnIG9mIHZvbHVtZSAnOTNlMzZjZGMtYWIxYi00MWVjLWFjN2Yt
OTY2Y2YzODU2YjU5JyB3aXRoIGNvcnJlY3QgbmV0d29yayBhcyBubyBnbHVzdGVyIG5ldHdvcmsg
Zm91bmQgaW4gY2x1c3RlciAnNzViZDY0ZGUtMDRiMi00YTk5LTljZDAtYjYzZTkxOWI5YWNhJzxv
OnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJm
b250LXNpemU6MTIuMHB0O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90Oyxz
ZXJpZiI+MjAxNi0wNi0yMyAyMjowODo1OSw2MTYgV0FSTiZuYnNwOyBbb3JnLm92aXJ0LmVuZ2lu
ZS5jb3JlLnZkc2Jyb2tlci5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFJldHVybkZvclhtbFJw
Y10gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTc2KSBbMWMxY2Y0Zl0gQ291bGQgbm90
IGFzc29jaWF0ZQ0KIGJyaWNrICdvdmlydDM4Mzk6L21udC9kYXRhL3NyLXZvbHVtZTAxJyBvZiB2
b2x1bWUgJzkzZTM2Y2RjLWFiMWItNDFlYy1hYzdmLTk2NmNmMzg1NmI1OScgd2l0aCBjb3JyZWN0
IG5ldHdvcmsgYXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRl
LTA0YjItNGE5OS05Y2QwLWI2M2U5MTliOWFjYSc8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBj
bGFzcz0iTXNvTm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWls
eTomcXVvdDtUaW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPjIwMTYtMDYtMjMgMjI6MDg6NTks
NjE4IFdBUk4mbmJzcDsgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5H
bHVzdGVyVm9sdW1lc0xpc3RSZXR1cm5Gb3JYbWxScGNdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVy
X1dvcmtlci03NikgWzFjMWNmNGZdIENvdWxkIG5vdCBhc3NvY2lhdGUNCiBicmljayAnb3ZpcnQz
NDM1Oi9tbnQvZGF0YS9kaXN0cmlidXRlZCcgb2Ygdm9sdW1lICdiODg3YjA1ZS0yZWE2LTQ5NmUt
OTU1Mi0xNTVkNjU4ZWVhYTYnIHdpdGggY29ycmVjdCBuZXR3b3JrIGFzIG5vIGdsdXN0ZXIgbmV0
d29yayBmb3VuZCBpbiBjbHVzdGVyICc3NWJkNjRkZS0wNGIyLTRhOTktOWNkMC1iNjNlOTE5Yjlh
Y2EnPG86cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5
bGU9ImZvbnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1
b3Q7LHNlcmlmIj4yMDE2LTA2LTIzIDIyOjA4OjU5LDYyMCBXQVJOJm5ic3A7IFtvcmcub3ZpcnQu
ZW5naW5lLmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9y
WG1sUnBjXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3Vs
ZCBub3QgYXNzb2NpYXRlDQogYnJpY2sgJ292aXJ0MzYzNzovbW50L2RhdGEvZGlzdHJpYnV0ZWQn
IG9mIHZvbHVtZSAnYjg4N2IwNWUtMmVhNi00OTZlLTk1NTItMTU1ZDY1OGVlYWE2JyB3aXRoIGNv
cnJlY3QgbmV0d29yayBhcyBubyBnbHVzdGVyIG5ldHdvcmsgZm91bmQgaW4gY2x1c3RlciAnNzVi
ZDY0ZGUtMDRiMi00YTk5LTljZDAtYjYzZTkxOWI5YWNhJzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4N
CjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0O2ZvbnQt
ZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+MjAxNi0wNi0yMyAyMjow
ODo1OSw2MjIgV0FSTiZuYnNwOyBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5nbHVz
dGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFJldHVybkZvclhtbFJwY10gKERlZmF1bHRRdWFydHpTY2hl
ZHVsZXJfV29ya2VyLTc2KSBbMWMxY2Y0Zl0gQ291bGQgbm90IGFzc29jaWF0ZQ0KIGJyaWNrICdv
dmlydDM4Mzk6L21udC9kYXRhL2Rpc3RyaWJ1dGVkJyBvZiB2b2x1bWUgJ2I4ODdiMDVlLTJlYTYt
NDk2ZS05NTUyLTE1NWQ2NThlZWFhNicgd2l0aCBjb3JyZWN0IG5ldHdvcmsgYXMgbm8gZ2x1c3Rl
ciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5OS05Y2QwLWI2M2U5
MTliOWFjYSc8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNvTm9ybWFsIj48c3Bh
biBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtUaW1lcyBOZXcgUm9t
YW4mcXVvdDssc2VyaWYiPjIwMTYtMDYtMjMgMjI6MDg6NTksNjI0IFdBUk4mbmJzcDsgW29yZy5v
dmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lc0xpc3RSZXR1
cm5Gb3JYbWxScGNdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03NikgWzFjMWNmNGZd
IENvdWxkIG5vdCBhc3NvY2lhdGUNCiBicmljayAnb3ZpcnQzNDM1Oi9tbnQvZGF0YS9pc28nIG9m
IHZvbHVtZSAnODlmMzI0NTctYzhjMy00OTBlLWI0OTEtMTZkZDI3ZGUwMDczJyB3aXRoIGNvcnJl
Y3QgbmV0d29yayBhcyBubyBnbHVzdGVyIG5ldHdvcmsgZm91bmQgaW4gY2x1c3RlciAnNzViZDY0
ZGUtMDRiMi00YTk5LTljZDAtYjYzZTkxOWI5YWNhJzxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjxw
IGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0O2ZvbnQtZmFt
aWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+MjAxNi0wNi0yMyAyMjowODo1
OSw2MjYgV0FSTiZuYnNwOyBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5nbHVzdGVy
LkdsdXN0ZXJWb2x1bWVzTGlzdFJldHVybkZvclhtbFJwY10gKERlZmF1bHRRdWFydHpTY2hlZHVs
ZXJfV29ya2VyLTc2KSBbMWMxY2Y0Zl0gQ291bGQgbm90IGFzc29jaWF0ZQ0KIGJyaWNrICdvdmly
dDM2Mzc6L21udC9kYXRhL2lzbycgb2Ygdm9sdW1lICc4OWYzMjQ1Ny1jOGMzLTQ5MGUtYjQ5MS0x
NmRkMjdkZTAwNzMnIHdpdGggY29ycmVjdCBuZXR3b3JrIGFzIG5vIGdsdXN0ZXIgbmV0d29yayBm
b3VuZCBpbiBjbHVzdGVyICc3NWJkNjRkZS0wNGIyLTRhOTktOWNkMC1iNjNlOTE5YjlhY2EnPG86
cD48L286cD48L3NwYW4+PC9wPg0KPHAgY2xhc3M9Ik1zb05vcm1hbCI+PHNwYW4gc3R5bGU9ImZv
bnQtc2l6ZToxMi4wcHQ7Zm9udC1mYW1pbHk6JnF1b3Q7VGltZXMgTmV3IFJvbWFuJnF1b3Q7LHNl
cmlmIj4yMDE2LTA2LTIzIDIyOjA4OjU5LDYyOCBXQVJOJm5ic3A7IFtvcmcub3ZpcnQuZW5naW5l
LmNvcmUudmRzYnJva2VyLmdsdXN0ZXIuR2x1c3RlclZvbHVtZXNMaXN0UmV0dXJuRm9yWG1sUnBj
XSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsxYzFjZjRmXSBDb3VsZCBub3Qg
YXNzb2NpYXRlDQogYnJpY2sgJ292aXJ0MzgzOTovbW50L2RhdGEvaXNvJyBvZiB2b2x1bWUgJzg5
ZjMyNDU3LWM4YzMtNDkwZS1iNDkxLTE2ZGQyN2RlMDA3Mycgd2l0aCBjb3JyZWN0IG5ldHdvcmsg
YXMgbm8gZ2x1c3RlciBuZXR3b3JrIGZvdW5kIGluIGNsdXN0ZXIgJzc1YmQ2NGRlLTA0YjItNGE5
OS05Y2QwLWI2M2U5MTliOWFjYSc8bzpwPjwvbzpwPjwvc3Bhbj48L3A+DQo8cCBjbGFzcz0iTXNv
Tm9ybWFsIj48c3BhbiBzdHlsZT0iZm9udC1zaXplOjEyLjBwdDtmb250LWZhbWlseTomcXVvdDtU
aW1lcyBOZXcgUm9tYW4mcXVvdDssc2VyaWYiPjIwMTYtMDYtMjMgMjI6MDg6NTksNjI5IElORk8m
bmJzcDsgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIuZ2x1c3Rlci5HbHVzdGVyVm9s
dW1lc0xpc3RWRFNDb21tYW5kXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFsx
YzFjZjRmXSBGSU5JU0gsIEdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmQsDQogcmV0dXJuOiB7
Yjg4N2IwNWUtMmVhNi00OTZlLTk1NTItMTU1ZDY1OGVlYWE2PW9yZy5vdmlydC5lbmdpbmUuY29y
ZS5jb21tb24uYnVzaW5lc3NlbnRpdGllcy5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVFbnRpdHlAZDhj
OTAzOWIsIDkzZTM2Y2RjLWFiMWItNDFlYy1hYzdmLTk2NmNmMzg1NmI1OT1vcmcub3ZpcnQuZW5n
aW5lLmNvcmUuY29tbW9uLmJ1c2luZXNzZW50aXRpZXMuZ2x1c3Rlci5HbHVzdGVyVm9sdW1lRW50
aXR5QGZlNWFiMDE5LCA4OWYzMjQ1Ny1jOGMzLTQ5MGUtYjQ5MS0xNmRkMjdkZTAwNzM9b3JnLm92
aXJ0LmVuZ2luZS5jb3JlLmNvbW1vbi5idXNpbmVzc2VudGl0aWVzLmdsdXN0ZXIuR2x1c3RlclZv
bHVtZUVudGl0eUA5YTU2ZDYzM30sDQogbG9nIGlkOiA0ODVhMDYxMTxvOnA+PC9vOnA+PC9zcGFu
PjwvcD4NCjxwIGNsYXNzPSJNc29Ob3JtYWwiPjxzcGFuIHN0eWxlPSJmb250LXNpemU6MTIuMHB0
O2ZvbnQtZmFtaWx5OiZxdW90O1RpbWVzIE5ldyBSb21hbiZxdW90OyxzZXJpZiI+MjAxNi0wNi0y
MyAyMjowOTowNCw2NDUgSU5GTyZuYnNwOyBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tl
ci5nbHVzdGVyLkdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2No
ZWR1bGVyX1dvcmtlci04OSkgW10gU1RBUlQsIEdsdXN0ZXJWb2x1bWVzTGlzdFZEU0NvbW1hbmQo
SG9zdE5hbWUNCiA9IG92aXJ0MzgzOSwgR2x1c3RlclZvbHVtZXNMaXN0VkRTUGFyYW1ldGVyczp7
cnVuQXN5bmM9J3RydWUnLCBob3N0SWQ9JzMyYzUwMGU1LTI2OGQtNDI2YS05YTRhLTEwODUzNWU2
NzcyMid9KSwgbG9nIGlkOiA0MWI2NDc5ZDxvOnA+PC9vOnA+PC9zcGFuPjwvcD4NCjwvZGl2Pg0K
PC9ib2R5Pg0KPC9odG1sPg0K
--_000_CO2PR0801MB0743180FC142ED11D7D5D3E0A62E0CO2PR0801MB0743_--
11:32 p.m.
This is a multi-part message in MIME format.
--------------070800090401040005080909
Content-Type: text/plain; charset=windows-1252; format=flowed
Content-Transfer-Encoding: 8bit
Can you post the gluster mount logs from the node where paused VM was
running (under
/var/log/glusterfs/rhev-datarhev-data-center-mnt-glusterSD<mount-path>.log)
?
Which version of glusterfs are you running?
On 06/24/2016 07:49 AM, Bill Bill wrote:
Hello,
Have 3 nodes running both oVirt and Gluster on 4 SSDs each. At the
moment, there are two physical nics, one has public internet access
and the other is a non-routable network used for ovirtmgmt & gluster.
In the logical networks, I have selected gluster for the nonroutable
network running ovirtmgmt and gluster however, two VMs randomly pause
for what seems like no reason. They can both be resumed without issue.
One test VM has 4GB of memory and a small disk no problems with this
one. Two others have 800GB disks and 32GB of RAM both vms exhibit
the same issue.
I also see these in the oVirt dashboard:
Failed to update OVF disks 9e60328d-29af-4533-84f9-633d87f548a7, OVF
data isn't updated on those OVF stores (Data Center xxxxx, Storage
Domain sr-volume01).
Jun 23, 2016 9:54:03 PM
VDSM command failed: Could not acquire resource. Probably resource
factory threw an exception.: ()
///////////////
VM xxxxx has been paused due to unknown storage error.
///////////////
In the error log on the engine, I see these:
ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-7) [10caf93e] Correlation ID: null, Call Stack:
null, Custom Event ID: -1, Message: VM xxxxxx has been paused due to
unknown storage error.
INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-11) [10caf93e] Correlation ID: null, Call
Stack: null, Custom Event ID: -1, Message: VM xxxxxx has recovered
from paused back to up.
///////////////
Hostnames are all local to /etc/hosts on all servers they also
resolve without issue from each host.
//////////////
2016-06-23 22:08:59,611 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,614 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,616 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,618 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,620 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,622 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,624 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,626 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,628 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network as no
gluster network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,629 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] FINISH,
GlusterVolumesListVDSCommand, return:
{b887b05e-2ea6-496e-9552-155d658eeaa6=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d8c9039b,
93e36cdc-ab1b-41ec-ac7f-966cf3856b59=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fe5ab019,
89f32457-c8c3-490e-b491-16dd27de0073=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9a56d633},
log id: 485a0611
2016-06-23 22:09:04,645 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-89) [] START,
GlusterVolumesListVDSCommand(HostName = ovirt3839,
GlusterVolumesListVDSParameters:{runAsync='true',
hostId='32c500e5-268d-426a-9a4a-108535e67722'}), log id: 41b6479d
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
--------------070800090401040005080909
Content-Type: text/html; charset=windows-1252
Content-Transfer-Encoding: 8bit
<html>
<head>
<meta content="text/html; charset=windows-1252"
http-equiv="Content-Type">
</head>
<body text="#000000" bgcolor="#FFFFFF">
Can you post the gluster mount logs from the node where paused VM
was running (under
/var/log/glusterfs/rhev-datarhev-data-center-mnt-glusterSD<mount-path>.log)
? <br>
Which version of glusterfs are you running?<br>
<br>
<div class="moz-cite-prefix">On 06/24/2016 07:49 AM, Bill Bill
wrote:<br>
</div>
<blockquote
cite="mid:CO2PR0801MB0743180FC142ED11D7D5D3E0A62E0@CO2PR0801MB0743.namprd08.prod.outlook.com"
type="cite">
<meta http-equiv="Content-Type" content="text/html;
charset=windows-1252">
<meta name="Generator" content="Microsoft Word 15 (filtered
medium)">
<style><!--
/* Font Definitions */
@font-face
{font-family:"Cambria Math";
panose-1:2 4 5 3 5 4 6 3 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0in;
margin-bottom:.0001pt;
font-size:11.0pt;
font-family:"Calibri",sans-serif;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:#954F72;
text-decoration:underline;}
.MsoChpDefault
{mso-style-type:export-only;}
@page WordSection1
{size:8.5in 11.0in;
margin:1.0in 1.0in 1.0in 1.0in;}
div.WordSection1
{page:WordSection1;}
--></style>
<div class="WordSection1">
<p class="MsoNormal">Hello,</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">Have 3 nodes running both oVirt and Gluster
on 4 SSDs each. At the moment, there are two physical nics,
one has public internet access and the other is a non-routable
network used for ovirtmgmt & gluster.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">In the logical networks, I have selected
gluster for the nonroutable network running ovirtmgmt and
gluster however, two VMs randomly pause for what seems like
no reason. They can both be resumed without issue.</p>
<p class="MsoNormal"><o:p> </o:p></p>
<p class="MsoNormal">One test VM has 4GB of memory and a small
disk no problems with this one. Two others have 800GB disks
and 32GB of RAM both vms exhibit the same issue.</p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">I also see these in the oVirt
dashboard:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">
<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">Failed to update OVF disks
9e60328d-29af-4533-84f9-633d87f548a7, OVF data isn't updated
on those OVF stores (Data Center xxxxx, Storage Domain
sr-volume01).<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">Jun 23, 2016 9:54:03
PM<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">VDSM command failed: Could not acquire
resource. Probably resource factory threw an exception.:
()<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">///////////////<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">VM xxxxx has been paused due to unknown
storage error.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">///////////////<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">In the error log on the engine, I see
these:<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-7) [10caf93e] Correlation ID: null,
Call Stack: null, Custom Event ID: -1, Message: VM xxxxxx
has been paused due to unknown storage
error.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-11) [10caf93e] Correlation ID: null,
Call Stack: null, Custom Event ID: -1, Message: VM xxxxxx
has recovered from paused back to
up.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">///////////////<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">Hostnames are all local to /etc/hosts on
all servers they also resolve without issue from each
host.<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">//////////////<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif"><o:p> </o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,611 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3435:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,614 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3637:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,616 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3839:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,618 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3435:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,620 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3637:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,622 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3839:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,624 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3435:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,626 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3637:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,628 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not
associate brick 'ovirt3839:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073' with correct network
as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:08:59,629 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] FINISH,
GlusterVolumesListVDSCommand, return:
{b887b05e-2ea6-496e-9552-155d658eeaa6=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d8c9039b,
93e36cdc-ab1b-41ec-ac7f-966cf3856b59=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fe5ab019,
89f32457-c8c3-490e-b491-16dd27de0073=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9a56d633},
log id: 485a0611<o:p></o:p></span></p>
<p class="MsoNormal"><span
style="font-size:12.0pt;font-family:"Times New
Roman",serif">2016-06-23 22:09:04,645 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-89) [] START,
GlusterVolumesListVDSCommand(HostName = ovirt3839,
GlusterVolumesListVDSParameters:{runAsync='true',
hostId='32c500e5-268d-426a-9a4a-108535e67722'}), log id:
41b6479d<o:p></o:p></span></p>
</div>
<br>
<fieldset class="mimeAttachmentHeader"></fieldset>
<br>
<pre wrap="">_______________________________________________
Users mailing list
<a class="moz-txt-link-abbreviated"
href="mailto:Users@ovirt.org">Users@ovirt.org</a>
<a class="moz-txt-link-freetext"
href="http://lists.ovirt.org/mailman/listinfo/users">http://...
</pre>
</blockquote>
<br>
</body>
</html>
--------------070800090401040005080909--
12:07 a.m.
Hi Bill,
After glusterfs 3.7.11, around 4-5 bugs were found in sharding and
replicate modules and fixed, some of them causing the VM(s) to pause. Could
you share the glusterfs client logs from around the time the issue was
seen? This will help me confirm it's the same issue, or even debug further
if this is a new issue.
-Krutika
On Fri, Jun 24, 2016 at 10:02 AM, Sahina Bose <sabose(a)redhat.com> wrote:
Can you post the gluster mount logs from the node where paused VM
was
running (under
/var/log/glusterfs/rhev-datarhev-data-center-mnt-glusterSD<mount-path>.log)
?
Which version of glusterfs are you running?
On 06/24/2016 07:49 AM, Bill Bill wrote:
Hello,
Have 3 nodes running both oVirt and Gluster on 4 SSD’s each. At the
moment, there are two physical nics, one has public internet access and the
other is a non-routable network used for ovirtmgmt & gluster.
In the logical networks, I have selected gluster for the nonroutable
network running ovirtmgmt and gluster however, two VM’s randomly pause for
what seems like no reason. They can both be resumed without issue.
One test VM has 4GB of memory and a small disk – no problems with this
one. Two others have 800GB disks and 32GB of RAM – both vm’s exhibit the
same issue.
I also see these in the oVirt dashboard:
Failed to update OVF disks 9e60328d-29af-4533-84f9-633d87f548a7, OVF data
isn't updated on those OVF stores (Data Center xxxxx, Storage Domain
sr-volume01).
Jun 23, 2016 9:54:03 PM
VDSM command failed: Could not acquire resource. Probably resource factory
threw an exception.: ()
///////////////
VM xxxxx has been paused due to unknown storage error.
///////////////
In the error log on the engine, I see these:
ERROR
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-7) [10caf93e] Correlation ID: null, Call Stack:
null, Custom Event ID: -1, Message: VM xxxxxx has been paused due to
unknown storage error.
INFO
[org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector]
(ForkJoinPool-1-worker-11) [10caf93e] Correlation ID: null, Call Stack:
null, Custom Event ID: -1, Message: VM xxxxxx has recovered from paused
back to up.
///////////////
Hostnames are all local to /etc/hosts on all servers – they also resolve
without issue from each host.
//////////////
2016-06-23 22:08:59,611 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,614 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,616 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/sr-volume01' of volume
'93e36cdc-ab1b-41ec-ac7f-966cf3856b59' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,618 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,620 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,622 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/distributed' of volume
'b887b05e-2ea6-496e-9552-155d658eeaa6' with correct network as no gluster
network found in cluster '75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,624 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3435:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073'
with correct network as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,626 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3637:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073'
with correct network as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,628 WARN
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] Could not associate brick
'ovirt3839:/mnt/data/iso' of volume
'89f32457-c8c3-490e-b491-16dd27de0073'
with correct network as no gluster network found in cluster
'75bd64de-04b2-4a99-9cd0-b63e919b9aca'
2016-06-23 22:08:59,629 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-76) [1c1cf4f] FINISH,
GlusterVolumesListVDSCommand, return:
{b887b05e-2ea6-496e-9552-155d658eeaa6=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@d8c9039b,
93e36cdc-ab1b-41ec-ac7f-966cf3856b59=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@fe5ab019,
89f32457-c8c3-490e-b491-16dd27de0073=org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeEntity@9a56d633},
log id: 485a0611
2016-06-23 22:09:04,645 INFO
[org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListVDSCommand]
(DefaultQuartzScheduler_Worker-89) [] START,
GlusterVolumesListVDSCommand(HostName = ovirt3839,
GlusterVolumesListVDSParameters:{runAsync='true',
hostId='32c500e5-268d-426a-9a4a-108535e67722'}), log id: 41b6479d
_______________________________________________
Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2883
days inactive
2886
days old
2 comments
3 participants
participants (3)
-
Bill Bill -
Krutika Dhananjay -
Sahina Bose