Failed to connect Host to the Storage Domains hosted_storage.
--Sig_/YOnKOzEqBUm8Rn2KNWH9.d_ Content-Type: multipart/mixed; boundary="MP_/bIQEAQ/NhL84eB5JxRKZOmO" --MP_/bIQEAQ/NhL84eB5JxRKZOmO Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable Content-Disposition: inline Hello, I'm in the process of upgrading from 3.5.x to 3.6.x. My hosted engine and hosts in the primary cluster are all upgraded and appear to be running fine. I have a second cluster of 2 machines which are just regular hosts, without the hosted-engine. Both have been marked non-operational, with the following messages logged about every 5 minutes: =09 Failed to connect Host perses to Storage Pool Default Host perses cannot access the Storage Domain(s) hosted_storage attached to = the Data Center Default. Setting Host state to Non-Operational. =09 Host perses reports about one of the Active Storage Domains as Problematic. =09 Failed to connect Host perses to Storage Servers =09 Failed to connect Host perses to the Storage Domains hosted_storage. I could see the normal storage/iso/export domains mounted on the host, and the VMs running on the host are fine. I shut down the VMs on one host, put it in maintenance mode, installed 3.6 repo and ran yum update. All went well, but when I activated the host, same deal. I've attached the engine log snippet for the activation attempt. Robert --=20 Senior Software Engineer @ Parsons --MP_/bIQEAQ/NhL84eB5JxRKZOmO Content-Type: application/octet-stream; name=engine.log-0722 Content-Transfer-Encoding: base64 Content-Disposition: attachment; filename=engine.log-0722 MjAxNi0wNy0yMiAwOToyMjoyOCw3OTIgSU5GTyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS5ibGwu QWN0aXZhdGVWZHNDb21tYW5kXSAob3JnLm92aXJ0LnRocmVhZC5wb29sLTgtdGhyZWFkLTI0KSBb MWE2MDA5MGFdIEJlZm9yZSBhY3F1aXJpbmcgbG9jayBpbiBvcmRlciB0byBwcmV2ZW50IG1vbml0 b3JpbmcgZm9yIGhvc3QgJ3BlcnNlcycgZnJvbSBkYXRhLWNlbnRlciAnRGVmYXVsdCcKMjAxNi0w Ny0yMiAwOToyMjoyOCw3OTMgSU5GTyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS5ibGwuQWN0aXZh dGVWZHNDb21tYW5kXSAob3JnLm92aXJ0LnRocmVhZC5wb29sLTgtdGhyZWFkLTI0KSBbMWE2MDA5 MGFdIExvY2sgYWNxdWlyZWQsIGZyb20gbm93IGEgbW9uaXRvcmluZyBvZiBob3N0IHdpbGwgYmUg c2tpcHBlZCBmb3IgaG9zdCAncGVyc2VzJyBmcm9tIGRhdGEtY2VudGVyICdEZWZhdWx0JwoyMDE2 LTA3LTIyIDA5OjIyOjI4LDgyMCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tl ci5TZXRWZHNTdGF0dXNWRFNDb21tYW5kXSAob3JnLm92aXJ0LnRocmVhZC5wb29sLTgtdGhyZWFk LTI0KSBbMWE2MDA5MGFdIFNUQVJULCBTZXRWZHNTdGF0dXNWRFNDb21tYW5kKEhvc3ROYW1lID0g cGVyc2VzLCBTZXRWZHNTdGF0dXNWRFNDb21tYW5kUGFyYW1ldGVyczp7cnVuQXN5bmM9J3RydWUn LCBob3N0SWQ9JzU3OWRmM2MwLWM3OTMtNGYwNC1hNTU5LWYwYWVlMDgzNTkwOCcsIHN0YXR1cz0n VW5hc3NpZ25lZCcsIG5vbk9wZXJhdGlvbmFsUmVhc29uPSdOT05FJywgc3RvcFNwbUZhaWx1cmVM b2dnZWQ9J2ZhbHNlJywgbWFpbnRlbmFuY2VSZWFzb249J251bGwnfSksIGxvZyBpZDogMTg4ODcz MGUKMjAxNi0wNy0yMiAwOToyMjoyOCw4MzkgSU5GTyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS52 ZHNicm9rZXIuU2V0VmRzU3RhdHVzVkRTQ29tbWFuZF0gKG9yZy5vdmlydC50aHJlYWQucG9vbC04 LXRocmVhZC0yNCkgWzFhNjAwOTBhXSBGSU5JU0gsIFNldFZkc1N0YXR1c1ZEU0NvbW1hbmQsIGxv ZyBpZDogMTg4ODczMGUKMjAxNi0wNy0yMiAwOToyMjoyOCw5ODAgSU5GTyAgW29yZy5vdmlydC5l bmdpbmUuY29yZS5ibGwuQWN0aXZhdGVWZHNDb21tYW5kXSAob3JnLm92aXJ0LnRocmVhZC5wb29s LTgtdGhyZWFkLTI0KSBbMWE2MDA5MGFdIEFjdGl2YXRlIGZpbmlzaGVkLiBMb2NrIHJlbGVhc2Vk LiBNb25pdG9yaW5nIGNhbiBydW4gbm93IGZvciBob3N0ICdwZXJzZXMnIGZyb20gZGF0YS1jZW50 ZXIgJ0RlZmF1bHQnCjIwMTYtMDctMjIgMDk6MjI6MjksMDY1IElORk8gIFtvcmcub3ZpcnQuZW5n aW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1ZGl0bG9naGFuZGxpbmcuQXVkaXRMb2dEaXJlY3Rvcl0g KG9yZy5vdmlydC50aHJlYWQucG9vbC04LXRocmVhZC0yNCkgWzFhNjAwOTBhXSBDb3JyZWxhdGlv biBJRDogMWE2MDA5MGEsIEpvYiBJRDogMzgyYWNhNWMtNDIwNy00ZjU2LWE2ZTYtMTI3ZDZiNGY2 ZTZhLCBDYWxsIFN0YWNrOiBudWxsLCBDdXN0b20gRXZlbnQgSUQ6IC0xLCBNZXNzYWdlOiBBY3Rp dmF0aW9uIG9mIGhvc3QgcGVyc2VzIGluaXRpYXRlZCBieSBhZG1pbkBpbnRlcm5hbC4KMjAxNi0w Ny0yMiAwOToyMjoyOSwxNTkgSU5GTyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS5ibGwuQWN0aXZh dGVWZHNDb21tYW5kXSAob3JnLm92aXJ0LnRocmVhZC5wb29sLTgtdGhyZWFkLTI0KSBbMWE2MDA5 MGFdIExvY2sgZnJlZWQgdG8gb2JqZWN0ICdFbmdpbmVMb2NrOntleGNsdXNpdmVMb2Nrcz0nWzU3 OWRmM2MwLWM3OTMtNGYwNC1hNTU5LWYwYWVlMDgzNTkwOD08VkRTLCBBQ1RJT05fVFlQRV9GQUlM RURfT0JKRUNUX0xPQ0tFRD5dJywgc2hhcmVkTG9ja3M9J251bGwnfScKMjAxNi0wNy0yMiAwOToy MjozMCwyMDEgSU5GTyAgW29yZy5vdmlydC52ZHNtLmpzb25ycGMuY2xpZW50LnJlYWN0b3JzLlJl YWN0b3JDbGllbnRdIChTU0wgU3RvbXAgUmVhY3RvcikgW10gQ29ubmVjdGluZyB0byBwZXJzZXMu bmV0c2VjLzEwLjcxLjEwLjIyMAoyMDE2LTA3LTIyIDA5OjIyOjMwLDg1OCBJTkZPICBbb3JnLm92 aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9rZXIuR2V0SGFyZHdhcmVJbmZvVkRTQ29t bWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTk0KSBbNGFkOGRiZmNdIFNUQVJU LCBHZXRIYXJkd2FyZUluZm9WRFNDb21tYW5kKEhvc3ROYW1lID0gcGVyc2VzLCBWZHNJZEFuZFZk c1ZEU0NvbW1hbmRQYXJhbWV0ZXJzQmFzZTp7cnVuQXN5bmM9J3RydWUnLCBob3N0SWQ9JzU3OWRm M2MwLWM3OTMtNGYwNC1hNTU5LWYwYWVlMDgzNTkwOCcsIHZkcz0nSG9zdFtwZXJzZXMsNTc5ZGYz YzAtYzc5My00ZjA0LWE1NTktZjBhZWUwODM1OTA4XSd9KSwgbG9nIGlkOiAxNDFhOGU2NAoyMDE2 LTA3LTIyIDA5OjIyOjMwLDk3NiBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tl ci52ZHNicm9rZXIuR2V0SGFyZHdhcmVJbmZvVkRTQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hl ZHVsZXJfV29ya2VyLTk0KSBbNGFkOGRiZmNdIEZJTklTSCwgR2V0SGFyZHdhcmVJbmZvVkRTQ29t bWFuZCwgbG9nIGlkOiAxNDFhOGU2NAoyMDE2LTA3LTIyIDA5OjIyOjMxLDE2OCBJTkZPICBbb3Jn Lm92aXJ0LmVuZ2luZS5jb3JlLmJsbC5IYW5kbGVWZHNDcHVGbGFnc09yQ2x1c3RlckNoYW5nZWRD b21tYW5kXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFs2OTFhMmFkZV0gUnVu bmluZyBjb21tYW5kOiBIYW5kbGVWZHNDcHVGbGFnc09yQ2x1c3RlckNoYW5nZWRDb21tYW5kIGlu dGVybmFsOiB0cnVlLiBFbnRpdGllcyBhZmZlY3RlZCA6ICBJRDogNTc5ZGYzYzAtYzc5My00ZjA0 LWE1NTktZjBhZWUwODM1OTA4IFR5cGU6IFZEUwoyMDE2LTA3LTIyIDA5OjIyOjMxLDE4NCBJTkZP ICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmJsbC5Jbml0VmRzT25VcENvbW1hbmRdIChEZWZhdWx0 UXVhcnR6U2NoZWR1bGVyX1dvcmtlci05NCkgWzZhZTljOTRlXSBSdW5uaW5nIGNvbW1hbmQ6IElu aXRWZHNPblVwQ29tbWFuZCBpbnRlcm5hbDogdHJ1ZS4gRW50aXRpZXMgYWZmZWN0ZWQgOiAgSUQ6 IDAwMDAwMDAyLTAwMDItMDAwMi0wMDAyLTAwMDAwMDAwMDEzNCBUeXBlOiBTdG9yYWdlUG9vbAoy MDE2LTA3LTIyIDA5OjIyOjMxLDIxMiBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmJsbC5z dG9yYWdlLkNvbm5lY3RIb3N0VG9TdG9yYWdlUG9vbFNlcnZlcnNDb21tYW5kXSAoRGVmYXVsdFF1 YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFszZjA0YTkzYl0gUnVubmluZyBjb21tYW5kOiBDb25u ZWN0SG9zdFRvU3RvcmFnZVBvb2xTZXJ2ZXJzQ29tbWFuZCBpbnRlcm5hbDogdHJ1ZS4gRW50aXRp ZXMgYWZmZWN0ZWQgOiAgSUQ6IDAwMDAwMDAyLTAwMDItMDAwMi0wMDAyLTAwMDAwMDAwMDEzNCBU eXBlOiBTdG9yYWdlUG9vbAoyMDE2LTA3LTIyIDA5OjIyOjMxLDIzMSBJTkZPICBbb3JnLm92aXJ0 LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9rZXIuQ29ubmVjdFN0b3JhZ2VTZXJ2ZXJWRFND b21tYW5kXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFszZjA0YTkzYl0gU1RB UlQsIENvbm5lY3RTdG9yYWdlU2VydmVyVkRTQ29tbWFuZChIb3N0TmFtZSA9IHBlcnNlcywgU3Rv cmFnZVNlcnZlckNvbm5lY3Rpb25NYW5hZ2VtZW50VkRTUGFyYW1ldGVyczp7cnVuQXN5bmM9J3Ry dWUnLCBob3N0SWQ9JzU3OWRmM2MwLWM3OTMtNGYwNC1hNTU5LWYwYWVlMDgzNTkwOCcsIHN0b3Jh Z2VQb29sSWQ9JzAwMDAwMDAyLTAwMDItMDAwMi0wMDAyLTAwMDAwMDAwMDEzNCcsIHN0b3JhZ2VU eXBlPSdORlMnLCBjb25uZWN0aW9uTGlzdD0nW1N0b3JhZ2VTZXJ2ZXJDb25uZWN0aW9uczp7aWQ9 JzZjYmEwNDM4LWUxNWQtNDc5MS1hYTRhLTdlYjVkMGY5ZmIwOCcsIGNvbm5lY3Rpb249J292aXJ0 LW5mcy5uZXRzZWM6L292aXJ0L3N0b3JhZ2UnLCBpcW49J251bGwnLCB2ZnNUeXBlPSdudWxsJywg bW91bnRPcHRpb25zPSdudWxsJywgbmZzVmVyc2lvbj0nVjQnLCBuZnNSZXRyYW5zPSdudWxsJywg bmZzVGltZW89J251bGwnLCBpZmFjZT0nbnVsbCcsIG5ldElmYWNlTmFtZT0nbnVsbCd9LCBTdG9y YWdlU2VydmVyQ29ubmVjdGlvbnM6e2lkPScxYTQ5NGI2NC1iNmQ0LTRjNGUtOTM2ZC0xZjNjNTlk MTc4NTEnLCBjb25uZWN0aW9uPSdvdmlydC1uZnMubmV0c2VjOi9vdmlydC9pc28nLCBpcW49J251 bGwnLCB2ZnNUeXBlPSdudWxsJywgbW91bnRPcHRpb25zPSdudWxsJywgbmZzVmVyc2lvbj0nVjQn LCBuZnNSZXRyYW5zPSdudWxsJywgbmZzVGltZW89J251bGwnLCBpZmFjZT0nbnVsbCcsIG5ldElm YWNlTmFtZT0nbnVsbCd9LCBTdG9yYWdlU2VydmVyQ29ubmVjdGlvbnM6e2lkPSc2YzUyN2E1Mi05 Y2ViLTRkYTEtODdiNi0yNWE2NTYyMTQwNjYnLCBjb25uZWN0aW9uPSdvdmlydC1uZnMubmV0c2Vj Oi9vdmlydC9leHBvcnQnLCBpcW49J251bGwnLCB2ZnNUeXBlPSdudWxsJywgbW91bnRPcHRpb25z PSdudWxsJywgbmZzVmVyc2lvbj0nVjQnLCBuZnNSZXRyYW5zPSdudWxsJywgbmZzVGltZW89J251 bGwnLCBpZmFjZT0nbnVsbCcsIG5ldElmYWNlTmFtZT0nbnVsbCd9LCBTdG9yYWdlU2VydmVyQ29u bmVjdGlvbnM6e2lkPSc4Njc5YjA3Ny0zZGUwLTRkOTctYjY4OC1mNDljNjk4OGYwY2EnLCBjb25u ZWN0aW9uPSdvdmlydC1uZnMubmV0c2VjOi9vdmlydC9ob3N0ZWQtZW5naW5lJywgaXFuPSdudWxs JywgdmZzVHlwZT0nbnVsbCcsIG1vdW50T3B0aW9ucz0nbnVsbCcsIG5mc1ZlcnNpb249J251bGwn LCBuZnNSZXRyYW5zPSdudWxsJywgbmZzVGltZW89J251bGwnLCBpZmFjZT0nbnVsbCcsIG5ldElm YWNlTmFtZT0nbnVsbCd9XSd9KSwgbG9nIGlkOiAzMDUwYzZmMQoyMDE2LTA3LTIyIDA5OjIyOjMx LDkyNSBFUlJPUiBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmRhbC5kYmJyb2tlci5hdWRpdGxvZ2hh bmRsaW5nLkF1ZGl0TG9nRGlyZWN0b3JdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci05 NCkgWzNmMDRhOTNiXSBDb3JyZWxhdGlvbiBJRDogbnVsbCwgQ2FsbCBTdGFjazogbnVsbCwgQ3Vz dG9tIEV2ZW50IElEOiAtMSwgTWVzc2FnZTogRmFpbGVkIHRvIGNvbm5lY3QgSG9zdCBwZXJzZXMg dG8gdGhlIFN0b3JhZ2UgRG9tYWlucyBob3N0ZWRfc3RvcmFnZS4KMjAxNi0wNy0yMiAwOToyMjoz MSw5MjUgSU5GTyAgW29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIudmRzYnJva2VyLkNv bm5lY3RTdG9yYWdlU2VydmVyVkRTQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29y a2VyLTk0KSBbM2YwNGE5M2JdIEZJTklTSCwgQ29ubmVjdFN0b3JhZ2VTZXJ2ZXJWRFNDb21tYW5k LCByZXR1cm46IHs2YzUyN2E1Mi05Y2ViLTRkYTEtODdiNi0yNWE2NTYyMTQwNjY9MCwgODY3OWIw NzctM2RlMC00ZDk3LWI2ODgtZjQ5YzY5ODhmMGNhPTQ3NywgNmNiYTA0MzgtZTE1ZC00NzkxLWFh NGEtN2ViNWQwZjlmYjA4PTAsIDFhNDk0YjY0LWI2ZDQtNGM0ZS05MzZkLTFmM2M1OWQxNzg1MT0w fSwgbG9nIGlkOiAzMDUwYzZmMQoyMDE2LTA3LTIyIDA5OjIyOjMxLDkzMyBFUlJPUiBbb3JnLm92 aXJ0LmVuZ2luZS5jb3JlLmRhbC5kYmJyb2tlci5hdWRpdGxvZ2hhbmRsaW5nLkF1ZGl0TG9nRGly ZWN0b3JdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci05NCkgWzNmMDRhOTNiXSBDb3Jy ZWxhdGlvbiBJRDogbnVsbCwgQ2FsbCBTdGFjazogbnVsbCwgQ3VzdG9tIEV2ZW50IElEOiAtMSwg TWVzc2FnZTogVGhlIGVycm9yIG1lc3NhZ2UgZm9yIGNvbm5lY3Rpb24gb3ZpcnQtbmZzLm5ldHNl Yzovb3ZpcnQvaG9zdGVkLWVuZ2luZSByZXR1cm5lZCBieSBWRFNNIHdhczogUHJvYmxlbSB3aGls ZSB0cnlpbmcgdG8gbW91bnQgdGFyZ2V0CjIwMTYtMDctMjIgMDk6MjI6MzEsOTM0IEVSUk9SIFtv cmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLnN0b3JhZ2UuQmFzZUZzU3RvcmFnZUhlbHBlcl0gKERl ZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTk0KSBbM2YwNGE5M2JdIFRoZSBjb25uZWN0aW9u IHdpdGggZGV0YWlscyAnb3ZpcnQtbmZzLm5ldHNlYzovb3ZpcnQvaG9zdGVkLWVuZ2luZScgZmFp bGVkIGJlY2F1c2Ugb2YgZXJyb3IgY29kZSAnNDc3JyBhbmQgZXJyb3IgbWVzc2FnZSBpczogcHJv YmxlbSB3aGlsZSB0cnlpbmcgdG8gbW91bnQgdGFyZ2V0CjIwMTYtMDctMjIgMDk6MjI6MzEsOTQ0 IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLnN0b3JhZ2UuQ29ubmVjdEhvc3RUb1N0 b3JhZ2VQb29sU2VydmVyc0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci05 NCkgWzNmMDRhOTNiXSBIb3N0ICdwZXJzZXMnIHN0b3JhZ2UgY29ubmVjdGlvbiB3YXMgZmFpbGVk IAoyMDE2LTA3LTIyIDA5OjIyOjMxLDk1OCBXQVJOICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLmRh bC5kYmJyb2tlci5hdWRpdGxvZ2hhbmRsaW5nLkF1ZGl0TG9nRGlyZWN0b3JdIChEZWZhdWx0UXVh cnR6U2NoZWR1bGVyX1dvcmtlci05NCkgWzNmMDRhOTNiXSBDb3JyZWxhdGlvbiBJRDogM2YwNGE5 M2IsIENhbGwgU3RhY2s6IG51bGwsIEN1c3RvbSBFdmVudCBJRDogLTEsIE1lc3NhZ2U6IEZhaWxl ZCB0byBjb25uZWN0IEhvc3QgcGVyc2VzIHRvIFN0b3JhZ2UgU2VydmVycwoyMDE2LTA3LTIyIDA5 OjIyOjMxLDk2NyBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9r ZXIuQ29ubmVjdFN0b3JhZ2VQb29sVkRTQ29tbWFuZF0gKG9yZy5vdmlydC50aHJlYWQucG9vbC04 LXRocmVhZC0zNykgW10gU1RBUlQsIENvbm5lY3RTdG9yYWdlUG9vbFZEU0NvbW1hbmQoSG9zdE5h bWUgPSBwZXJzZXMsIENvbm5lY3RTdG9yYWdlUG9vbFZEU0NvbW1hbmRQYXJhbWV0ZXJzOntydW5B c3luYz0ndHJ1ZScsIGhvc3RJZD0nNTc5ZGYzYzAtYzc5My00ZjA0LWE1NTktZjBhZWUwODM1OTA4 JywgdmRzSWQ9JzU3OWRmM2MwLWM3OTMtNGYwNC1hNTU5LWYwYWVlMDgzNTkwOCcsIHN0b3JhZ2VQ b29sSWQ9JzAwMDAwMDAyLTAwMDItMDAwMi0wMDAyLTAwMDAwMDAwMDEzNCcsIG1hc3RlclZlcnNp b249JzEnfSksIGxvZyBpZDogN2M3ZTY1NjgKMjAxNi0wNy0yMiAwOToyMjozMiw3MzIgSU5GTyAg W29yZy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIudmRzYnJva2VyLkNvbm5lY3RTdG9yYWdl UG9vbFZEU0NvbW1hbmRdIChvcmcub3ZpcnQudGhyZWFkLnBvb2wtOC10aHJlYWQtMzcpIFtdIEZJ TklTSCwgQ29ubmVjdFN0b3JhZ2VQb29sVkRTQ29tbWFuZCwgbG9nIGlkOiA3YzdlNjU2OAoyMDE2 LTA3LTIyIDA5OjIyOjMyLDgxNCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tl ci52ZHNicm9rZXIuRGVzdHJveVZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dv cmtlci03NikgWzc0YzU1ZGU5XSBTVEFSVCwgRGVzdHJveVZEU0NvbW1hbmQoSG9zdE5hbWUgPSBw ZXJzZXMsIERlc3Ryb3lWbVZEU0NvbW1hbmRQYXJhbWV0ZXJzOntydW5Bc3luYz0ndHJ1ZScsIGhv c3RJZD0nNTc5ZGYzYzAtYzc5My00ZjA0LWE1NTktZjBhZWUwODM1OTA4Jywgdm1JZD0nYTZhMDhh NDktMWRiYS00ZjcyLWI5ZjYtYWVhN2ZkNjIzOGVlJywgZm9yY2U9J2ZhbHNlJywgc2Vjb25kc1Rv V2FpdD0nMCcsIGdyYWNlZnVsbHk9J2ZhbHNlJywgcmVhc29uPScnLCBpZ25vcmVOb1ZtPSd0cnVl J30pLCBsb2cgaWQ6IDZhMmRhZTg1CjIwMTYtMDctMjIgMDk6MjI6MzIsODUwIEVSUk9SIFtvcmcu b3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLmlyc2Jyb2tlci5JcnNQcm94eURhdGFdIChvcmcu b3ZpcnQudGhyZWFkLnBvb2wtOC10aHJlYWQtMzcpIFtdIERvbWFpbiAnMmRhYmEwYWItMmIzZC00 MDI2LWJjZmMtMWNkMDcxYzMwMDM4Omhvc3RlZF9zdG9yYWdlJyB3YXMgcmVwb3J0ZWQgd2l0aCBl cnJvciBjb2RlICczNTgnCjIwMTYtMDctMjIgMDk6MjI6MzIsODUxIEVSUk9SIFtvcmcub3ZpcnQu ZW5naW5lLmNvcmUuYmxsLkluaXRWZHNPblVwQ29tbWFuZF0gKG9yZy5vdmlydC50aHJlYWQucG9v bC04LXRocmVhZC0zNykgW10gU3RvcmFnZSBEb21haW4gJ2hvc3RlZF9zdG9yYWdlJyBvZiBwb29s ICdEZWZhdWx0JyBpcyBpbiBwcm9ibGVtIGluIGhvc3QgJ3BlcnNlcycKMjAxNi0wNy0yMiAwOToy MjozMiw4OTIgRVJST1IgW29yZy5vdmlydC5lbmdpbmUuY29yZS5kYWwuZGJicm9rZXIuYXVkaXRs b2doYW5kbGluZy5BdWRpdExvZ0RpcmVjdG9yXSAob3JnLm92aXJ0LnRocmVhZC5wb29sLTgtdGhy ZWFkLTM3KSBbXSBDb3JyZWxhdGlvbiBJRDogbnVsbCwgQ2FsbCBTdGFjazogbnVsbCwgQ3VzdG9t IEV2ZW50IElEOiAtMSwgTWVzc2FnZTogSG9zdCBwZXJzZXMgcmVwb3J0cyBhYm91dCBvbmUgb2Yg dGhlIEFjdGl2ZSBTdG9yYWdlIERvbWFpbnMgYXMgUHJvYmxlbWF0aWMuCjIwMTYtMDctMjIgMDk6 MjI6MzIsOTQ5IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUuYmxsLlNldE5vbk9wZXJhdGlv bmFsVmRzQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTk0KSBbMzFhNTEy MjhdIFJ1bm5pbmcgY29tbWFuZDogU2V0Tm9uT3BlcmF0aW9uYWxWZHNDb21tYW5kIGludGVybmFs OiB0cnVlLiBFbnRpdGllcyBhZmZlY3RlZCA6ICBJRDogNTc5ZGYzYzAtYzc5My00ZjA0LWE1NTkt ZjBhZWUwODM1OTA4IFR5cGU6IFZEUwoyMDE2LTA3LTIyIDA5OjIyOjMyLDk1MyBJTkZPICBbb3Jn Lm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci5TZXRWZHNTdGF0dXNWRFNDb21tYW5kXSAoRGVm YXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFszMWE1MTIyOF0gU1RBUlQsIFNldFZkc1N0 YXR1c1ZEU0NvbW1hbmQoSG9zdE5hbWUgPSBwZXJzZXMsIFNldFZkc1N0YXR1c1ZEU0NvbW1hbmRQ YXJhbWV0ZXJzOntydW5Bc3luYz0ndHJ1ZScsIGhvc3RJZD0nNTc5ZGYzYzAtYzc5My00ZjA0LWE1 NTktZjBhZWUwODM1OTA4Jywgc3RhdHVzPSdOb25PcGVyYXRpb25hbCcsIG5vbk9wZXJhdGlvbmFs UmVhc29uPSdTVE9SQUdFX0RPTUFJTl9VTlJFQUNIQUJMRScsIHN0b3BTcG1GYWlsdXJlTG9nZ2Vk PSdmYWxzZScsIG1haW50ZW5hbmNlUmVhc29uPSdudWxsJ30pLCBsb2cgaWQ6IDYyYjhjZWU1CjIw MTYtMDctMjIgMDk6MjI6MzIsOTY3IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJv a2VyLlNldFZkc1N0YXR1c1ZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtl ci05NCkgWzMxYTUxMjI4XSBGSU5JU0gsIFNldFZkc1N0YXR1c1ZEU0NvbW1hbmQsIGxvZyBpZDog NjJiOGNlZTUKMjAxNi0wNy0yMiAwOToyMjozMiw5OTIgV0FSTiAgW29yZy5vdmlydC5lbmdpbmUu Y29yZS5kYWwuZGJicm9rZXIuYXVkaXRsb2doYW5kbGluZy5BdWRpdExvZ0RpcmVjdG9yXSAoRGVm YXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFszMWE1MTIyOF0gQ29ycmVsYXRpb24gSUQ6 IDMxYTUxMjI4LCBKb2IgSUQ6IDM0MmM5Y2ZjLTQ2NjQtNDAwNi05Y2U3LThlOGY0OWIyNTQyMCwg Q2FsbCBTdGFjazogbnVsbCwgQ3VzdG9tIEV2ZW50IElEOiAtMSwgTWVzc2FnZTogSG9zdCBwZXJz ZXMgY2Fubm90IGFjY2VzcyB0aGUgU3RvcmFnZSBEb21haW4ocykgaG9zdGVkX3N0b3JhZ2UgYXR0 YWNoZWQgdG8gdGhlIERhdGEgQ2VudGVyIERlZmF1bHQuIFNldHRpbmcgSG9zdCBzdGF0ZSB0byBO b24tT3BlcmF0aW9uYWwuCjIwMTYtMDctMjIgMDk6MjI6MzMsMDEwIFdBUk4gIFtvcmcub3ZpcnQu ZW5naW5lLmNvcmUuZGFsLmRiYnJva2VyLmF1ZGl0bG9naGFuZGxpbmcuQXVkaXRMb2dEaXJlY3Rv cl0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29ya2VyLTk0KSBbMzFhNTEyMjhdIENvcnJlbGF0 aW9uIElEOiBudWxsLCBDYWxsIFN0YWNrOiBudWxsLCBDdXN0b20gRXZlbnQgSUQ6IC0xLCBNZXNz YWdlOiBGYWlsZWQgdG8gdmVyaWZ5IFBvd2VyIE1hbmFnZW1lbnQgY29uZmlndXJhdGlvbiBmb3Ig SG9zdCBwZXJzZXMuCjIwMTYtMDctMjIgMDk6MjI6MzMsMDIxIElORk8gIFtvcmcub3ZpcnQuZW5n aW5lLmNvcmUudmRzYnJva2VyLnZkc2Jyb2tlci5EZXN0cm95VkRTQ29tbWFuZF0gKERlZmF1bHRR dWFydHpTY2hlZHVsZXJfV29ya2VyLTc2KSBbNzRjNTVkZTldIEZJTklTSCwgRGVzdHJveVZEU0Nv bW1hbmQsIGxvZyBpZDogNmEyZGFlODUKMjAxNi0wNy0yMiAwOToyMjozMywwMzEgV0FSTiAgW29y Zy5vdmlydC5lbmdpbmUuY29yZS5kYWwuZGJicm9rZXIuYXVkaXRsb2doYW5kbGluZy5BdWRpdExv Z0RpcmVjdG9yXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFszMWE1MTIyOF0g Q29ycmVsYXRpb24gSUQ6IDZhZTljOTRlLCBDYWxsIFN0YWNrOiBudWxsLCBDdXN0b20gRXZlbnQg SUQ6IC0xLCBNZXNzYWdlOiBGYWlsZWQgdG8gY29ubmVjdCBIb3N0IHBlcnNlcyB0byBTdG9yYWdl IFBvb2wgRGVmYXVsdAoyMDE2LTA3LTIyIDA5OjIyOjMzLDAzNSBJTkZPICBbb3JnLm92aXJ0LmVu Z2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9rZXIuRGVzdHJveVZEU0NvbW1hbmRdIChEZWZhdWx0 UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03NikgWzc0YzU1ZGU5XSBTVEFSVCwgRGVzdHJveVZEU0Nv bW1hbmQoSG9zdE5hbWUgPSBwZXJzZXMsIERlc3Ryb3lWbVZEU0NvbW1hbmRQYXJhbWV0ZXJzOnty dW5Bc3luYz0ndHJ1ZScsIGhvc3RJZD0nNTc5ZGYzYzAtYzc5My00ZjA0LWE1NTktZjBhZWUwODM1 OTA4Jywgdm1JZD0nNDI3YzcyZTUtMWFiYi00Nzc0LTk4OWUtMDVlYjI3OTNmOTU1JywgZm9yY2U9 J2ZhbHNlJywgc2Vjb25kc1RvV2FpdD0nMCcsIGdyYWNlZnVsbHk9J2ZhbHNlJywgcmVhc29uPScn LCBpZ25vcmVOb1ZtPSd0cnVlJ30pLCBsb2cgaWQ6IDc3MmRjZTgxCjIwMTYtMDctMjIgMDk6MjI6 MzMsMDQyIElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLlZtc1N0YXRpc3Rp Y3NGZXRjaGVyXSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNDQpIFs0ZWQ3MjFjN10g RmV0Y2hlZCAyIFZNcyBmcm9tIFZEUyAnM2U4YjUxY2MtYzM1OC00YzVjLWI3ZTMtMjY1OGQ0NzNj NzZhJwoyMDE2LTA3LTIyIDA5OjIyOjMzLDA3MSBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3Jl LmJsbC5IYW5kbGVWZHNWZXJzaW9uQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJfV29y a2VyLTk0KSBbNzcxMzU3MThdIFJ1bm5pbmcgY29tbWFuZDogSGFuZGxlVmRzVmVyc2lvbkNvbW1h bmQgaW50ZXJuYWw6IHRydWUuIEVudGl0aWVzIGFmZmVjdGVkIDogIElEOiA1NzlkZjNjMC1jNzkz LTRmMDQtYTU1OS1mMGFlZTA4MzU5MDggVHlwZTogVkRTCjIwMTYtMDctMjIgMDk6MjI6MzMsMDc1 IElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJva2VyLkhvc3RNb25pdG9yaW5nXSAo RGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItOTQpIFs3NzEzNTcxOF0gSG9zdCAncGVyc2Vz Jyg1NzlkZjNjMC1jNzkzLTRmMDQtYTU1OS1mMGFlZTA4MzU5MDgpIGlzIGFscmVhZHkgaW4gTm9u T3BlcmF0aW9uYWwgc3RhdHVzIGZvciByZWFzb24gJ1NUT1JBR0VfRE9NQUlOX1VOUkVBQ0hBQkxF Jy4gU2V0Tm9uT3BlcmF0aW9uYWxWZHMgY29tbWFuZCBpcyBza2lwcGVkLgoyMDE2LTA3LTIyIDA5 OjIyOjMzLDA5NyBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9r ZXIuRGVzdHJveVZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03Nikg Wzc0YzU1ZGU5XSBGSU5JU0gsIERlc3Ryb3lWRFNDb21tYW5kLCBsb2cgaWQ6IDc3MmRjZTgxCjIw MTYtMDctMjIgMDk6MjI6MzMsMTExIElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUudmRzYnJv a2VyLnZkc2Jyb2tlci5EZXN0cm95VkRTQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hlZHVsZXJf V29ya2VyLTc2KSBbNzRjNTVkZTldIFNUQVJULCBEZXN0cm95VkRTQ29tbWFuZChIb3N0TmFtZSA9 IHBlcnNlcywgRGVzdHJveVZtVkRTQ29tbWFuZFBhcmFtZXRlcnM6e3J1bkFzeW5jPSd0cnVlJywg aG9zdElkPSc1NzlkZjNjMC1jNzkzLTRmMDQtYTU1OS1mMGFlZTA4MzU5MDgnLCB2bUlkPScyOGFj YzgwOS00ZjczLTQ1MWUtOTRiMS1hOWNhMmVhYTU4MDEnLCBmb3JjZT0nZmFsc2UnLCBzZWNvbmRz VG9XYWl0PScwJywgZ3JhY2VmdWxseT0nZmFsc2UnLCByZWFzb249JycsIGlnbm9yZU5vVm09J3Ry dWUnfSksIGxvZyBpZDogNjA4ZGM5NDEKMjAxNi0wNy0yMiAwOToyMjozMyw2NzcgSU5GTyAgW29y Zy5vdmlydC5lbmdpbmUuY29yZS52ZHNicm9rZXIudmRzYnJva2VyLkRlc3Ryb3lWRFNDb21tYW5k XSAoRGVmYXVsdFF1YXJ0elNjaGVkdWxlcl9Xb3JrZXItNzYpIFs3NGM1NWRlOV0gRklOSVNILCBE ZXN0cm95VkRTQ29tbWFuZCwgbG9nIGlkOiA2MDhkYzk0MQoyMDE2LTA3LTIyIDA5OjIyOjMzLDcy NCBJTkZPICBbb3JnLm92aXJ0LmVuZ2luZS5jb3JlLnZkc2Jyb2tlci52ZHNicm9rZXIuRGVzdHJv eVZEU0NvbW1hbmRdIChEZWZhdWx0UXVhcnR6U2NoZWR1bGVyX1dvcmtlci03NikgWzc0YzU1ZGU5 XSBTVEFSVCwgRGVzdHJveVZEU0NvbW1hbmQoSG9zdE5hbWUgPSBwZXJzZXMsIERlc3Ryb3lWbVZE U0NvbW1hbmRQYXJhbWV0ZXJzOntydW5Bc3luYz0ndHJ1ZScsIGhvc3RJZD0nNTc5ZGYzYzAtYzc5 My00ZjA0LWE1NTktZjBhZWUwODM1OTA4Jywgdm1JZD0nNmE0NDcyZmMtMzA0Zi00NWNlLWJlNTEt NzQ4YjcxY2YyMzYxJywgZm9yY2U9J2ZhbHNlJywgc2Vjb25kc1RvV2FpdD0nMCcsIGdyYWNlZnVs bHk9J2ZhbHNlJywgcmVhc29uPScnLCBpZ25vcmVOb1ZtPSd0cnVlJ30pLCBsb2cgaWQ6IDVhZDcx Y2M4CjIwMTYtMDctMjIgMDk6MjI6MzQsNzUxIElORk8gIFtvcmcub3ZpcnQuZW5naW5lLmNvcmUu dmRzYnJva2VyLnZkc2Jyb2tlci5EZXN0cm95VkRTQ29tbWFuZF0gKERlZmF1bHRRdWFydHpTY2hl ZHVsZXJfV29ya2VyLTc2KSBbNzRjNTVkZTldIEZJTklTSCwgRGVzdHJveVZEU0NvbW1hbmQsIGxv ZyBpZDogNWFkNzFjYzgK --MP_/bIQEAQ/NhL84eB5JxRKZOmO-- --Sig_/YOnKOzEqBUm8Rn2KNWH9.d_ Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAleSJAAACgkQ7/fVLLY1mngZ3wCeJ0rHNqB6/YIPrqYzkMZTOTQn eV4AoJrEIMhU+g48yw8KxUR4Mc5kK8+Q =OoqL -----END PGP SIGNATURE----- --Sig_/YOnKOzEqBUm8Rn2KNWH9.d_--
On Fri, Jul 22, 2016 at 3:47 PM, Robert Story <rstory@tislabs.com> wrote:
Hello,
I'm in the process of upgrading from 3.5.x to 3.6.x. My hosted engine and hosts in the primary cluster are all upgraded and appear to be running fine.
I have a second cluster of 2 machines which are just regular hosts, without the hosted-engine. Both have been marked non-operational, with the following messages logged about every 5 minutes:
Failed to connect Host perses to Storage Pool Default
Host perses cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational.
Host perses reports about one of the Active Storage Domains as Problematic.
Failed to connect Host perses to Storage Servers
Failed to connect Host perses to the Storage Domains hosted_storage.
I could see the normal storage/iso/export domains mounted on the host, and the VMs running on the host are fine.
In 3.5 only the hosts involved in hosted-engine have to access the hosted-engine storage domain. With 3.6 we introduced the capabilities to manage the engine VM from the engine itself so the engine has to import in the hosted-engine storage domain. This means that all the hosts in the datacenter that contains the cluster with the hosted-engine hosts have now to be able to connect the hosted-engine storage domain. Can you please check the ACL on the storage server (NFS or iSCSI) that you use to expose the hosted-engine storage domain?
I shut down the VMs on one host, put it in maintenance mode, installed 3.6 repo and ran yum update. All went well, but when I activated the host, same deal.
I've attached the engine log snippet for the activation attempt.
Robert
-- Senior Software Engineer @ Parsons
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
This is a multi-part message in MIME format. --------------5AA8F5EA1CB77C22C605199C Content-Type: text/plain; charset=windows-1252 Content-Transfer-Encoding: 8bit Hello, I also see from the logs that all your Storage-Domains that work are mounted as nfsVersion='V4' but ovirt-nfs.netsec:/ovirt/hosted-engine is mounted as nfsVersion='null'. Bye Am 22.07.2016 um 16:17 schrieb Simone Tiraboschi:
Hello,
I'm in the process of upgrading from 3.5.x to 3.6.x. My hosted engine and hosts in the primary cluster are all upgraded and appear to be running fine.
I have a second cluster of 2 machines which are just regular hosts, without the hosted-engine. Both have been marked non-operational, with the following messages logged about every 5 minutes:
Failed to connect Host perses to Storage Pool Default
Host perses cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational.
Host perses reports about one of the Active Storage Domains as Problematic.
Failed to connect Host perses to Storage Servers
Failed to connect Host perses to the Storage Domains hosted_storage.
I could see the normal storage/iso/export domains mounted on the host, and the VMs running on the host are fine. In 3.5 only the hosts involved in hosted-engine have to access the hosted-engine storage domain. With 3.6 we introduced the capabilities to manage the engine VM from
On Fri, Jul 22, 2016 at 3:47 PM, Robert Story <rstory@tislabs.com> wrote: the engine itself so the engine has to import in the hosted-engine storage domain. This means that all the hosts in the datacenter that contains the cluster with the hosted-engine hosts have now to be able to connect the hosted-engine storage domain.
Can you please check the ACL on the storage server (NFS or iSCSI) that you use to expose the hosted-engine storage domain?
I shut down the VMs on one host, put it in maintenance mode, installed 3.6 repo and ran yum update. All went well, but when I activated the host, same deal.
I've attached the engine log snippet for the activation attempt.
Robert
-- Senior Software Engineer @ Parsons
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
-- *Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <mailto:rs@databay.de> *Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de> Sitz/Amtsgericht Aachen HRB:8437 USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Wilhelm Dohmen ------------------------------------------------------------------------ --------------5AA8F5EA1CB77C22C605199C Content-Type: multipart/related; boundary="------------A8A08E7B1C43AB7B2AA6FD78" --------------A8A08E7B1C43AB7B2AA6FD78 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p>Hello,</p> <p>I also see from the logs that all your Storage-Domains that work are mounted as nfsVersion='V4' but ovirt-nfs.netsec:/ovirt/hosted-engine is mounted as nfsVersion='null'.</p> <p>Bye<br> </p> <br> <div class="moz-cite-prefix">Am 22.07.2016 um 16:17 schrieb Simone Tiraboschi:<br> </div> <blockquote cite="mid:CAN8-ONoS0cGbN3BBib32sUgEmiaCKg-_J4HJiTPDvJSjQ4RCyw@mail.gmail.com" type="cite"> <pre wrap="">On Fri, Jul 22, 2016 at 3:47 PM, Robert Story <a class="moz-txt-link-rfc2396E" href="mailto:rstory@tislabs.com"><rstory@tislabs.com></a> wrote: </pre> <blockquote type="cite"> <pre wrap="">Hello, I'm in the process of upgrading from 3.5.x to 3.6.x. My hosted engine and hosts in the primary cluster are all upgraded and appear to be running fine. I have a second cluster of 2 machines which are just regular hosts, without the hosted-engine. Both have been marked non-operational, with the following messages logged about every 5 minutes: Failed to connect Host perses to Storage Pool Default Host perses cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational. Host perses reports about one of the Active Storage Domains as Problematic. Failed to connect Host perses to Storage Servers Failed to connect Host perses to the Storage Domains hosted_storage. I could see the normal storage/iso/export domains mounted on the host, and the VMs running on the host are fine. </pre> </blockquote> <pre wrap=""> In 3.5 only the hosts involved in hosted-engine have to access the hosted-engine storage domain. With 3.6 we introduced the capabilities to manage the engine VM from the engine itself so the engine has to import in the hosted-engine storage domain. This means that all the hosts in the datacenter that contains the cluster with the hosted-engine hosts have now to be able to connect the hosted-engine storage domain. Can you please check the ACL on the storage server (NFS or iSCSI) that you use to expose the hosted-engine storage domain? </pre> <blockquote type="cite"> <pre wrap="">I shut down the VMs on one host, put it in maintenance mode, installed 3.6 repo and ran yum update. All went well, but when I activated the host, same deal. I've attached the engine log snippet for the activation attempt. Robert -- Senior Software Engineer @ Parsons _______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> <div class="moz-signature">-- <br> <p> </p> <table border="0" cellpadding="0" cellspacing="0"> <tbody> <tr> <td colspan="3"><img src="cid:part1.28604224.8B5314B3@databay.de" height="30" border="0" width="151"></td> </tr> <tr> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Ralf Schenk</b><br> fon +49 (0) 24 05 / 40 83 70<br> fax +49 (0) 24 05 / 40 83 759<br> mail <a href="mailto:rs@databay.de"><font color="#FF0000"><b>rs@databay.de</b></font></a><br> </font> </td> <td width="30"> </td> <td valign="top"> <font face="Verdana, Arial, sans-serif" size="-1"><br> <b>Databay AG</b><br> Jens-Otto-Krag-Straße 11<br> D-52146 Würselen<br> <a href="http://www.databay.de"><font color="#FF0000"><b>www.databay.de</b></font></a> </font> </td> </tr> <tr> <td colspan="3" valign="top"> <font face="Verdana, Arial, sans-serif" size="1"><br> Sitz/Amtsgericht Aachen HRB:8437 USt-IdNr.: DE 210844202<br> Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns<br> Aufsichtsratsvorsitzender: Wilhelm Dohmen </font> </td> </tr> </tbody> </table> <hr color="#000000" noshade="noshade" size="1" width="100%"> </div> </body> </html> --------------A8A08E7B1C43AB7B2AA6FD78 Content-Type: image/gif; name="logo_databay_email.gif" Content-Transfer-Encoding: base64 Content-ID: <part1.28604224.8B5314B3@databay.de> Content-Disposition: inline; filename="logo_databay_email.gif" R0lGODlhlwAeAMQAAObm5v9QVf/R0oKBgfDw8NfX105MTLi3t/r6+sfHx/+rrf98gC0sLP8L EhIQEKalpf/g4ZmYmHd2dmppaf8uNP/y8v8cIv+Ym//AwkE/P46NjRwbG11cXP8ABwUDA/// /yH5BAAAAAAALAAAAACXAB4AAAX/4CeOYnUJZKqubOu+cCzPNA0tVnfVfO//wGAKk+t0Ap+K QMFUYCDCqHRKJVUWDaPRUsFktZ1G4AKtms9o1gKsFVS+7I5ll67bpd647hPQawNld4KDMQJF bA07F35aFBiEkJEpfXEBjx8KjI0Vkp2DEIdaCySgFBShbEgrCQOtrq+uEQcALQewrQUjEbe8 rgkkD7y5KhMZB3drqSoVFQhdlHGXKQYe1dbX2BvHKwzY1RMiAN7j1xEjBeTmKeIeD3cYCxRf FigvChRxFJwkBBvk5A7cpZhAjgGCDwn+kfslgto4CSoSehh2BwEEBQvowDAUR0EKdArHZTg4 4oDCXBFC/3qj9SEluZEpHnjYQFIGgpo1KgSasYjNKBImrzF4NaFbNgIjCGRQeIyVKwneOLzS cLCAg38OWI4Y4GECgQcSOEwYcADnh6/FNjAwoGFYAQ0atI4AAFeEFwsLFLiJUQEfGH0kNGAD x8+oNQdIRQg+7NCaOhIgD8sVgYADNsPVGI5YWjRqzQTdHDDIYHRDLokaUhCglkFEJi0NKJhl 0RP2TsvXUg88KiLBVWsZrF6DmMKlNYMqglqTik1guN8OBgAgkGCpB+L9ugK4iSCBvwEfECw1 kILrBpa1jVCQIQBRvbP+rlEcQVAoSevWyv6uhpwE12uEkQAAZucpVw1xIsjkgf8B863mQVYt eQATCZYJZJ5WBfij2wfpHcEeHGG8Z+BMszVWDXkfKLhceJhBSAJ+1ThH32AfRFZNayNAtUFi wFSTSwEHJIYAAQU84IADwyjIEALU9MchG+vFgIF7W2GDI2T7HfjBgNcgKQKMHmwjgnCSpeCb ULRkdxhF1CDY40RjgmUAA/v1J5FAKW2gGSZscBFDMraNgJs1AYpAAGYP5jJoNQ4Y4Gh8jpFg HH9mgbmWo1l6oA4C3Ygp6UwEIFBfNRtkMIBlKMLnAXgAXLWhXXH85EIFqMhGGZgDEKArABGA ed0HI4bk5qgnprCYSt88B6dqS0FEEAMPJDCdCJYViur/B1BlwGMJqDTwnhqxJgUpo0ceOQ4D 0yEakpMm/jqCRMgWm2I1j824Y6vLvuuPjHnqOJkIgP6xzwp5sCFNsCFp88Gxh11lrjfDcNrc CEx64/CD3iAHlQcMUEQXvcA+qBkBB4Q2X1CusjBlJdKMYAKI6g28MbKN5hJsBAXknHOwutn4 oFYqkpqAzjnPbE0u1PxmwAQGXLWBbvhuIIEGEnRjlAHO4SvhbCNAkwoGzEBwgV9U0lfu2WiX OkDEGaCdKgl0nk2YkWdPOCDabvaGdkAftL1LlgwCM+7Tq11V71IO7LkM2XE0YAHMYMhqqK6U V165CpaHukLmiXFO8XSVzzakX+UH6TrmAajPNxfqByTQec41AeBPvSwIALkmAnuiexCsca3C BajgfsROuxcPA8kHQJX4DAIwjnsAvhsvfXHWKEwDAljg7sj03L9wwAQTxOWD2AE0YP75eCkw cPfs+xACADs= --------------A8A08E7B1C43AB7B2AA6FD78-- --------------5AA8F5EA1CB77C22C605199C--
On Fri, Jul 22, 2016 at 4:48 PM, Ralf Schenk <rs@databay.de> wrote:
Hello,
I also see from the logs that all your Storage-Domains that work are mounted as nfsVersion='V4' but ovirt-nfs.netsec:/ovirt/hosted-engine is mounted as nfsVersion='null'.
Hi Robert, unfortunately Ralf is right: I reproduced the issue. The auto-import procedure for the hosted-engine storage domain ignores the nfsVersion parameter and so we don't have a value for that in the engine DB. On hosted-engine hosts, the agent mounts the hosted-engine storage domain before the engine and so everything is fine since the agent knows that it's nfsv4. The issue comes with the hosts of that datacenter not involved in hosted-engine: in this case the engine simply tries to mount without the nfsVersion parameter and so, if the NFS server cannot be access over nfsv3, the mount could fail and the host will be declared as not operation. I opened a ticket to track it: https://bugzilla.redhat.com/show_bug.cgi?id=1359265 If you need a quick fix you can: - fix the configuration of your storage server to allow it to be accessed also over nfsv3 - edit the configuration of the storage connection in the engine DB on the engine VM to add the missing parameter. Something like: # sudo -u postgresl psql \c engine; select * from storage_server_connections; UPDATE storage_server_connections SET nfs_version = '4' WHERE connection = 'ovirt-nfs.netsec:/ovirt/hosted-engine'; commit; select * from storage_server_connections;
Bye
Am 22.07.2016 um 16:17 schrieb Simone Tiraboschi:
On Fri, Jul 22, 2016 at 3:47 PM, Robert Story <rstory@tislabs.com> <rstory@tislabs.com> wrote:
Hello,
I'm in the process of upgrading from 3.5.x to 3.6.x. My hosted engine and hosts in the primary cluster are all upgraded and appear to be running fine.
I have a second cluster of 2 machines which are just regular hosts, without the hosted-engine. Both have been marked non-operational, with the following messages logged about every 5 minutes:
Failed to connect Host perses to Storage Pool Default
Host perses cannot access the Storage Domain(s) hosted_storage attached to the Data Center Default. Setting Host state to Non-Operational.
Host perses reports about one of the Active Storage Domains as Problematic.
Failed to connect Host perses to Storage Servers
Failed to connect Host perses to the Storage Domains hosted_storage.
I could see the normal storage/iso/export domains mounted on the host, and the VMs running on the host are fine.
In 3.5 only the hosts involved in hosted-engine have to access the hosted-engine storage domain. With 3.6 we introduced the capabilities to manage the engine VM from the engine itself so the engine has to import in the hosted-engine storage domain. This means that all the hosts in the datacenter that contains the cluster with the hosted-engine hosts have now to be able to connect the hosted-engine storage domain.
Can you please check the ACL on the storage server (NFS or iSCSI) that you use to expose the hosted-engine storage domain?
I shut down the VMs on one host, put it in maintenance mode, installed 3.6 repo and ran yum update. All went well, but when I activated the host, same deal.
I've attached the engine log snippet for the activation attempt.
Robert
-- Senior Software Engineer @ Parsons
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing listUsers@ovirt.orghttp://lists.ovirt.org/mailman/listinfo/users
--
*Ralf Schenk* fon +49 (0) 24 05 / 40 83 70 fax +49 (0) 24 05 / 40 83 759 mail *rs@databay.de* <rs@databay.de>
*Databay AG* Jens-Otto-Krag-Straße 11 D-52146 Würselen *www.databay.de* <http://www.databay.de>
Sitz/Amtsgericht Aachen • HRB:8437 • USt-IdNr.: DE 210844202 Vorstand: Ralf Schenk, Dipl.-Ing. Jens Conze, Aresch Yavari, Dipl.-Kfm. Philipp Hermanns Aufsichtsratsvorsitzender: Wilhelm Dohmen ------------------------------
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--Sig_/X22DB.eulTCYHh1dWExXoR9 Content-Type: text/plain; charset=US-ASCII Content-Transfer-Encoding: quoted-printable On Fri, 22 Jul 2016 18:21:15 +0200 Simone wrote: ST> On Fri, Jul 22, 2016 at 4:48 PM, Ralf Schenk <rs@databay.de> wrote: ST>=20 ST> > Hello, ST> > ST> > I also see from the logs that all your Storage-Domains that work are ST> > mounted as nfsVersion=3D'V4' but ovirt-nfs.netsec:/ovirt/hosted-engin= e is ST> > mounted as nfsVersion=3D'null'. ST> > =20 ST>=20 ST> Hi Robert, ST> unfortunately Ralf is right: I reproduced the issue. ST>=20 ST> The auto-import procedure for the hosted-engine storage domain ignores = the ST> nfsVersion parameter and so we don't have a value for that in the engin= e DB. ST> On hosted-engine hosts, the agent mounts the hosted-engine storage doma= in ST> before the engine and so everything is fine since the agent knows that = it's ST> nfsv4. ST>=20 ST> The issue comes with the hosts of that datacenter not involved in ST> hosted-engine: in this case the engine simply tries to mount without ST> the nfsVersion parameter and so, if the NFS server cannot be access over ST> nfsv3, the mount could fail and the host will be declared as not operat= ion. ST>=20 ST> I opened a ticket to track it: ST> https://bugzilla.redhat.com/show_bug.cgi?id=3D1359265 ST>=20 ST> If you need a quick fix you can: ST> - fix the configuration of your storage server to allow it to be access= ed ST> also over nfsv3 ST> - edit the configuration of the storage connection in the engine DB on = the ST> engine VM to add the missing parameter. Something like: ST> # sudo -u postgresl psql ST> \c engine; ST> select * from storage_server_connections; ST> UPDATE storage_server_connections SET nfs_version =3D '4' WHERE connec= tion =3D ST> 'ovirt-nfs.netsec:/ovirt/hosted-engine'; ST> commit; ST> select * from storage_server_connections; Thanks for that workaround. I've added it to the bugzilla. The easy manual workaround I tried was: # mkdir /rhev/data-center/mnt/ovirt-nfs.localdomain:_ovirt_hosted-engine # /usr/bin/mount -t nfs -o soft,nosharecache,timeo=3D600,retrans=3D6,nfsver= s=3D4 \ ovirt-nfs.localdomain:/ovirt/hosted-engine \ /rhev/data-center/mnt/ovirt-nfs.netsec:_ovirt_hosted-engine which got the hosts operational again. Thanks for all the help! Robert --=20 Senior Software Engineer @ Parsons --Sig_/X22DB.eulTCYHh1dWExXoR9 Content-Type: application/pgp-signature Content-Description: OpenPGP digital signature -----BEGIN PGP SIGNATURE----- Version: GnuPG v2 iEYEARECAAYFAleSXGYACgkQ7/fVLLY1mni5EQCeLunYa08x3AYFhHw+qmQ6dvJk bJMAmgJRfIXizUfQfsYasI13kHvSlGhn =50gl -----END PGP SIGNATURE----- --Sig_/X22DB.eulTCYHh1dWExXoR9--
participants (3)
-
Ralf Schenk -
Robert Story -
Simone Tiraboschi