Re: Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
Il giorno mer 7 lug 2021 alle ore 06:20 Nur Imam Febrianto < nur_imam@outlook.com> ha scritto:
Already tried it at one 4.4.7 host, and it solves the issue.
Maybe this issue should marked as a critical one, because the host is unusable at all if upgraded to 4.4.7.
It is: https://bugzilla.redhat.com/show_bug.cgi?id=1979624
😊
Regards,
Nur Imam Febrianto
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
*From: *Nur Imam Febrianto <nur_imam@outlook.com> *Sent: *07 July 2021 9:02 *To: *Klaas Demter <klaasdemter@gmail.com>; users@ovirt.org *Subject: *[ovirt-users] Re: Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
Where should I done this ?
At the Host ? or at HE ?
Thanks.
Regards,
Nur Imam Febrianto
*From: *Klaas Demter <klaasdemter@gmail.com> *Sent: *07 July 2021 3:31 *To: *users@ovirt.org *Subject: *[ovirt-users] Re: Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
https://bugzilla.redhat.com/show_bug.cgi?id=1979624 <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1979624&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267123705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=OCi7gileElpvUqvg4hu6C3jynnkfT8Tr35fH%2FAOVz24%3D&reserved=0>
run: semodule -B; touch /.autorelabel; reboot
report back if it fixes everything
On 7/6/21 5:40 PM, Nur Imam Febrianto wrote:
I’m having similar problem like this. 15 host, 7 of them already upgraded to 4.4.7 and I can’t migrate any VM or HE from 4.4.6 host to 4.4.7.
Regards,
Nur Imam Febrianto
*From: *Sandro Bonazzola <sbonazzo@redhat.com> *Sent: *06 July 2021 19:37 *To: *oVirt Users <users@ovirt.org>; Arik Hadas <ahadas@redhat.com> *Subject: *[ovirt-users] Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
Hi,
I update the hosted engine to 4.4.7 and one of the 2 nodes where the engine is running.
Current status is:
- Hosted engine at 4.4.7 running on Node 0
- Node 0 at 4.4.6
- Node 1 at 4.4.7
Now, moving Node 0 to maintenance successfully moved the SPM from Node 0 to Node 1 but while trying to migrate hosted engine I get on Node 0 vdsm.log:
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=()) from=::ffff:10.46.8.133,35048, task_id=f12d7694-d2b5-4658-9e0d-3f0dc54aca93 (api:48)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={'1996dc3b-d33f-49cb-b32a-8f7b1d50af5e': {'code': 0, 'lastCheck': '3.0', 'delay': '0.00114065', 'valid': True, 'version': 5, 'acq
uired': True, 'actual': True}} from=::ffff:10.46.8.133,35048, task_id=f12d7694-d2b5-4658-9e0d-3f0dc54aca93 (api:54)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] START multipath_health() from=::ffff:10.46.8.133,35048, task_id=6515fac9-830a-4b6a-904e-cc1262e87f01 (api:48)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] FINISH multipath_health return={} from=::ffff:10.46.8.133,35048, task_id=6515fac9-830a-4b6a-904e-cc1262e87f01 (api:54)
2021-07-06 12:25:07,883+0000 ERROR (migsrc/b2072331) [virt.vm] (vmId='b2072331-1558-4186-86b4-fa83af8eba95') can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission de
nied (migration:294)
2021-07-06 12:25:07,888+0000 INFO (jsonrpc/5) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::ffff:10.46.8.133,35048 (api:54)
2021-07-06 12:25:08,166+0000 ERROR (migsrc/b2072331) [virt.vm] (vmId='b2072331-1558-4186-86b4-fa83af8eba95') Failed to migrate (migration:467)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 441, in _regular_run
time.time(), machineParams
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 537, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 626, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 555, in _perform_migration
self._migration_flags)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 159, in call
return getattr(self._vm._dom, name)(*a, **kw)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 101, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2119, in migrateToURI3
raise libvirtError('virDomainMigrateToURI3() failed')
libvirt.libvirtError: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied
2021-07-06 12:25:08,197+0000 INFO (jsonrpc/6) [api.virt] START getMigrationStatus() from=::ffff:10.46.8.133,35048, flow_id=4e86b85d, vmId=b2072331-1558-4186-86b4-fa83af8eba95 (api:48)
2021-07-06 12:25:08,197+0000 INFO (jsonrpc/6) [api.virt] FINISH getMigrationStatus return={'status': {'code': 0, 'message': 'Done'}, 'migrationStats': {'status': {'code': 12, 'message': 'Fatal error during migr
ation'}, 'progress': 0}} from=::ffff:10.46.8.133,35048, flow_id=4e86b85d, vmId=b2072331-1558-4186-86b4-fa83af8eba95 (api:54)
On node 0:
# ls -lZ /run/libvirt/common/system.token ls: cannot access '/run/libvirt/common/system.token': No such file or directory
On node 1:
# ls -lZ /run/libvirt/common/system.token -rw-------. 1 root root system_u:object_r:virt_var_run_t:s0 32 Jul 6 09:29 /run/libvirt/common/system.token
any clue?
--
*Sandro Bonazzola*
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
sbonazzo@redhat.com
*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fprivacy-policy.html&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267133662%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RlGTFMuWe81IZWRJt2T10bYOPoZD3XHj9gEsCOhX71Q%3D&reserved=0>
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267143618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BxU5ysRuB4bXlqOFSJLkjUQ8dDorwMIn%2F4dvo6QH0GM%3D&reserved=0>
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2JG4HZW6RDTXCO... <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2F2JG4HZW6RDTXCOHI2AE5ZCORIDSRRAGU%2F&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267143618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kwVh7wXoO7XxksEcOs%2BJPLTI8m%2FYNTlkxIShy58rz%2BA%3D&reserved=0>
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6UMV243KWTWNOX...
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
Il giorno mer 7 lug 2021 alle ore 09:12 Sandro Bonazzola < sbonazzo@redhat.com> ha scritto:
Il giorno mer 7 lug 2021 alle ore 06:20 Nur Imam Febrianto < nur_imam@outlook.com> ha scritto:
Already tried it at one 4.4.7 host, and it solves the issue.
Maybe this issue should marked as a critical one, because the host is unusable at all if upgraded to 4.4.7.
Tracking also the workaround in release notes: https://github.com/oVirt/ovirt-site/pull/2523
😊
Regards,
Nur Imam Febrianto
Sent from Mail <https://go.microsoft.com/fwlink/?LinkId=550986> for Windows 10
*From: *Nur Imam Febrianto <nur_imam@outlook.com> *Sent: *07 July 2021 9:02 *To: *Klaas Demter <klaasdemter@gmail.com>; users@ovirt.org *Subject: *[ovirt-users] Re: Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
Where should I done this ?
At the Host ? or at HE ?
Thanks.
Regards,
Nur Imam Febrianto
*From: *Klaas Demter <klaasdemter@gmail.com> *Sent: *07 July 2021 3:31 *To: *users@ovirt.org *Subject: *[ovirt-users] Re: Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
https://bugzilla.redhat.com/show_bug.cgi?id=1979624 <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fbugzilla.redhat.com%2Fshow_bug.cgi%3Fid%3D1979624&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267123705%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=OCi7gileElpvUqvg4hu6C3jynnkfT8Tr35fH%2FAOVz24%3D&reserved=0>
run: semodule -B; touch /.autorelabel; reboot
report back if it fixes everything
On 7/6/21 5:40 PM, Nur Imam Febrianto wrote:
I’m having similar problem like this. 15 host, 7 of them already upgraded to 4.4.7 and I can’t migrate any VM or HE from 4.4.6 host to 4.4.7.
Regards,
Nur Imam Febrianto
*From: *Sandro Bonazzola <sbonazzo@redhat.com> *Sent: *06 July 2021 19:37 *To: *oVirt Users <users@ovirt.org>; Arik Hadas <ahadas@redhat.com> *Subject: *[ovirt-users] Failing to migrate hosted engine from 4.4.6 host to 4.4.7 host
Hi,
I update the hosted engine to 4.4.7 and one of the 2 nodes where the engine is running.
Current status is:
- Hosted engine at 4.4.7 running on Node 0
- Node 0 at 4.4.6
- Node 1 at 4.4.7
Now, moving Node 0 to maintenance successfully moved the SPM from Node 0 to Node 1 but while trying to migrate hosted engine I get on Node 0 vdsm.log:
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] START repoStats(domains=()) from=::ffff:10.46.8.133,35048, task_id=f12d7694-d2b5-4658-9e0d-3f0dc54aca93 (api:48)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] FINISH repoStats return={'1996dc3b-d33f-49cb-b32a-8f7b1d50af5e': {'code': 0, 'lastCheck': '3.0', 'delay': '0.00114065', 'valid': True, 'version': 5, 'acq
uired': True, 'actual': True}} from=::ffff:10.46.8.133,35048, task_id=f12d7694-d2b5-4658-9e0d-3f0dc54aca93 (api:54)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] START multipath_health() from=::ffff:10.46.8.133,35048, task_id=6515fac9-830a-4b6a-904e-cc1262e87f01 (api:48)
2021-07-06 12:25:07,882+0000 INFO (jsonrpc/5) [vdsm.api] FINISH multipath_health return={} from=::ffff:10.46.8.133,35048, task_id=6515fac9-830a-4b6a-904e-cc1262e87f01 (api:54)
2021-07-06 12:25:07,883+0000 ERROR (migsrc/b2072331) [virt.vm] (vmId='b2072331-1558-4186-86b4-fa83af8eba95') can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission de
nied (migration:294)
2021-07-06 12:25:07,888+0000 INFO (jsonrpc/5) [api.host] FINISH getStats return={'status': {'code': 0, 'message': 'Done'}, 'info': (suppressed)} from=::ffff:10.46.8.133,35048 (api:54)
2021-07-06 12:25:08,166+0000 ERROR (migsrc/b2072331) [virt.vm] (vmId='b2072331-1558-4186-86b4-fa83af8eba95') Failed to migrate (migration:467)
Traceback (most recent call last):
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 441, in _regular_run
time.time(), machineParams
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 537, in _startUnderlyingMigration
self._perform_with_conv_schedule(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 626, in _perform_with_conv_schedule
self._perform_migration(duri, muri)
File "/usr/lib/python3.6/site-packages/vdsm/virt/migration.py", line 555, in _perform_migration
self._migration_flags)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 159, in call
return getattr(self._vm._dom, name)(*a, **kw)
File "/usr/lib/python3.6/site-packages/vdsm/virt/virdomain.py", line 101, in f
ret = attr(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/libvirtconnection.py", line 131, in wrapper
ret = f(*args, **kwargs)
File "/usr/lib/python3.6/site-packages/vdsm/common/function.py", line 94, in wrapper
return func(inst, *args, **kwargs)
File "/usr/lib64/python3.6/site-packages/libvirt.py", line 2119, in migrateToURI3
raise libvirtError('virDomainMigrateToURI3() failed')
libvirt.libvirtError: can't connect to virtlogd: Unable to open system token /run/libvirt/common/system.token: Permission denied
2021-07-06 12:25:08,197+0000 INFO (jsonrpc/6) [api.virt] START getMigrationStatus() from=::ffff:10.46.8.133,35048, flow_id=4e86b85d, vmId=b2072331-1558-4186-86b4-fa83af8eba95 (api:48)
2021-07-06 12:25:08,197+0000 INFO (jsonrpc/6) [api.virt] FINISH getMigrationStatus return={'status': {'code': 0, 'message': 'Done'}, 'migrationStats': {'status': {'code': 12, 'message': 'Fatal error during migr
ation'}, 'progress': 0}} from=::ffff:10.46.8.133,35048, flow_id=4e86b85d, vmId=b2072331-1558-4186-86b4-fa83af8eba95 (api:54)
On node 0:
# ls -lZ /run/libvirt/common/system.token ls: cannot access '/run/libvirt/common/system.token': No such file or directory
On node 1:
# ls -lZ /run/libvirt/common/system.token -rw-------. 1 root root system_u:object_r:virt_var_run_t:s0 32 Jul 6 09:29 /run/libvirt/common/system.token
any clue?
--
*Sandro Bonazzola*
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
sbonazzo@redhat.com
*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*
_______________________________________________
Users mailing list -- users@ovirt.org
To unsubscribe send an email to users-leave@ovirt.org
Privacy Statement: https://www.ovirt.org/privacy-policy.html <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fprivacy-policy.html&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267133662%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=RlGTFMuWe81IZWRJt2T10bYOPoZD3XHj9gEsCOhX71Q%3D&reserved=0>
oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.ovirt.org%2Fcommunity%2Fabout%2Fcommunity-guidelines%2F&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267143618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=BxU5ysRuB4bXlqOFSJLkjUQ8dDorwMIn%2F4dvo6QH0GM%3D&reserved=0>
List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/2JG4HZW6RDTXCO... <https://apac01.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.ovirt.org%2Farchives%2Flist%2Fusers%40ovirt.org%2Fmessage%2F2JG4HZW6RDTXCOHI2AE5ZCORIDSRRAGU%2F&data=04%7C01%7C%7C7cdd04b19a5c4a0168ed08d940eb36d7%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C637612201267143618%7CUnknown%7CTWFpbGZsb3d8eyJWIjoiMC4wLjAwMDAiLCJQIjoiV2luMzIiLCJBTiI6Ik1haWwiLCJXVCI6Mn0%3D%7C1000&sdata=kwVh7wXoO7XxksEcOs%2BJPLTI8m%2FYNTlkxIShy58rz%2BA%3D&reserved=0>
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6UMV243KWTWNOX...
--
Sandro Bonazzola
MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV
Red Hat EMEA <https://www.redhat.com/>
sbonazzo@redhat.com <https://www.redhat.com/>
*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
-- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo@redhat.com <https://www.redhat.com/> *Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*
participants (1)
-
Sandro Bonazzola