5:20 p.m.
I have an ovirt cluster running ovirt 4.0 and I am seeing several errors
when I attempt to put one of our nodes into maintenance mode. The logs
on the source server show errors as follows.
Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: operation aborted:
migration job: canceled by client
Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: internal error: qemu
unexpectedly closed the monitor: 2017-02-23T15:12:58.289459Z qemu-kvm: warning: CPU(s) not
present in any NUMA nodes: 2 3 4 5 6 7 8 9 10 11 12 13 14 15
2017-02-23T15:12:58.289684Z qemu-kvm: warning: All CPU(s) up to maxcpus should be
described in NUMA config
2017-02-23T15:15:07.889891Z qemu-kvm: Unknown combination of migration flags: 0
2017-02-23T15:15:07.890821Z qemu-kvm: error while loading state section id 2(ram)
2017-02-23T15:15:07.892357Z qemu-kvm: load of migration failed: Invalid argument
This cluster does *not* have NUMA enabled so I am not sure why this
error is happening. Some migrations did succeed after being restarted
on a different host however I have two VMs that appear to be stuck. Is
there a way to resolve this?
5:28 p.m.
On 02/23/2017 04:20 PM, Michael Watters wrote:
I have an ovirt cluster running ovirt 4.0 and I am seeing several
errors
when I attempt to put one of our nodes into maintenance mode. The logs
on the source server show errors as follows.
Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: operation aborted:
migration job: canceled by client
Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: internal error: qemu
unexpectedly closed the monitor: 2017-02-23T15:12:58.289459Z qemu-kvm: warning: CPU(s) not
present in any NUMA nodes: 2 3 4 5 6 7 8 9 10 11 12 13 14 15
2017-02-23T15:12:58.289684Z qemu-kvm: warning: All CPU(s) up to maxcpus should be
described in NUMA config
2017-02-23T15:15:07.889891Z qemu-kvm: Unknown combination of migration flags: 0
2017-02-23T15:15:07.890821Z qemu-kvm: error while loading state section id 2(ram)
2017-02-23T15:15:07.892357Z qemu-kvm: load of migration failed: Invalid argument
This cluster does *not* have NUMA enabled so I am not sure why this
error is happening.
It's one implementation detail. NUMA is enabled transparently because it
is required for memory hotplug support.
It should be fully transparent.
Some migrations did succeed after being restarted
on a different host however I have two VMs that appear to be stuck. Is
there a way to resolve this?
The load/save state errors are most often found when the two sides of
the migration have different and incompatible version of QEMU.
In turn, this is quite often a bug, because forward migrations (e.g.
from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs.
So, which version of libvirt and qemu do you have on the sides of
failing migration paths?
Bests,
--
Francesco Romani
Red Hat Engineering Virtualization R & D
IRC: fromani
5:43 p.m.
On 02/23/2017 10:28 AM, Francesco Romani wrote:
On 02/23/2017 04:20 PM, Michael Watters wrote:
> I have an ovirt cluster running ovirt 4.0 and I am seeing several errors
> when I attempt to put one of our nodes into maintenance mode. The logs
> on the source server show errors as follows.
>
> Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: operation
aborted: migration job: canceled by client
> Feb 23 10:15:08 ovirt-node-production3.example.com libvirtd[18800]: internal error:
qemu unexpectedly closed the monitor: 2017-02-23T15:12:58.289459Z qemu-kvm: warning:
CPU(s) not present in any NUMA nodes: 2 3 4 5 6 7 8 9 10 11 12 13 14 15
>
2017-02-23T15:12:58.289684Z qemu-kvm: warning: All CPU(s) up to maxcpus should be
described in NUMA config
>
2017-02-23T15:15:07.889891Z qemu-kvm: Unknown combination of migration flags: 0
>
2017-02-23T15:15:07.890821Z qemu-kvm: error while loading state section id 2(ram)
>
2017-02-23T15:15:07.892357Z qemu-kvm: load of migration failed: Invalid argument
>
> This cluster does *not* have NUMA enabled so I am not sure why this
> error is happening.
It's one implementation detail. NUMA is enabled transparently because it
is required for memory hotplug support.
It should be fully transparent.
> Some migrations did succeed after being restarted
> on a different host however I have two VMs that appear to be stuck. Is
> there a way to resolve this?
The load/save state errors are most often found when the two sides of
the migration have different and incompatible version of QEMU.
In turn, this is quite often a bug, because forward migrations (e.g.
from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs.
So, which version of libvirt and qemu do you have on the sides of
failing migration paths?
Bests,
5:46 p.m.
On 02/23/2017 10:28 AM, Francesco Romani wrote:
The load/save state errors are most often found when the two sides of
the migration have different and incompatible version of QEMU.
In turn, this is quite often a bug, because forward migrations (e.g.
from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs.
I think you're on to something there. The destination server is running
ovirt 3.6 while the source server is on 4.0. The cluster compatibility
level is also set to 3.6 since I have not upgrade every host node yet.
6:08 p.m.
I canceled the migration and manually moved the VM to another host
running ovirt 4.0. The source node was then able to set itself to
maintenance mode without any errors.
On 02/23/2017 10:46 AM, Michael Watters wrote:
On 02/23/2017 10:28 AM, Francesco Romani wrote:
> The load/save state errors are most often found when the two sides of
> the migration have different and incompatible version of QEMU.
> In turn, this is quite often a bug, because forward migrations (e.g.
> from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs.
I think you're on to something there. The destination server is running
ovirt 3.6 while the source server is on 4.0. The cluster compatibility
level is also set to 3.6 since I have not upgrade every host node yet.
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
6:27 p.m.
On 23 Feb 2017, at 17:08, Michael Watters <wattersm(a)watters.ws>
wrote:
I canceled the migration and manually moved the VM to another host
running ovirt 4.0. The source node was then able to set itself to
maintenance mode without any errors.
On 02/23/2017 10:46 AM, Michael Watters wrote:
>
> On 02/23/2017 10:28 AM, Francesco Romani wrote:
>> The load/save state errors are most often found when the two sides of
>> the migration have different and incompatible version of QEMU.
>> In turn, this is quite often a bug, because forward migrations (e.g.
>> from 2.3.0 to 2.4.0) are always supported, for obvious upgrade needs.
> I think you're on to something there. The destination server is running
> ovirt 3.6 while the source server is on 4.0. The cluster compatibility
> level is also set to 3.6 since I have not upgrade every host node yet.
that is supported and works when the versions are really the latest ones.
I suggest to check the repos, it might be you are not getting updates for CentOS or
qemu-kvm-ev packages on either 3.6 or 4.0 side
>
>
> _______________________________________________
> Users mailing list
> Users(a)ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________
Users mailing list
Users(a)ovirt.org
http://lists.ovirt.org/mailman/listinfo/users
2830
days inactive
2830
days old
5 comments
3 participants
participants (3)
-
Francesco Romani -
Michael Watters -
Michal Skrivanek