3:57 p.m.
Hi Milan,
Thanks for your reply. I checked the firewall, and saw that both the bond0
interface and the VLAN interface bond0.20 had been added to the default
zone, which I believe should provide the necessary firewall access (output
below)
I double-checked the destination host's VDSM logs and wasn't able to find
any warning or error-level logs during the migration timeframe.
I checked the migration_port_* and *_port settings in qemu.conf and
libvirtd.conf and all lines are commented. I have not modified either file.
[root@vhost2 vdsm]# firewall-cmd --list-all
public (active)
target: default
icmp-block-inversion: no
interfaces: bond0 bond0.20 em1 em2 migration ovirtmgmt p1p1
sources:
services: cockpit dhcpv6-client libvirt-tls ovirt-imageio ovirt-vmconsole
snmp ssh vdsm
ports: 1311/tcp 22/tcp 6081/udp 5666/tcp
protocols:
masquerade: no
forward-ports:
source-ports:
icmp-blocks:
rich rules:
On Mon, Jan 20, 2020 at 6:29 AM Milan Zamazal <mzamazal(a)redhat.com> wrote:
Ben <gravyfish(a)gmail.com> writes:
> Hi, I'm pretty stuck at the moment so I hope someone can help me.
>
> I have an oVirt 4.3 data center with two hosts. Recently, I attempted to
> segregate migration traffic from the the standard ovirtmgmt network,
where
> the VM traffic and all other traffic resides.
>
> I set up the VLAN on my router and switch, and created LACP bonds on both
> hosts, tagging them with the VLAN ID. I confirmed the routes work fine,
and
> traffic speeds are as expected. MTU is set to 9000.
>
> After configuring the migration network in the cluster and dragging and
> dropping it onto the bonds on each host, VMs fail to migrate.
>
> oVirt is not reporting any issues with the network interfaces or sync
with
> the hosts. However, when I attempt to live-migrate a VM, progress gets to
> 1% and stalls. The transfer rate is 0Mbps, and the operation eventually
> fails.
>
> I have not been able to identify anything useful in the VDSM logs on the
> source or destination hosts, or in the engine logs. It repeats the below
> WARNING and INFO logs for the duration of the process, then logs the last
> entries when it fails. I can provide more logs if it would help. I'm not
> even sure where to start -- since I am a novice at networking, at best,
my
> suspicion the entire time was that something is misconfigured in my
> network. However, the routes are good, speed tests are fine, and I can't
> find anything else wrong with the connections. It's not impacting any
other
> traffic over the bond interfaces.
>
> Are there other requirements that must be met for VMs to migrate over a
> separate interface/network?
Hi, did you check your firewall settings? Are the required ports open?
See migration_port_* options in /etc/libvirt/qemu.conf and *_port
options in /etc/libvirt/libvirtd.conf.
Is there any error reported in the destination vdsm.log?
Regards,
Milan
> 2020-01-12 03:18:28,245-0500 WARN (migmon/a24fd7e3) [virt.vm]
> (vmId='a24fd7e3-161c-451e-8880-b3e7e1f7d86f') Migration stalling:
remaining
> (4191MiB) > lowmark (4191MiB). (migration:854)
> 2020-01-12 03:18:28,245-0500 INFO (migmon/a24fd7e3) [virt.vm]
> (vmId='a24fd7e3-161c-451e-8880-b3e7e1f7d86f') Migration Progress: 930.341
> seconds elapsed, 1% of data processed, total data: 4192MB, processed
data:
> 0MB, remaining data: 4191MB, transfer speed 0MBps, zero pages: 149MB,
> compressed: 0MB, dirty rate: 0, memory iteration: 1 (migration:881)
> 2020-01-12 03:18:31,386-0500 ERROR (migsrc/a24fd7e3) [virt.vm]
> (vmId='a24fd7e3-161c-451e-8880-b3e7e1f7d86f') operation failed: migration
> out job: unexpectedly failed (migration:282)
> 2020-01-12 03:18:32,695-0500 ERROR (migsrc/a24fd7e3) [virt.vm]
> (vmId='a24fd7e3-161c-451e-8880-b3e7e1f7d86f') Failed to migrate
> (migration:450)
> File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line
431,
> in _regular_run
> time.time(), migrationParams, machineParams
> File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line
505,
> in _startUnderlyingMigration
> File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line
591,
> in _perform_with_conv_schedule
> self._perform_migration(duri, muri)
> File "/usr/lib/python2.7/site-packages/vdsm/virt/migration.py", line
525,
> in _perform_migration
> self._migration_flags)
> libvirtError: operation failed: migration out job: unexpectedly failed
> 2020-01-12 03:18:40,880-0500 INFO (jsonrpc/6) [api.virt] FINISH
> getMigrationStatus return={'status': {'message': 'Done',
'code': 0},
> 'migrationStats': {'status': {'message': 'Fatal error
during migration',
> 'code': 12}, 'progress': 1L}} from=::ffff:10.0.0.20,41462,
> vmId=a24fd7e3-161c-451e-8880-b3e7e1f7d86f (api:54)
> _______________________________________________
> Users mailing list -- users(a)ovirt.org
> To unsubscribe send an email to users-leave(a)ovirt.org
> Privacy Statement: https://www.ovirt.org/site/privacy-policy/
> oVirt Code of Conduct:
https://www.ovirt.org/community/about/community-guidelines/
> List Archives:
>
https://lists.ovirt.org/archives/list/users@ovirt.org/message/PB3TQTFXWKA...