virt-v2v paused by system after one hour or a bit more
Hi, I am migrating some vmware VM's from an NFS Storage via rhv-upload in virt-v2v, what is working good. But now I try to move some bigger VM's with several disks and sadly after a while (I would guess around an hour) the Ovirt-engine shows me "Paused by system" instead of transfering, so when the next disk should be imported, it will fail In the ovirt-engine.log I see the following lines for the remaining 4 disks. 2024-03-21 06:14:06,815-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-35) [f61b3906-804d-470f-8524-6507081fbdec] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:17,915-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14) [aef8e312-d811-4a39-b5fb-342157209bce] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:24,959-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) [860b012d-78a4-49f8-a875-52f4299c8298] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:46,099-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [f93869ee-2ecb-4f54-b3e9-b12259637b0b] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. There are 2 strange things. 1. When I start virt-v2v it will create all 6 disks and set them to transferring, but virt-v2v will import one after the other, what leads to kind of unused/timing out transferring tickets. 2. When I copy the disk images to a local disk before, it works. Maybe just because of faster transfer speeds. Is there a possibility to transfer parallel or maybe extend the timeout? Cause basically everything is working, but it's just timing out Thanks in advance for any suggestions
On Thu, Mar 21, 2024 at 12:44 PM Claus Serbe via Users <users@ovirt.org> wrote:
Hi,
I am migrating some vmware VM's from an NFS Storage via rhv-upload in virt-v2v, what is working good.
But now I try to move some bigger VM's with several disks and sadly after a while (I would guess around an hour) the Ovirt-engine shows me "Paused by system" instead of transfering, so when the next disk should be imported, it will fail
In the ovirt-engine.log I see the following lines for the remaining 4 disks.
2024-03-21 06:14:06,815-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-35) [f61b3906-804d-470f-8524-6507081fbdec] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:17,915-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14) [aef8e312-d811-4a39-b5fb-342157209bce] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:24,959-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) [860b012d-78a4-49f8-a875-52f4299c8298] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:46,099-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [f93869ee-2ecb-4f54-b3e9-b12259637b0b] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity.
There are 2 strange things.
1. When I start virt-v2v it will create all 6 disks and set them to transferring, but virt-v2v will import one after the other, what leads to kind of unused/timing out transferring tickets.
This is a incorrect usage of ovirt-imageio API in virt-v2v, please report it here: https://github.com/libguestfs/virt-v2v/issues The right way to use the API is: 1. Start transfer 2. Upload the data 3. End the transfer It does not matter if you create all the disk at the start of the operation, but starting a transfer must be done right before you upload the data.
2. When I copy the disk images to a local disk before, it works. Maybe just because of faster transfer speeds.
Is there a possibility to transfer parallel or maybe extend the timeout?
Sure you can upload in parallel, but I'm not sure virt-v2v will probably have issues importing in parallel from other systems (e.g. vmware would not allow this). We tested uploading 10 100g images in parallel using the ovirt-img tool, each upload using 4 connections (total of 40 upload connections). You may be able to extend the timeout, but this is not recommended since your system will not clean up quickly after a bad client disconnects uncleanly without ending the transfer. Unfortunately I don't remember the which timeout should be modified on the engine side, maybe Arik or Albert can help with this. Nir
Hello Here's the technique we use : - create manually the vm on ovirt with same disks (same size that original but you can choose target type, thin provision or preallocated) - on any node, force activating the disks to make them writable at the os level (lvm, vgchange...) - if the disk type is the same on target and destination, you can use dd over netcat to copy the disks - if the type is not the same, you might use qemu-img convert over netcat If you have physical access to the node, you might use a flat backup Another workaround is to backup/restore the vm with a backup tool that works both with vmware and ovirt... I would say vprotect or vinchin Best regards -------------------------------- Cyril VINH-TUNG INVITU Computer & Network Engineering BP 32 - 98713 Papeete - French Polynesia Tél: +689 40 46 11 99 contact@invitu.com www.invitu.com Le jeu. 21 mars 2024, 00:44, Claus Serbe via Users <users@ovirt.org> a écrit :
Hi,
I am migrating some vmware VM's from an NFS Storage via rhv-upload in virt-v2v, what is working good.
But now I try to move some bigger VM's with several disks and sadly after a while (I would guess around an hour) the Ovirt-engine shows me "Paused by system" instead of transfering, so when the next disk should be imported, it will fail
In the ovirt-engine.log I see the following lines for the remaining 4 disks.
2024-03-21 06:14:06,815-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-35) [f61b3906-804d-470f-8524-6507081fbdec] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:17,915-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-14) [aef8e312-d811-4a39-b5fb-342157209bce] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:24,959-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-85) [860b012d-78a4-49f8-a875-52f4299c8298] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity. 2024-03-21 06:14:46,099-04 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-65) [f93869ee-2ecb-4f54-b3e9-b12259637b0b] EVENT_ID: UPLOAD_IMAGE_PAUSED_BY_SYSTEM_TIMEOUT(1,071), Upload was paused by system. Reason: timeout due to transfer inactivity.
There are 2 strange things.
1. When I start virt-v2v it will create all 6 disks and set them to transferring, but virt-v2v will import one after the other, what leads to kind of unused/timing out transferring tickets. 2. When I copy the disk images to a local disk before, it works. Maybe just because of faster transfer speeds.
Is there a possibility to transfer parallel or maybe extend the timeout?
Cause basically everything is working, but it's just timing out
Thanks in advance for any suggestions _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XK46HMLSR2FNYP...
On Thu, Mar 21, 2024 at 7:03 PM Cyril VINH-TUNG <cyril@invitu.com> wrote:
Hello
Here's the technique we use : - create manually the vm on ovirt with same disks (same size that original but you can choose target type, thin provision or preallocated) - on any node, force activating the disks to make them writable at the os level (lvm, vgchange...) - if the disk type is the same on target and destination, you can use dd over netcat to copy the disks - if the type is not the same, you might use qemu-img convert over netcat
This is very fragile and dangerous, you must really know what you are doing :-) If you already imported the disks from the other system, uploading them to any storage domain in any wanted (and supported) image format and allocation policy is much easier and faster using ovirt-img. See https://www.ovirt.org/media/ovirt-img-v8.pdf
If you have physical access to the node, you might use a flat backup
Another workaround is to backup/restore the vm with a backup tool that works both with vmware and ovirt... I would say vprotect or vinchin
This should also work, but when importing vms from vmware, it is not enough to copy the disk data a is, you want to modify it to remove vmware bits and add the ovirt bits - virt-v2v does all this for you. Nir
participants (3)
-
Claus Serbe -
Cyril VINH-TUNG -
Nir Soffer