<html>Hi,<br />Yes, on 2018-02-16 (vdsm logs) I tried with a VM standing on ginger (192.168.0.6) migrated (or failed to migrate...) to victor (192.168.0.5), while the engine.log in the first mail on 2018-02-12 was for VMs standing on victor, migrated (or failed to migrate...) to ginger. Symptoms were exactly the same, in both directions, and VMs works like a charm before, and even after (migration "killed" by a poweroff of VMs).<br />Am I the only one experimenting this problem ?<br /><br /><br />Thanks<br />--<br /><style type="text/css">.Text1 {
color: black;
font-size:9pt;
font-family:Verdana;
}
.Text2 {
color: black;
font-size:7pt;
font-family:Verdana;
}</style><p class="Text1">Cordialement,<br /><br /><b>Frank Soyer </b><br /> </p><br /><br />Le Jeudi, Février 22, 2018 00:45 CET, Maor Lipchuk <mlipchuk@redhat.com> a écrit:<br /> <blockquote type="cite" cite="CAJ1JNOe9Yi5XnFWvqOYhpoMuhkXOKAR=NOWafRkRHLXuOTtwtg@mail.gmail.com"><div dir="ltr">Hi Frank,<div> </div><div>Sorry about the delay repond.</div><div>I've been going through the logs you attached, although I could not find any specific indication why the migration failed because of the disk you were mentionning.</div><div>Does this VM run with both disks on the target host without migration?</div><div> </div><div>Regards,</div><div>Maor</div><div> </div></div><div class="gmail_extra"> <div class="gmail_quote">On Fri, Feb 16, 2018 at 11:03 AM, fsoyer <span dir="ltr"><<a target="_blank" href="mailto:fsoyer@systea.fr">fsoyer@systea.fr</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex">Hi Maor,<br />sorry for the double post, I've change the email adress of my account and supposed that I'd need to re-post it.<br />And thank you for your time. Here are the logs. I added a vdisk to an existing VM : it no more migrates, needing to poweroff it after minutes. Then simply deleting the second disk makes migrate it in exactly 9s without problem ! <br /><a target="_blank" href="https://gist.github.com/fgth/4707446331d201eef574ac31b6e89561">https://gist.github.com/fgth/<wbr />4707446331d201eef574ac31b6e895<wbr />61</a><br /><a target="_blank" href="https://gist.github.com/fgth/f8de9c22664aee53722af676bff8719d">https://gist.github.com/fgth/<wbr />f8de9c22664aee53722af676bff871<wbr />9d</a><br /><br />--<p class="m_-4299273321983674487Text1">Cordialement,<br /><br /><b>Frank Soyer </b></p><div class="HOEnZb"><div class="h5">Le Mercredi, Février 14, 2018 11:04 CET, Maor Lipchuk <<a target="_blank" href="mailto:mlipchuk@redhat.com">mlipchuk@redhat.com</a>> a écrit:<br /> <blockquote type="cite" cite="http://CAJ1JNOcD3ZX6hYG4TJ0-_umSgw6-wtJoC__HRbarc1io-Y-6Jw@mail.gmail.com"><div dir="ltr">Hi Frank,<div> </div><div>I already replied on your last email.</div><div><div>Can you provide the VDSM logs from the time of the migration failure for both hosts:</div><div> <span style="font-family:monospace"> </span><a style="font-family:monospace" target="_blank" href="http://ginger.local.systea.fr/">ginger.local.systea.f</a>r and <a style="font-family:monospace" target="_blank" href="http://victor.local.systea.fr/">v<wbr />ictor.local.systea.fr</a></div><div> </div><div>Thanks,</div><div>Maor</div></div></div><div class="gmail_extra"> <div class="gmail_quote">On Wed, Feb 14, 2018 at 11:23 AM, fsoyer <span dir="ltr"><<a target="_blank" href="mailto:fsoyer@systea.fr">fsoyer@systea.fr</a>></span> wrote:<blockquote class="gmail_quote" style="margin:0 0 0 .8ex;border-left:1px #ccc solid;padding-left:1ex"><p>Hi all,<br />I discovered yesterday a problem when migrating VM with more than one vdisk.<br />On our test servers (oVirt4.1, shared storage with Gluster), I created 2 VMs needed for a test, from a template with a 20G vdisk. On this VMs I added a 100G vdisk (for this tests I didn't want to waste time to extend the existing vdisks... But I lost time finally...). The VMs with the 2 vdisks works well.<br />Now I saw some updates waiting on the host. I tried to put it in maintenance... But it stopped on the two VM. They were marked "migrating", but no more accessible. Other (small) VMs with only 1 vdisk was migrated without problem at the same time.<br />I saw that a kvm process for the (big) VMs was launched on the source AND destination host, but after tens of minutes, the migration and the VMs was always freezed. I tried to cancel the migration for the VMs : failed. The only way to stop it was to poweroff the VMs : the kvm process died on the 2 hosts and the GUI alerted on a failed migration.<br />In doubt, I tried to delete the second vdisk on one of this VMs : it migrates then without error ! And no access problem.<br />I tried to extend the first vdisk of the second VM, the delete the second vdisk : it migrates now without problem ! <br /><br />So after another test with a VM with 2 vdisks, I can say that this blocked the migration process :(<br /><br />In engine.log, for a VMs with 1 vdisk migrating well, we see :</p><blockquote>2018-02-12 16:46:29,705+01 INFO [org.ovirt.engine.core.bll.Mi<wbr />grateVmToServerCommand] (default task-28) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] Lock Acquired to object 'EngineLock:{exclusiveLocks='[<wbr />3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d=VM]', sharedLocks=''}'<br />2018-02-12 16:46:29,955+01 INFO [org.ovirt.engine.core.bll.Mi<wbr />grateVmToServerCommand] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d Type: VMAction group MIGRATE_VM with role type USER<br />2018-02-12 16:46:30,261+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.MigrateVDSCommand] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] START, MigrateVDSCommand( MigrateVDSCommandParameters:{r<wbr />unAsync='true', hostId='ce3938b1-b23f-4d22-840<wbr />a-f17d7cd87bb1', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d', srcHost='192.168.0.6', dstVdsId='d569c2dd-8f30-4878-8<wbr />aea-858db285cf69', dstHost='<a target="_blank" href="http://192.168.0.5:54321">192.168.0.5:54321</a>', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{n<wbr />ame=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]'}), log id: 14f61ee0<br />2018-02-12 16:46:30,262+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateBrokerVDS<wbr />Command] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] START, MigrateBrokerVDSCommand(HostNa<wbr />me = <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, MigrateVDSCommandParameters:{r<wbr />unAsync='true', hostId='ce3938b1-b23f-4d22-840<wbr />a-f17d7cd87bb1', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d', srcHost='192.168.0.6', dstVdsId='d569c2dd-8f30-4878-8<wbr />aea-858db285cf69', dstHost='<a target="_blank" href="http://192.168.0.5:54321">192.168.0.5:54321</a>', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{n<wbr />ame=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]'}), log id: 775cd381<br />2018-02-12 16:46:30,277+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateBrokerVDS<wbr />Command] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] FINISH, MigrateBrokerVDSCommand, log id: 775cd381<br />2018-02-12 16:46:30,285+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.MigrateVDSCommand] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 14f61ee0<br />2018-02-12 16:46:30,301+01 INFO [org.ovirt.engine.core.dal.db<wbr />broker.auditloghandling.AuditL<wbr />ogDirector] (org.ovirt.thread.pool-6-threa<wbr />d-32) [2f712024-5982-46a8-82c8-fd829<wbr />3da5725] EVENT_ID: VM_MIGRATION_START(62), Correlation ID: 2f712024-5982-46a8-82c8-fd8293<wbr />da5725, Job ID: 4bd19aa9-cc99-4d02-884e-5a1e85<wbr />7a7738, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Migration started (VM: Oracle_SECONDARY, Source: <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, Destination: <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, User: admin@internal-authz).<br />2018-02-12 16:46:31,106+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.FullListVDSComma<wbr />nd] (DefaultQuartzScheduler9) [54a65b66] START, FullListVDSCommand(HostName = <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, FullListVDSCommandParameters:{<wbr />runAsync='true', hostId='ce3938b1-b23f-4d22-840<wbr />a-f17d7cd87bb1', vmIds='[3f57e669-5e4c-4d10-85c<wbr />c-d573004a099d]'}), log id: 54b4b435<br />2018-02-12 16:46:31,147+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.FullListVDSComma<wbr />nd] (DefaultQuartzScheduler9) [54a65b66] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel<wbr />7.3.0, tabletEnable=true, pid=1493, guestDiskMapping={0QEMU_QEMU_H<wbr />ARDDISK_d890fa68-fba4-4f49-9={<wbr />name=/dev/sda}, QEMU_DVD-ROM_QM00003={name=/de<wbr />v/sr0}}, transparentHugePages=true, timeOffset=0, cpuType=Nehalem, smp=2, pauseCode=NOERR, guestNumaNodes=[Ljava.lang.Obj<wbr />ect;@1d9042cd, smartcardEnable=false, custom={device_fbddd528-7d93-4<wbr />9c6-a286-180e021cb274device_87<wbr />9c93ab-4df1-435c-af02-565039fc<wbr />c254=VmDevice:{id='VmDeviceId:<wbr />{deviceId='879c93ab-4df1-435c-<wbr />af02-565039fcc254', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-a4<wbr />aa6f1908dbdevice_017b5e59-01c4<wbr />-4aac-bf0c-b5d9557284d6=VmDevi<wbr />ce:{id='VmDeviceId:{deviceId='<wbr />017b5e59-01c4-4aac-bf0c-<wbr />b5d9557284d6', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='tablet', type='UNKNOWN', bootOrder='0', specParams='[]', address='{bus=0, type=usb, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='input0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274=VmDevice:{id='<wbr />VmDeviceId:{deviceId='fbddd528<wbr />-7d93-49c6-a286-180e021cb274', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db=VmDevice:{id='<wbr />VmDeviceId:{deviceId='<wbr />8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}}, vmType=kvm, memSize=8192, smpCoresPerSocket=1, vmName=Oracle_SECONDARY, nice=0, status=Migration Source, maxMemSize=32768, bootMenuEnable=false, vmId=3f57e669-5e4c-4d10-85cc-d<wbr />573004a099d, numOfIoThreads=2, smpThreadsPerCore=1, memGuaranteedSize=8192, kvmEnable=true, pitReinjection=false, displayNetwork=ovirtmgmt, devices=[Ljava.lang.Object;@28<wbr />ae66d7, display=vnc, maxVCpus=16, clientIp=, statusTime=4299484520, maxMemSlots=16}], log id: 54b4b435<br />2018-02-12 16:46:31,150+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmsStatisticsFe<wbr />tcher] (DefaultQuartzScheduler1) [27fac647] Fetched 3 VMs from VDS 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69'<br />2018-02-12 16:46:31,151+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (DefaultQuartzScheduler9) [54a65b66] Received a vnc Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {device=vnc, specParams={displayNetwork=ovi<wbr />rtmgmt, keyMap=fr, displayIp=192.168.0.6}, type=graphics, deviceId=813957b1-446a-4e88-9e<wbr />40-9fe76d2c442d, port=5901}<br />2018-02-12 16:46:31,151+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (DefaultQuartzScheduler9) [54a65b66] Received a lease Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {lease_id=3f57e669-5e4c-4d10-8<wbr />5cc-d573004a099d, sd_id=1e51cecc-eb2e-47d0-b185-<wbr />920fdc7afa16, deviceId={uuid=a09949aa-5642-4<wbr />b6d-94a4-8b0d04257be5}, offset=6291456, device=lease, path=/rhev/data-center/mnt/glu<wbr />sterSD/192.168.0.6:_DATA01/1e5<wbr />1cecc-eb2e-47d0-b185-920fdc7af<wbr />a16/dom_md/xleases, type=lease}<br />2018-02-12 16:46:31,152+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler1) [27fac647] VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d'(Oracle_SECONDARY) was unexpectedly detected as 'MigratingTo' on VDS 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69'(<a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.<wbr />fr</a>) (expected on 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1')<br />2018-02-12 16:46:31,152+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler1) [27fac647] VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d' is migrating to VDS 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69'(<a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.<wbr />fr</a>) ignoring it in the refresh until migration is done<br />....<br />2018-02-12 16:46:41,631+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-11) [] VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d' was reported as Down on VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'(<a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.<wbr />fr</a>)<br />2018-02-12 16:46:41,632+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.DestroyVDSComman<wbr />d] (ForkJoinPool-1-worker-11) [] START, DestroyVDSCommand(HostName = <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, DestroyVmVDSCommandParameters:<wbr />{runAsync='true', hostId='ce3938b1-b23f-4d22-840<wbr />a-f17d7cd87bb1', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d', force='false', secondsToWait='0', gracefully='false', reason='', ignoreNoVm='true'}), log id: 560eca57<br />2018-02-12 16:46:41,650+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.DestroyVDSComman<wbr />d] (ForkJoinPool-1-worker-11) [] FINISH, DestroyVDSCommand, log id: 560eca57<br />2018-02-12 16:46:41,650+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-11) [] VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d'(Oracle_SECONDARY) moved from 'MigratingFrom' --> 'Down'<br />2018-02-12 16:46:41,651+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-11) [] Handing over VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d'(Oracle_SECONDARY) to Host 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69'. Setting VM to status 'MigratingTo'<br />2018-02-12 16:46:42,163+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (ForkJoinPool-1-worker-4) [] VM '3f57e669-5e4c-4d10-85cc-d5730<wbr />04a099d'(Oracle_SECONDARY) moved from 'MigratingTo' --> 'Up'<br />2018-02-12 16:46:42,169+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateStatusVDS<wbr />Command] (ForkJoinPool-1-worker-4) [] START, MigrateStatusVDSCommand(HostNa<wbr />me = <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, MigrateStatusVDSCommandParamet<wbr />ers:{runAsync='true', hostId='d569c2dd-8f30-4878-8ae<wbr />a-858db285cf69', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}), log id: 7a25c281<br />2018-02-12 16:46:42,174+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateStatusVDS<wbr />Command] (ForkJoinPool-1-worker-4) [] FINISH, MigrateStatusVDSCommand, log id: 7a25c281<br />2018-02-12 16:46:42,194+01 INFO [org.ovirt.engine.core.dal.db<wbr />broker.auditloghandling.AuditL<wbr />ogDirector] (ForkJoinPool-1-worker-4) [] EVENT_ID: VM_MIGRATION_DONE(63), Correlation ID: 2f712024-5982-46a8-82c8-fd8293<wbr />da5725, Job ID: 4bd19aa9-cc99-4d02-884e-5a1e85<wbr />7a7738, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Migration completed (VM: Oracle_SECONDARY, Source: <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, Destination: <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, Duration: 11 seconds, Total: 11 seconds, Actual downtime: (N/A))<br />2018-02-12 16:46:42,201+01 INFO [org.ovirt.engine.core.bll.Mi<wbr />grateVmToServerCommand] (ForkJoinPool-1-worker-4) [] Lock freed to object 'EngineLock:{exclusiveLocks='[<wbr />3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d=VM]', sharedLocks=''}'<br />2018-02-12 16:46:42,203+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.FullListVDSComma<wbr />nd] (ForkJoinPool-1-worker-4) [] START, FullListVDSCommand(HostName = <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, FullListVDSCommandParameters:{<wbr />runAsync='true', hostId='d569c2dd-8f30-4878-8ae<wbr />a-858db285cf69', vmIds='[3f57e669-5e4c-4d10-85c<wbr />c-d573004a099d]'}), log id: 7cc65298<br />2018-02-12 16:46:42,254+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.FullListVDSComma<wbr />nd] (ForkJoinPool-1-worker-4) [] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel<wbr />7.3.0, afterMigrationStatus=, tabletEnable=true, pid=18748, guestDiskMapping={}, transparentHugePages=true, timeOffset=0, cpuType=Nehalem, smp=2, guestNumaNodes=[Ljava.lang.Obj<wbr />ect;@760085fd, custom={device_fbddd528-7d93-4<wbr />9c6-a286-180e021cb274device_87<wbr />9c93ab-4df1-435c-af02-565039fc<wbr />c254=VmDevice:{id='VmDeviceId:<wbr />{deviceId='879c93ab-4df1-435c-<wbr />af02-565039fcc254', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-a4<wbr />aa6f1908dbdevice_017b5e59-01c4<wbr />-4aac-bf0c-b5d9557284d6=VmDevi<wbr />ce:{id='VmDeviceId:{deviceId='<wbr />017b5e59-01c4-4aac-bf0c-<wbr />b5d9557284d6', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='tablet', type='UNKNOWN', bootOrder='0', specParams='[]', address='{bus=0, type=usb, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='input0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274=VmDevice:{id='<wbr />VmDeviceId:{deviceId='fbddd528<wbr />-7d93-49c6-a286-180e021cb274', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db=VmDevice:{id='<wbr />VmDeviceId:{deviceId='<wbr />8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}}, vmType=kvm, memSize=8192, smpCoresPerSocket=1, vmName=Oracle_SECONDARY, nice=0, status=Up, maxMemSize=32768, bootMenuEnable=false, vmId=3f57e669-5e4c-4d10-85cc-d<wbr />573004a099d, numOfIoThreads=2, smpThreadsPerCore=1, smartcardEnable=false, maxMemSlots=16, kvmEnable=true, pitReinjection=false, displayNetwork=ovirtmgmt, devices=[Ljava.lang.Object;@2e<wbr />4d3dd3, memGuaranteedSize=8192, maxVCpus=16, clientIp=, statusTime=<a value="+14304259600" target="_blank" href="tel:(430)%20425-9600">4304259600</a>, display=vnc}], log id: 7cc65298<br />2018-02-12 16:46:42,257+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (ForkJoinPool-1-worker-4) [] Received a vnc Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {device=vnc, specParams={displayNetwork=ovi<wbr />rtmgmt, keyMap=fr, displayIp=192.168.0.5}, type=graphics, deviceId=813957b1-446a-4e88-9e<wbr />40-9fe76d2c442d, port=5901}<br />2018-02-12 16:46:42,257+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (ForkJoinPool-1-worker-4) [] Received a lease Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {lease_id=3f57e669-5e4c-4d10-8<wbr />5cc-d573004a099d, sd_id=1e51cecc-eb2e-47d0-b185-<wbr />920fdc7afa16, deviceId={uuid=a09949aa-5642-4<wbr />b6d-94a4-8b0d04257be5}, offset=6291456, device=lease, path=/rhev/data-center/mnt/glu<wbr />sterSD/192.168.0.6:_DATA01/1e5<wbr />1cecc-eb2e-47d0-b185-920fdc7af<wbr />a16/dom_md/xleases, type=lease}<br />2018-02-12 16:46:46,260+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.FullListVDSComma<wbr />nd] (DefaultQuartzScheduler5) [7fcb200a] FINISH, FullListVDSCommand, return: [{acpiEnable=true, emulatedMachine=pc-i440fx-rhel<wbr />7.3.0, afterMigrationStatus=, tabletEnable=true, pid=18748, guestDiskMapping={0QEMU_QEMU_H<wbr />ARDDISK_d890fa68-fba4-4f49-9={<wbr />name=/dev/sda}, QEMU_DVD-ROM_QM00003={name=/de<wbr />v/sr0}}, transparentHugePages=true, timeOffset=0, cpuType=Nehalem, smp=2, guestNumaNodes=[Ljava.lang.Obj<wbr />ect;@77951faf, custom={device_fbddd528-7d93-4<wbr />9c6-a286-180e021cb274device_87<wbr />9c93ab-4df1-435c-af02-565039fc<wbr />c254=VmDevice:{id='VmDeviceId:<wbr />{deviceId='879c93ab-4df1-435c-<wbr />af02-565039fcc254', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='channel0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-a4<wbr />aa6f1908dbdevice_017b5e59-01c4<wbr />-4aac-bf0c-b5d9557284d6=VmDevi<wbr />ce:{id='VmDeviceId:{deviceId='<wbr />017b5e59-01c4-4aac-bf0c-<wbr />b5d9557284d6', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='tablet', type='UNKNOWN', bootOrder='0', specParams='[]', address='{bus=0, type=usb, port=1}', managed='false', plugged='true', readOnly='false', deviceAlias='input0', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274=VmDevice:{id='<wbr />VmDeviceId:{deviceId='fbddd528<wbr />-7d93-49c6-a286-180e021cb274', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='ide', type='CONTROLLER', bootOrder='0', specParams='[]', address='{slot=0x01, bus=0x00, domain=0x0000, type=pci, function=0x1}', managed='false', plugged='true', readOnly='false', deviceAlias='ide', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}, device_fbddd528-7d93-49c6-a286<wbr />-180e021cb274device_879c93ab-<wbr />4df1-435c-af02-565039fcc254dev<wbr />ice_8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db=VmDevice:{id='<wbr />VmDeviceId:{deviceId='<wbr />8945f61a-abbe-4156-8485-<wbr />a4aa6f1908db', vmId='3f57e669-5e4c-4d10-85cc-<wbr />d573004a099d'}', device='unix', type='CHANNEL', bootOrder='0', specParams='[]', address='{bus=0, controller=0, type=virtio-serial, port=2}', managed='false', plugged='true', readOnly='false', deviceAlias='channel1', customProperties='[]', snapshotId='null', logicalName='null', hostDevice='null'}}, vmType=kvm, memSize=8192, smpCoresPerSocket=1, vmName=Oracle_SECONDARY, nice=0, status=Up, maxMemSize=32768, bootMenuEnable=false, vmId=3f57e669-5e4c-4d10-85cc-d<wbr />573004a099d, numOfIoThreads=2, smpThreadsPerCore=1, smartcardEnable=false, maxMemSlots=16, kvmEnable=true, pitReinjection=false, displayNetwork=ovirtmgmt, devices=[Ljava.lang.Object;@28<wbr />6410fd, memGuaranteedSize=8192, maxVCpus=16, clientIp=, statusTime=<a value="+14304263620" target="_blank" href="tel:(430)%20426-3620">4304263620</a>, display=vnc}], log id: 58cdef4c<br />2018-02-12 16:46:46,267+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (DefaultQuartzScheduler5) [7fcb200a] Received a vnc Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {device=vnc, specParams={displayNetwork=ovi<wbr />rtmgmt, keyMap=fr, displayIp=192.168.0.5}, type=graphics, deviceId=813957b1-446a-4e88-9e<wbr />40-9fe76d2c442d, port=5901}<br />2018-02-12 16:46:46,268+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmDevicesMonito<wbr />ring] (DefaultQuartzScheduler5) [7fcb200a] Received a lease Device without an address when processing VM 3f57e669-5e4c-4d10-85cc-d57300<wbr />4a099d devices, skipping device: {lease_id=3f57e669-5e4c-4d10-8<wbr />5cc-d573004a099d, sd_id=1e51cecc-eb2e-47d0-b185-<wbr />920fdc7afa16, deviceId={uuid=a09949aa-5642-4<wbr />b6d-94a4-8b0d04257be5}, offset=6291456, device=lease, path=/rhev/data-center/mnt/glu<wbr />sterSD/192.168.0.6:_DATA01/1e5<wbr />1cecc-eb2e-47d0-b185-920fdc7af<wbr />a16/dom_md/xleases, type=lease}<p> </p></blockquote><br />For the VM with 2 vdisks we see :<blockquote><p>2018-02-12 16:49:06,112+01 INFO [org.ovirt.engine.core.bll.Mi<wbr />grateVmToServerCommand] (default task-50) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] Lock Acquired to object 'EngineLock:{exclusiveLocks='[<wbr />f7d4ec12-627a-4b83-b59e-886400<wbr />d55474=VM]', sharedLocks=''}'<br />2018-02-12 16:49:06,407+01 INFO [org.ovirt.engine.core.bll.Mi<wbr />grateVmToServerCommand] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] Running command: MigrateVmToServerCommand internal: false. Entities affected : ID: f7d4ec12-627a-4b83-b59e-886400<wbr />d55474 Type: VMAction group MIGRATE_VM with role type USER<br />2018-02-12 16:49:06,712+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.MigrateVDSCommand] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] START, MigrateVDSCommand( MigrateVDSCommandParameters:{r<wbr />unAsync='true', hostId='d569c2dd-8f30-4878-8ae<wbr />a-858db285cf69', vmId='f7d4ec12-627a-4b83-b59e-<wbr />886400d55474', srcHost='192.168.0.5', dstVdsId='ce3938b1-b23f-4d22-8<wbr />40a-f17d7cd87bb1', dstHost='<a target="_blank" href="http://192.168.0.6:54321">192.168.0.6:54321</a>', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{n<wbr />ame=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]'}), log id: 3702a9e0<br />2018-02-12 16:49:06,713+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateBrokerVDS<wbr />Command] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] START, MigrateBrokerVDSCommand(HostNa<wbr />me = <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, MigrateVDSCommandParameters:{r<wbr />unAsync='true', hostId='d569c2dd-8f30-4878-8ae<wbr />a-858db285cf69', vmId='f7d4ec12-627a-4b83-b59e-<wbr />886400d55474', srcHost='192.168.0.5', dstVdsId='ce3938b1-b23f-4d22-8<wbr />40a-f17d7cd87bb1', dstHost='<a target="_blank" href="http://192.168.0.6:54321">192.168.0.6:54321</a>', migrationMethod='ONLINE', tunnelMigration='false', migrationDowntime='0', autoConverge='true', migrateCompressed='false', consoleAddress='null', maxBandwidth='500', enableGuestEvents='true', maxIncomingMigrations='2', maxOutgoingMigrations='2', convergenceSchedule='[init=[{n<wbr />ame=setDowntime, params=[100]}], stalling=[{limit=1, action={name=setDowntime, params=[150]}}, {limit=2, action={name=setDowntime, params=[200]}}, {limit=3, action={name=setDowntime, params=[300]}}, {limit=4, action={name=setDowntime, params=[400]}}, {limit=6, action={name=setDowntime, params=[500]}}, {limit=-1, action={name=abort, params=[]}}]]'}), log id: 1840069c<br />2018-02-12 16:49:06,724+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.vdsbroker.MigrateBrokerVDS<wbr />Command] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] FINISH, MigrateBrokerVDSCommand, log id: 1840069c<br />2018-02-12 16:49:06,732+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.MigrateVDSCommand] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] FINISH, MigrateVDSCommand, return: MigratingFrom, log id: 3702a9e0<br />2018-02-12 16:49:06,753+01 INFO [org.ovirt.engine.core.dal.db<wbr />broker.auditloghandling.AuditL<wbr />ogDirector] (org.ovirt.thread.pool-6-threa<wbr />d-49) [92b5af33-cb87-4142-b8fe-8b838<wbr />dd7458e] EVENT_ID: VM_MIGRATION_START(62), Correlation ID: 92b5af33-cb87-4142-b8fe-8b838d<wbr />d7458e, Job ID: f4f54054-f7c8-4481-8eda-d5a15c<wbr />383061, Call Stack: null, Custom ID: null, Custom Event ID: -1, Message: Migration started (VM: Oracle_PRIMARY, Source: <a target="_blank" href="http://ginger.local.systea.fr">ginger.local.systea.fr</a>, Destination: <a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.fr</a>, User: admin@internal-authz).<br />...<br />2018-02-12 16:49:16,453+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmsStatisticsFe<wbr />tcher] (DefaultQuartzScheduler4) [162a5bc3] Fetched 2 VMs from VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'<br />2018-02-12 16:49:16,455+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler4) [162a5bc3] VM 'f7d4ec12-627a-4b83-b59e-88640<wbr />0d55474'(Oracle_PRIMARY) was unexpectedly detected as 'MigratingTo' on VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'(<a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.<wbr />fr</a>) (expected on 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69')<br />2018-02-12 16:49:16,455+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler4) [162a5bc3] VM 'f7d4ec12-627a-4b83-b59e-88640<wbr />0d55474' is migrating to VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'(<a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.<wbr />fr</a>) ignoring it in the refresh until migration is done<br />...<br />2018-02-12 16:49:31,484+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler5) [11a7619a] VM 'f7d4ec12-627a-4b83-b59e-88640<wbr />0d55474'(Oracle_PRIMARY) was unexpectedly detected as 'MigratingTo' on VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'(<a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.<wbr />fr</a>) (expected on 'd569c2dd-8f30-4878-8aea-858db<wbr />285cf69')<br />2018-02-12 16:49:31,484+01 INFO [org.ovirt.engine.core.vdsbro<wbr />ker.monitoring.VmAnalyzer] (DefaultQuartzScheduler5) [11a7619a] VM 'f7d4ec12-627a-4b83-b59e-88640<wbr />0d55474' is migrating to VDS 'ce3938b1-b23f-4d22-840a-f17d7<wbr />cd87bb1'(<a target="_blank" href="http://victor.local.systea.fr">victor.local.systea.<wbr />fr</a>) ignoring it in the refresh until migration is done<br /> </p></blockquote><br />and so on, last lines repeated indefinitly for hours since we poweroff the VM...<br />Is this something known ? Any idea about that ?<br /><br />Thanks<br /><br />Ovirt 4.1.6, updated last at feb-13. Gluster 3.12.1.<br /><br />--<p class="m_-4299273321983674487m_8587729722327689770Text1">Cordialement,<br /><br /><b>Frank Soyer </b></p><br />______________________________<wbr />_________________<br />Users mailing list<br /><a target="_blank" href="mailto:Users@ovirt.org">Users@ovirt.org</a><br /><a rel="noreferrer" target="_blank" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman<wbr />/listinfo/users</a><br /> </blockquote></div></div></blockquote><br /> </div></div></blockquote></div></div></blockquote><br /> </html>