Hello,

I'm trying to import virtual machines from a KVM host (centos 7.2) to an oVirt 4.0.2 Cluster using the "import" feature on GUI. If the original VM is using RAW/QCOW2 files as storage, everything works fine. But if the original VM is using a special block special device as storage (like a LVM or SAN volume), it's simply not recognized. The VM does appear in the import list of the KVM host, but it's disk count is 0! Is this a known technical obstacle or am I doing something wrong ? below is the storage part of the xml describing the original VM : <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source dev='/dev/mapper/vg_01-lv_sys'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source dev='/dev/sdc'/> <target dev='vdb' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> </disk> We have hundreds of virtual machines in production with this type of configuration... How can we migrate them safely to oVirt? thanks Nelson _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

Thanks for your answer. I do not think that I'm experiencing that same bug exactly. The issue 1362525 (below) concerns a KVM VM with iSCSI disk being importend to RHEV that does not perform well. In my case, when oVirt GUI proposes me the list of machines on KVM host to import, each machine as a "disk count" next to it. - This "disk count" is correct if the VM has it's storage on a raw/qcow file (local) - and import goes well - But "disk count" is 0 (always) if the VM has it's storage on a block device (ie iscsi) - and import is not possible at all! It seems to me that current oVirt migration tool is unable to read/analyse distant VM machines with block device storage... Maybe I'm analysing this issue wrongly.

Thanks, nelson ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; Cc: users(a)ovirt.org Sent: Tuesday, August 23, 2016 12:19:13 PM Subject: Re: [ovirt-users] can't import vm from KVM host On 23.08.16 11:24, Nelson Lameiras wrote: > Hello, Hi, We do have a bug on that issue: https://bugzilla.redhat.com/show_bug.cgi?id=1362525 and its handled. Thanks, Shahar. > > I'm trying to import virtual machines from a KVM host (centos 7.2) to an oVirt 4.0.2 Cluster using the "import" feature on GUI. > > If the original VM is using RAW/QCOW2 files as storage, everything works fine. > > But if the original VM is using a special block special device as storage (like a LVM or SAN volume), it's simply not recognized. > The VM does appear in the import list of the KVM host, but it's disk count is 0! > > Is this a known technical obstacle or am I doing something wrong ? > > below is the storage part of the xml describing the original VM : > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/mapper/vg_01-lv_sys'/> > <target dev='vda' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > </disk> > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/sdc'/> > <target dev='vdb' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > </disk> > > We have hundreds of virtual machines in production with this type of configuration... How can we migrate them safely to oVirt? > > thanks > > Nelson > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users

I upgraded to oVirt 4.0.4 RC1 to test this issue (engine and hosts) and I still have the same problem. Migration GUI still shows zero disks when the source VM had block disk devices.

Can you confirm that the issue 1362525 has been integrated in this RC? Should I do some additional actions in order to see/make it work? Thanks, Nelson ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, "Nisim Simsolo" <nsimsolo(a)redhat.com&gt; Cc: users(a)ovirt.org Sent: Sunday, August 28, 2016 8:45:48 AM Subject: Re: [ovirt-users] can't import vm from KVM host On 25.08.16 11:32, Nelson Lameiras wrote: > Thanks for your answer. I do not think that I'm experiencing that same bug exactly. > > The issue 1362525 (below) concerns a KVM VM with iSCSI disk being importend to RHEV that does not perform well. > > In my case, when oVirt GUI proposes me the list of machines on KVM host to import, each machine as a "disk count" next to it. > - This "disk count" is correct if the VM has it's storage on a raw/qcow file (local) - and import goes well > - But "disk count" is 0 (always) if the VM has it's storage on a block device (ie iscsi) - and import is not possible at all! > > It seems to me that current oVirt migration tool is unable to read/analyse distant VM machines with block device storage... Maybe I'm analysing this issue wrongly. Yes, This was the issue as well - seeing zero disks when the source VM had block disk devices. Adding Nisim to verify it. > > Thanks, > nelson > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: users(a)ovirt.org > Sent: Tuesday, August 23, 2016 12:19:13 PM > Subject: Re: [ovirt-users] can't import vm from KVM host > > On 23.08.16 11:24, Nelson Lameiras wrote: > > Hello, > Hi, > > We do have a bug on that issue: > https://bugzilla.redhat.com/show_bug.cgi?id=1362525 > and its handled. > > Thanks, > Shahar. > > > > I'm trying to import virtual machines from a KVM host (centos 7.2) to an oVirt 4.0.2 Cluster using the "import" feature on GUI. > > > > If the original VM is using RAW/QCOW2 files as storage, everything works fine. > > > > But if the original VM is using a special block special device as storage (like a LVM or SAN volume), it's simply not recognized. > > The VM does appear in the import list of the KVM host, but it's disk count is 0! > > > > Is this a known technical obstacle or am I doing something wrong ? > > > > below is the storage part of the xml describing the original VM : > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/mapper/vg_01-lv_sys'/> > > <target dev='vda' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > > </disk> > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/sdc'/> > > <target dev='vdb' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > > </disk> > > > > We have hundreds of virtual machines in production with this type of configuration... How can we migrate them safely to oVirt? > > > > thanks > > > > Nelson > > _______________________________________________ > > Users mailing list > > Users(a)ovirt.org > > http://lists.ovirt.org/mailman/listinfo/users

Well I think I have a rather "classical" environment : This is the test setup I created (from scratch) in order to simulate future migrations (which will be used to migrate our hundreds of vm in production) On one side, I have a x86_64 centos7.2+KVM hosting one VM with 2 iscsi disks : 1 LVM block device /dev/mapper/vg_01-lv_sys, using virtio 1 block device /dev/sdc, using virtio on the other side, I have a x86_64 centos 7.2 oVirt cluster (2 hosts, same hardware as KVM hosts) installed with 4.0.3 oVirt (hosts and engine) fully upgraded to 4.0.4RC1 (hosts and engine) Target would be oVirt iscsi Data Domain disk (which is irrelevant because the problem I'm describing arises before selecting oVirt target) I'm using the GUI to test the native import capabilities of oVirt "Import VM" When selecting KVM import, putting correct URL data (using either tcp or ssh transport), I get a list of VM machines running in the KVM cluster (for this exemple, only the one VM described above), but the disk count is always 0. Which makes it impossible to import the VM. I should add that I made the same VM import using "local hard drives" (ie. local raw file) and it works as expected, I can see one or multiple disks and import them easily to oVirt. The problem seems to manifest itself only when importing disks using block devices directly. Do you need more information from me, or make more tests? thanks a lot, nelson

On 20.09.16 10:24, Nelson Lameiras wrote: > Here is the disk section of my vm, > > ... > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/mapper/vg_01-lv_sys'/> > <target dev='vda' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > </disk> > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/sdc'/> > <target dev='vdb' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > </disk> > ... > > nelson Thanks Nelson, bz 1362525 will not solve your issue. We will start work on that block device issue shortly and I hope that it will make it to the next release. Thank you, Shahar. > > > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org > Sent: Tuesday, September 20, 2016 8:04:41 AM > Subject: Re: [ovirt-users] can't import vm from KVM host > > On 19.09.16 18:11, Nelson Lameiras wrote: > > Well I think I have a rather "classical" environment : > > > > This is the test setup I created (from scratch) in order to simulate future migrations (which will be used to migrate our hundreds of vm in production) > > > > On one side, I have a x86_64 centos7.2+KVM hosting one VM with 2 iscsi disks : > > 1 LVM block device /dev/mapper/vg_01-lv_sys, using virtio > > 1 block device /dev/sdc, using virtio > > > > on the other side, I have a x86_64 centos 7.2 oVirt cluster (2 hosts, same hardware as KVM hosts) installed with 4.0.3 oVirt (hosts and engine) fully upgraded to 4.0.4RC1 (hosts and engine) > > Target would be oVirt iscsi Data Domain disk (which is irrelevant because the problem I'm describing arises before selecting oVirt target) > > > > I'm using the GUI to test the native import capabilities of oVirt "Import VM" > > When selecting KVM import, putting correct URL data (using either tcp or ssh transport), I get a list of VM machines running in the KVM cluster (for this exemple, only the one VM described above), but the disk count is always 0. Which makes it impossible to import the VM. > > > > I should add that I made the same VM import using "local hard drives" (ie. local raw file) and it works as expected, I can see one or multiple disks and import them easily to oVirt. The problem seems to manifest itself only when importing disks using block devices directly. > > > > Do you need more information from me, or make more tests? > > > > thanks a lot, > > nelson > Can you post the libvirt xml as attachment here (at least the disks sections): > > $ virsh -r dumpxml <vmname> > > Shahar.

On 20.09.16 10:24, Nelson Lameiras wrote: > Here is the disk section of my vm, > > ... > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/mapper/vg_01-lv_sys'/> > <target dev='vda' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > </disk> > <disk type='block' device='disk'> > <driver name='qemu' type='raw' cache='none' io='native'/> > <source dev='/dev/sdc'/> > <target dev='vdb' bus='virtio'/> > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > </disk> > ... > > nelson Thanks Nelson, bz 1362525 will not solve your issue. We will start work on that block device issue shortly and I hope that it will make it to the next release. Thank you, Shahar. > > > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org > Sent: Tuesday, September 20, 2016 8:04:41 AM > Subject: Re: [ovirt-users] can't import vm from KVM host > > On 19.09.16 18:11, Nelson Lameiras wrote: > > Well I think I have a rather "classical" environment : > > > > This is the test setup I created (from scratch) in order to simulate future migrations (which will be used to migrate our hundreds of vm in production) > > > > On one side, I have a x86_64 centos7.2+KVM hosting one VM with 2 iscsi disks : > > 1 LVM block device /dev/mapper/vg_01-lv_sys, using virtio > > 1 block device /dev/sdc, using virtio > > > > on the other side, I have a x86_64 centos 7.2 oVirt cluster (2 hosts, same hardware as KVM hosts) installed with 4.0.3 oVirt (hosts and engine) fully upgraded to 4.0.4RC1 (hosts and engine) > > Target would be oVirt iscsi Data Domain disk (which is irrelevant because the problem I'm describing arises before selecting oVirt target) > > > > I'm using the GUI to test the native import capabilities of oVirt "Import VM" > > When selecting KVM import, putting correct URL data (using either tcp or ssh transport), I get a list of VM machines running in the KVM cluster (for this exemple, only the one VM described above), but the disk count is always 0. Which makes it impossible to import the VM. > > > > I should add that I made the same VM import using "local hard drives" (ie. local raw file) and it works as expected, I can see one or multiple disks and import them easily to oVirt. The problem seems to manifest itself only when importing disks using block devices directly. > > > > Do you need more information from me, or make more tests? > > > > thanks a lot, > > nelson > Can you post the libvirt xml as attachment here (at least the disks sections): > > $ virsh -r dumpxml <vmname> > > Shahar.

Shahar, I took the liberty to try to patch our test setup with the fix I took from https://gerrit.ovirt.org/#/c/64272/4/ (lib/vdsm/v2v.py) I restarted vdsm afterwards Result : In the GUI import page, now when clicking on "load" button (just before guetting the list of vm to import on source), an error occurs : "Failed to communicate with the external provider, see log for additional details." Error on engine log : " 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] Failed in 'GetVmsFromExternalProviderVDS' method, for vds: 'virtintelan01.lbg.office.lyra'; host: 'virtintelan01.lbg.office.lyra': null 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] Command 'GetVmsFromExternalProviderVDSCommand(HostName = virtintelan01.lbg.office.lyra, GetVmsFromExternalProviderParameters:{runAsync='true', hostId='20b59a24-0098-461a-b4a3-7d6213b96c52', url='qemu+ssh://root@192.168.210.140/system', username='null', originType='KVM'})' execution failed: null 2016-09-29 14:13:48,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] FINISH, GetVmsFromExternalProviderVDSCommand, log id: 6e062276 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Query 'GetVmsFromExternalProviderQuery' failed: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114) [bll.jar:] at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.runVdsCommand(QueriesCommandBase.java:257) [bll.jar:] ... " In the host, I can see the following errors on /var/log/messages : " Sep 29 14:11:39 virtintelan01 journal: vdsm root ERROR Error getting disk size#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info#012 vol = conn.storageVolLookupByPath(disk['alias'])#012 File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath#012 if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self)#012libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/mapper/vg_00-lv_sys' Sep 29 14:11:39 virtintelan01 journal: vdsm root ERROR Error getting disk size#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info#012 vol = conn.storageVolLookupByPath(disk['alias'])#012 File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath#012 if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self)#012libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/sdc' " FYI "Volume de stockage introuvable" (french) = "storage volume not found" This error did not appear before patching with the fix. I repeated the operation 2 times with a clean setup + patch. Same behaviour. Maybe I'm doing something wrong?

my setup : ovirt-engine : centos 7.2 + ovirt 4.0.4 hosts : centos 7.2 + ovirt 4.0.4 + fix cordialement, regards, Nelson LAMEIRAS Lyra Network Service Projets et Processus Tel : +33 (0) 5 32 09 09 70 109 rue de l’innovation 31670 Labège - France www.lyra-network.com ----- Original Message ----- From: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; To: "Shahar Havivi" <shavivi(a)redhat.com&gt; Cc: users(a)ovirt.org Sent: Friday, September 23, 2016 11:55:08 AM Subject: Re: [ovirt-users] can't import vm from KVM host Shahar, Thanks! I'll keep an eye on this bug and try to be helpfull when testing the fix. nelson ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org Sent: Thursday, September 22, 2016 10:14:33 AM Subject: Re: [ovirt-users] can't import vm from KVM host Nelson, I opened a bug regarding your issue https://bugzilla.redhat.com/show_bug.cgi?id=1378340 And send a fix: https://gerrit.ovirt.org/#/c/64272/ Thank you for your help! Shahar. On 21.09.16 10:57, Shahar Havivi wrote: > On 20.09.16 10:24, Nelson Lameiras wrote: > > Here is the disk section of my vm, > > > > ... > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/mapper/vg_01-lv_sys'/> > > <target dev='vda' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> > > </disk> > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/sdc'/> > > <target dev='vdb' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> > > </disk> > > ... > > > > nelson > Thanks Nelson, > bz 1362525 will not solve your issue. > We will start work on that block device issue shortly and I hope that it will > make it to the next release. > > Thank you, > Shahar. > > > > > > > ----- Original Message ----- > > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > > Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org > > Sent: Tuesday, September 20, 2016 8:04:41 AM > > Subject: Re: [ovirt-users] can't import vm from KVM host > > > > On 19.09.16 18:11, Nelson Lameiras wrote: > > > Well I think I have a rather "classical" environment : > > > > > > This is the test setup I created (from scratch) in order to simulate future migrations (which will be used to migrate our hundreds of vm in production) > > > > > > On one side, I have a x86_64 centos7.2+KVM hosting one VM with 2 iscsi disks : > > > 1 LVM block device /dev/mapper/vg_01-lv_sys, using virtio > > > 1 block device /dev/sdc, using virtio > > > > > > on the other side, I have a x86_64 centos 7.2 oVirt cluster (2 hosts, same hardware as KVM hosts) installed with 4.0.3 oVirt (hosts and engine) fully upgraded to 4.0.4RC1 (hosts and engine) > > > Target would be oVirt iscsi Data Domain disk (which is irrelevant because the problem I'm describing arises before selecting oVirt target) > > > > > > I'm using the GUI to test the native import capabilities of oVirt "Import VM" > > > When selecting KVM import, putting correct URL data (using either tcp or ssh transport), I get a list of VM machines running in the KVM cluster (for this exemple, only the one VM described above), but the disk count is always 0. Which makes it impossible to import the VM. > > > > > > I should add that I made the same VM import using "local hard drives" (ie. local raw file) and it works as expected, I can see one or multiple disks and import them easily to oVirt. The problem seems to manifest itself only when importing disks using block devices directly. > > > > > > Do you need more information from me, or make more tests? > > > > > > thanks a lot, > > > nelson > > Can you post the libvirt xml as attachment here (at least the disks sections): > > > > $ virsh -r dumpxml <vmname> > > > > Shahar. _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

On 7 Oct 2016, at 09:42, Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: hello, any news on this issue ?

can I do anything to help ?

cordialement, regards, Nelson LAMEIRAS Lyra Network Service Projets et Processus Tel : +33 (0) 5 32 09 09 70 109 rue de l’innovation 31670 Labège - France www.lyra-network.com ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; Cc: users(a)ovirt.org Sent: Thursday, September 29, 2016 2:31:42 PM Subject: Re: [ovirt-users] can't import vm from KVM host On 29.09.16 14:22, Nelson Lameiras wrote: > Shahar, > > I took the liberty to try to patch our test setup with the fix I took from https://gerrit.ovirt.org/#/c/64272/4/ (lib/vdsm/v2v.py) > I restarted vdsm afterwards > > Result : > In the GUI import page, now when clicking on "load" button (just before guetting the list of vm to import on source), an error occurs : "Failed to communicate with the external provider, see log for additional details." > > Error on engine log : > > " > 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] Failed in 'GetVmsFromExternalProviderVDS' method, for vds: 'virtintelan01.lbg.office.lyra'; host: 'virtintelan01.lbg.office.lyra': null > 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] Command 'GetVmsFromExternalProviderVDSCommand(HostName = virtintelan01.lbg.office.lyra, GetVmsFromExternalProviderParameters:{runAsync='true', hostId='20b59a24-0098-461a-b4a3-7d6213b96c52', url='qemu+ssh://root@192.168.210.140/system', username='null', originType='KVM'})' execution failed: null > 2016-09-29 14:13:48,637 INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVmsFromExternalProviderVDSCommand] (default task-30) [] FINISH, GetVmsFromExternalProviderVDSCommand, log id: 6e062276 > 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Query 'GetVmsFromExternalProviderQuery' failed: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > at org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114) [bll.jar:] > at org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33) [bll.jar:] > at org.ovirt.engine.core.bll.QueriesCommandBase.runVdsCommand(QueriesCommandBase.java:257) [bll.jar:] > ... > " > > In the host, I can see the following errors on /var/log/messages : > > " > Sep 29 14:11:39 virtintelan01 journal: vdsm root ERROR Error getting disk size#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info#012 vol = conn.storageVolLookupByPath(disk['alias'])#012 File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath#012 if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self)#012libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/mapper/vg_00-lv_sys' > Sep 29 14:11:39 virtintelan01 journal: vdsm root ERROR Error getting disk size#012Traceback (most recent call last):#012 File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info#012 vol = conn.storageVolLookupByPath(disk['alias'])#012 File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath#012 if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self)#012libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/sdc' > " > > FYI "Volume de stockage introuvable" (french) = "storage volume not found" > > This error did not appear before patching with the fix. > > I repeated the operation 2 times with a clean setup + patch. Same behaviour. Maybe I'm doing something wrong? I need to check, Thanks for the info. > > my setup : > ovirt-engine : centos 7.2 + ovirt 4.0.4 > hosts : centos 7.2 + ovirt 4.0.4 + fix > > cordialement, regards, > Nelson LAMEIRAS > > Lyra Network > Service Projets et Processus > Tel : +33 (0) 5 32 09 09 70 > 109 rue de l’innovation > 31670 Labège - France > www.lyra-network.com > > ----- Original Message ----- > From: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > To: "Shahar Havivi" <shavivi(a)redhat.com&gt; > Cc: users(a)ovirt.org > Sent: Friday, September 23, 2016 11:55:08 AM > Subject: Re: [ovirt-users] can't import vm from KVM host > > Shahar, > > Thanks! > I'll keep an eye on this bug and try to be helpfull when testing the fix. > > nelson > > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org > Sent: Thursday, September 22, 2016 10:14:33 AM > Subject: Re: [ovirt-users] can't import vm from KVM host > > Nelson, > I opened a bug regarding your issue > https://bugzilla.redhat.com/show_bug.cgi?id=1378340 > > And send a fix: > https://gerrit.ovirt.org/#/c/64272/ > > Thank you for your help! > > Shahar. > > On 21.09.16 10:57, Shahar Havivi wrote: >> On 20.09.16 10:24, Nelson Lameiras wrote: >>> Here is the disk section of my vm, >>> >>> ... >>> <disk type='block' device='disk'> >>> <driver name='qemu' type='raw' cache='none' io='native'/> >>> <source dev='/dev/mapper/vg_01-lv_sys'/> >>> <target dev='vda' bus='virtio'/> >>> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> >>> </disk> >>> <disk type='block' device='disk'> >>> <driver name='qemu' type='raw' cache='none' io='native'/> >>> <source dev='/dev/sdc'/> >>> <target dev='vdb' bus='virtio'/> >>> <address type='pci' domain='0x0000' bus='0x00' slot='0x08' function='0x0'/> >>> </disk> >>> ... >>> >>> nelson >> Thanks Nelson, >> bz 1362525 will not solve your issue. >> We will start work on that block device issue shortly and I hope that it will >> make it to the next release. >> >> Thank you, >> Shahar. >> >>> >>> >>> ----- Original Message ----- >>> From: "Shahar Havivi" <shavivi(a)redhat.com&gt; >>> To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; >>> Cc: "Nisim Simsolo" <nsimsolo(a)redhat.com&gt;, users(a)ovirt.org >>> Sent: Tuesday, September 20, 2016 8:04:41 AM >>> Subject: Re: [ovirt-users] can't import vm from KVM host >>> >>> On 19.09.16 18:11, Nelson Lameiras wrote: >>>> Well I think I have a rather "classical" environment : >>>> >>>> This is the test setup I created (from scratch) in order to simulate future migrations (which will be used to migrate our hundreds of vm in production) >>>> >>>> On one side, I have a x86_64 centos7.2+KVM hosting one VM with 2 iscsi disks : >>>> 1 LVM block device /dev/mapper/vg_01-lv_sys, using virtio >>>> 1 block device /dev/sdc, using virtio >>>> >>>> on the other side, I have a x86_64 centos 7.2 oVirt cluster (2 hosts, same hardware as KVM hosts) installed with 4.0.3 oVirt (hosts and engine) fully upgraded to 4.0.4RC1 (hosts and engine) >>>> Target would be oVirt iscsi Data Domain disk (which is irrelevant because the problem I'm describing arises before selecting oVirt target) >>>> >>>> I'm using the GUI to test the native import capabilities of oVirt "Import VM" >>>> When selecting KVM import, putting correct URL data (using either tcp or ssh transport), I get a list of VM machines running in the KVM cluster (for this exemple, only the one VM described above), but the disk count is always 0. Which makes it impossible to import the VM. >>>> >>>> I should add that I made the same VM import using "local hard drives" (ie. local raw file) and it works as expected, I can see one or multiple disks and import them easily to oVirt. The problem seems to manifest itself only when importing disks using block devices directly. >>>> >>>> Do you need more information from me, or make more tests? >>>> >>>> thanks a lot, >>>> nelson >>> Can you post the libvirt xml as attachment here (at least the disks sections): >>> >>> $ virsh -r dumpxml <vmname> >>> >>> Shahar. > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

hello michal, Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. Please understand that my setup is the following: source host : KVM (hosting the VM to migrate) target host : oVirt 4.0.4 (patched) The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. Can this be the source of the error ?

2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001)

On 11 Oct 2016, at 16:01, Tomáš Golembiovský <tgolembi(a)redhat.com&gt; wrote: Hi, On Mon, 10 Oct 2016 12:21:47 +0200 (CEST) Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > hello michal, > > Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. > > Please understand that my setup is the following: > source host : KVM (hosting the VM to migrate) > target host : oVirt 4.0.4 (patched) > > The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. > This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. > > Can this be the source of the error ? To me this looks more like a bug in the engine, not in VDSM. The exception in the VDSM log is OK and it is not fatal. It is actually to be expected for block devices. VDSM is trying to find out the size of the disk, but the way it does that is tailored for volumes in storage pool. But the block device is not a volume in a storage pool and that is why libvirt complains. We can potentially skip the check for block devices to avoid error in the log, or provide some other logic of getting the size. But the real problem is this error in engine.log: > 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) We don't send the size (or rather send null value) for the disk and engine does not like that. I assume engine does not really consider all the optional VM properties as optional.

Tomas -- Tomáš Golembiovský <tgolembi(a)redhat.com&gt;

Hi. On Thu, 20 Oct 2016 14:10:15 +0200 (CEST) Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > Hello, > > Any update on this issue? Sorry, no update yet. Unfortunately the guy working on that issue is on holidays. He should be back next week. > Can I do anything to help? Of course we would appreciate any testing. We'll let you know when the updated patch is ready. Hopefully by the end of next week. Best regards, Tomáš Golembiovský _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

Hello, Any update on this issue? Can I do anything to help? cordialement, regards, Nelson LAMEIRAS Lyra Network Service Projets et Processus Tel : +33 (0) 5 32 09 09 70 109 rue de l’innovation 31670 Labège - France www.lyra-network.com ----- Original Message ----- From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt; To: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; Cc: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, users(a)ovirt.org Sent: Tuesday, October 11, 2016 4:11:36 PM Subject: Re: [ovirt-users] can't import vm from KVM host > On 11 Oct 2016, at 16:01, Tomáš Golembiovský <tgolembi(a)redhat.com&gt; wrote: > > Hi, > > On Mon, 10 Oct 2016 12:21:47 +0200 (CEST) > Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > >> hello michal, >> >> Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. >> >> Please understand that my setup is the following: >> source host : KVM (hosting the VM to migrate) >> target host : oVirt 4.0.4 (patched) >> >> The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. >> This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. >> >> Can this be the source of the error ? > > To me this looks more like a bug in the engine, not in VDSM. > > The exception in the VDSM log is OK and it is not fatal. It is actually > to be expected for block devices. VDSM is trying to find out the size of > the disk, but the way it does that is tailored for volumes in storage > pool. But the block device is not a volume in a storage pool and that is > why libvirt complains. We can potentially skip the check for block > devices to avoid error in the log, or provide some other logic of > getting the size. > > > But the real problem is this error in engine.log: > >> 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > > We don't send the size (or rather send null value) for the disk and > engine does not like that. I assume engine does not really consider all > the optional VM properties as optional. is it really optional in schema? even if it is, indeed it’s wrong:) the best would be to handle it on v2v.py side first to avoid libvirt err, but i don’t know if we can get that info via libvirt in any other way:/ > > > > Tomas > > -- > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; _______________________________________________ Users mailing list Users(a)ovirt.org http://lists.ovirt.org/mailman/listinfo/users

Hi Shahar, We try to prioritise vm behaviour predictability over ressources consumption, so "thin provisionning" is not a option for us. "Preallocated" is always selected over default behaviour. Nevertheless, while trying to import a KVM vm (from another host), I get a 0 disk count on the VM, which means I do not event get to the point where I can chose allocation policy (usually next screen). This is true with or without patch proposed below (assuming it has not changed since sept 29).

Can I give you any more information? cordialement, regards, Nelson LAMEIRAS Lyra Network Service Projets et Processus Tel : +33 (0) 5 32 09 09 70 109 rue de l’innovation 31670 Labège - France www.lyra-network.com ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; Cc: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org Sent: Monday, November 7, 2016 1:58:34 PM Subject: Re: [ovirt-users] can't import vm from KVM host Hi Nelson, Sorry for the long wait (I was on PTO). I just test the patch with the following disk xml: <disk type='block' device='disk'> <driver name='qemu' type='raw' cache='none' io='native'/> <source dev='/dev/disk/by-path/...lun0'/> <target dev='vda' bus='virtio'/> <address type='pci' domain='0x0000' bus='0x00' slot='0x07' function='0x0'/> </disk> I did manage to see the disk with its actual size, I was able to import the VM as well when I choose: "Allocation Policy": "Preallocated" (not the "Thin Provision" as is the default. Currently we need to pre allocated kvm block disk and cannot use the thin provision (we may need to disable it in the UI). Did you try that? It may related to the reason that the dev path you have is a /dev/mapper/ and this is the reason that libvirt not returning the disk size? (Tomas?) Shahar. On 20.10.16 14:10, Nelson Lameiras wrote: > Hello, > > Any update on this issue? > Can I do anything to help? > > cordialement, regards, > Nelson LAMEIRAS > > Lyra Network > Service Projets et Processus > Tel : +33 (0) 5 32 09 09 70 > 109 rue de l’innovation > 31670 Labège - France > www.lyra-network.com > > ----- Original Message ----- > From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt; > To: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; > Cc: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, users(a)ovirt.org > Sent: Tuesday, October 11, 2016 4:11:36 PM > Subject: Re: [ovirt-users] can't import vm from KVM host > > > On 11 Oct 2016, at 16:01, Tomáš Golembiovský <tgolembi(a)redhat.com&gt; wrote: > > > > Hi, > > > > On Mon, 10 Oct 2016 12:21:47 +0200 (CEST) > > Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > > > >> hello michal, > >> > >> Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. > >> > >> Please understand that my setup is the following: > >> source host : KVM (hosting the VM to migrate) > >> target host : oVirt 4.0.4 (patched) > >> > >> The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. > >> This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. > >> > >> Can this be the source of the error ? > > > > To me this looks more like a bug in the engine, not in VDSM. > > > > The exception in the VDSM log is OK and it is not fatal. It is actually > > to be expected for block devices. VDSM is trying to find out the size of > > the disk, but the way it does that is tailored for volumes in storage > > pool. But the block device is not a volume in a storage pool and that is > > why libvirt complains. We can potentially skip the check for block > > devices to avoid error in the log, or provide some other logic of > > getting the size. > > > > > > But the real problem is this error in engine.log: > > > >> 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > > > > We don't send the size (or rather send null value) for the disk and > > engine does not like that. I assume engine does not really consider all > > the optional VM properties as optional. > > is it really optional in schema? > even if it is, indeed it’s wrong:) > > the best would be to handle it on v2v.py side first to avoid libvirt err, but i don’t know if we can get that info via libvirt in any other way:/ > > > > > > > > > Tomas > > > > -- > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > _______________________________________________ > Users mailing list > Users(a)ovirt.org > http://lists.ovirt.org/mailman/listinfo/users

On 18.11.16 09:34, Nelson Lameiras wrote: > Hello Shahar, > > I've rebuild my test system and installed recently released ovirt 4.0.5 + gerrit patch => still no luck. vdsm always gives me same error : Hi, yes once you have the alias you are half way there. but your alias path looks wrong, it starts with white-space: '/ var/lib/libvirt/images/rhel7.1.img' check it via 'virsh -r dumpxml <vmnam>' and look if the white space are there two regarding the pool you can check it too via virsh virsh pool-list and all pool-* commands (to create one), if you have problems building pool let me know. Shahar. > > " > jsonrpc.Executor/6::ERROR::2016-11-17 16:38:56,320::v2v::934::root::(_add_disk_info) Error getting disk size > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info > vol = conn.storageVolLookupByPath(disk['alias']) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath > if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self) > libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/sdc' > jsonrpc.Executor/6::DEBUG::2016-11-17 16:38:56,325::__init__::555::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getExternalVMs' in bridge with [{'status': 'Down', 'graphics': 'spice', 'arch': 'x86_64', 'disks': [ {'capacity': '8589934592', 'format': 'RAW', 'dev': 'hdd', 'allocation': '8589942784', 'alias': '/ var/lib/libvirt/images/rhel7.1.img', 'type': 'disk'}, {'alias': '/dev/sdc', 'type': 'disk', 'dev': 'hdb', 'format': 'RAW'}], 'vmId': 'd0ddc4f6-a208-4286-a665-fb9e54d14bef', 'smp': 1, 'has_snapshots': False, 'video': 'qxl', 'memSize': 1 024, 'vmName': 'rhel7.1', 'networks': [{'model': 'virtio', 'macAddr': '52:54:00:00:d7:76', 'type': 'network'}]}] > " > > FYI I added a normal disk (file based) to the VM ir order to try to understand what's happening. As expected, this disk is seen by the oVirt (without patch) > > I do reckon something in the above log : > > 'disks': [ > {'capacity': '8589934592', 'format': 'RAW', 'dev': 'hdd', 'allocation': '8589942784', 'alias': '/ var/lib/libvirt/images/rhel7.1.img', 'type': 'disk'}, > {'alias': '/dev/sdc', 'type': 'disk', 'dev': 'hdb', 'format': 'RAW'} > ], > > the "alias" line is clearly new since it did not appear before de patch! something is working ;) > > I confirm that when I created the test VM (via virt-manager), I gave directly the path to the device (/dev/sdc), so there is no "pool" to be addressed defined on libvirt! > I read https://gerrit.ovirt.org/#/c/64272/4/ and I agree with Tomas findings : Currently the conversion process is expecting a volume on a pool and it's not able to use a block device directly. > > So I'm hopping a solution can be found to this problem. We have a few hundreds VM waiting to be migrated in production and we don't have a NFS server, so this is our only option so far. > > I'm able to test any patch quickly if needed, do not hesitate. > > cordialement, regards, > Nelson LAMEIRAS > > Lyra Network > Service Projets et Processus > Tel : +33 (0) 5 32 09 09 70 > 109 rue de l’innovation > 31670 Labège - France > www.lyra-network.com > > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt;, "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org > Sent: Tuesday, November 8, 2016 12:16:55 PM > Subject: Re: [ovirt-users] can't import vm from KVM host > > On 08.11.16 10:58, Nelson Lameiras wrote: > > Hi Shahar, > > > > We try to prioritise vm behaviour predictability over ressources consumption, so "thin provisionning" is not a option for us. "Preallocated" is always selected over default behaviour. > > > > Nevertheless, while trying to import a KVM vm (from another host), I get a 0 disk count on the VM, which means I do not event get to the point where I can chose allocation policy (usually next screen). > > This is true with or without patch proposed below (assuming it has not changed since sept 29). > > I assume that this line is failing: > vol = conn.storageVolLookupByPath(disk['alias']) > it may be the reason that the storage is not part of a pool. > (look at pool-xxx commands via virsh) > > If it doesn't work it may be related to a different API that we will need to > implement as Tomas suggested in the gerrit patch. > > > > > Can I give you any more information? > > > > cordialement, regards, > > Nelson LAMEIRAS > > > > Lyra Network > > Service Projets et Processus > > Tel : +33 (0) 5 32 09 09 70 > > 109 rue de l’innovation > > 31670 Labège - France > > www.lyra-network.com > > > > ----- Original Message ----- > > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; > > Cc: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org > > Sent: Monday, November 7, 2016 1:58:34 PM > > Subject: Re: [ovirt-users] can't import vm from KVM host > > > > Hi Nelson, > > > > Sorry for the long wait (I was on PTO). > > I just test the patch with the following disk xml: > > > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/disk/by-path/...lun0'/> > > <target dev='vda' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' > > slot='0x07' function='0x0'/> > > </disk> > > > > I did manage to see the disk with its actual size, > > I was able to import the VM as well when I choose: > > "Allocation Policy": "Preallocated" > > (not the "Thin Provision" as is the default. > > Currently we need to pre allocated kvm block disk and cannot use the thin > > provision (we may need to disable it in the UI). > > > > Did you try that? > > It may related to the reason that the dev path you have is a /dev/mapper/ and > > this is the reason that libvirt not returning the disk size? (Tomas?) > > > > Shahar. > > > > > > On 20.10.16 14:10, Nelson Lameiras wrote: > > > Hello, > > > > > > Any update on this issue? > > > Can I do anything to help? > > > > > > cordialement, regards, > > > Nelson LAMEIRAS > > > > > > Lyra Network > > > Service Projets et Processus > > > Tel : +33 (0) 5 32 09 09 70 > > > 109 rue de l’innovation > > > 31670 Labège - France > > > www.lyra-network.com > > > > > > ----- Original Message ----- > > > From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt; > > > To: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; > > > Cc: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, users(a)ovirt.org > > > Sent: Tuesday, October 11, 2016 4:11:36 PM > > > Subject: Re: [ovirt-users] can't import vm from KVM host > > > > > > > On 11 Oct 2016, at 16:01, Tomáš Golembiovský <tgolembi(a)redhat.com&gt; wrote: > > > > > > > > Hi, > > > > > > > > On Mon, 10 Oct 2016 12:21:47 +0200 (CEST) > > > > Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > > > > > > > >> hello michal, > > > >> > > > >> Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. > > > >> > > > >> Please understand that my setup is the following: > > > >> source host : KVM (hosting the VM to migrate) > > > >> target host : oVirt 4.0.4 (patched) > > > >> > > > >> The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. > > > >> This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. > > > >> > > > >> Can this be the source of the error ? > > > > > > > > To me this looks more like a bug in the engine, not in VDSM. > > > > > > > > The exception in the VDSM log is OK and it is not fatal. It is actually > > > > to be expected for block devices. VDSM is trying to find out the size of > > > > the disk, but the way it does that is tailored for volumes in storage > > > > pool. But the block device is not a volume in a storage pool and that is > > > > why libvirt complains. We can potentially skip the check for block > > > > devices to avoid error in the log, or provide some other logic of > > > > getting the size. > > > > > > > > > > > > But the real problem is this error in engine.log: > > > > > > > >> 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > > > > > > > > We don't send the size (or rather send null value) for the disk and > > > > engine does not like that. I assume engine does not really consider all > > > > the optional VM properties as optional. > > > > > > is it really optional in schema? > > > even if it is, indeed it’s wrong:) > > > > > > the best would be to handle it on v2v.py side first to avoid libvirt err, but i don’t know if we can get that info via libvirt in any other way:/ > > > > > > > > > > > > > > > > > > > Tomas > > > > > > > > -- > > > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > > > _______________________________________________ > > > Users mailing list > > > Users(a)ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/users

Hello Shahar, Actually, our production servers use pool storage (+block devices), so I quickly tried to import from them and I have the same error : 0 disks "declared" on each VM. I tried the patch you advised last week, and eventhough it didn't work, clearly the logs showed you are going in the right direction. I'm available to try other patches on my test setup. Do not hesitate to ask me.

cordialement, regards, Nelson LAMEIRAS Lyra Network Service Projets et Processus Tel : +33 (0) 5 32 09 09 70 109 rue de l’innovation 31670 Labège - France www.lyra-network.com ----- Original Message ----- From: "Shahar Havivi" <shavivi(a)redhat.com&gt; To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; Cc: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt;, "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org Sent: Sunday, November 20, 2016 9:36:14 AM Subject: Re: [ovirt-users] can't import vm from KVM host On 18.11.16 09:34, Nelson Lameiras wrote: > Hello Shahar, > > I've rebuild my test system and installed recently released ovirt 4.0.5 + gerrit patch => still no luck. vdsm always gives me same error : Hi, yes once you have the alias you are half way there. but your alias path looks wrong, it starts with white-space: '/ var/lib/libvirt/images/rhel7.1.img' check it via 'virsh -r dumpxml <vmnam>' and look if the white space are there two regarding the pool you can check it too via virsh virsh pool-list and all pool-* commands (to create one), if you have problems building pool let me know. Shahar. > > " > jsonrpc.Executor/6::ERROR::2016-11-17 16:38:56,320::v2v::934::root::(_add_disk_info) Error getting disk size > Traceback (most recent call last): > File "/usr/lib/python2.7/site-packages/vdsm/v2v.py", line 931, in _add_disk_info > vol = conn.storageVolLookupByPath(disk['alias']) > File "/usr/lib64/python2.7/site-packages/libvirt.py", line 4596, in storageVolLookupByPath > if ret is None:raise libvirtError('virStorageVolLookupByPath() failed', conn=self) > libvirtError: Volume de stockage introuvable : no storage vol with matching path '/dev/sdc' > jsonrpc.Executor/6::DEBUG::2016-11-17 16:38:56,325::__init__::555::jsonrpc.JsonRpcServer::(_handle_request) Return 'Host.getExternalVMs' in bridge with [{'status': 'Down', 'graphics': 'spice', 'arch': 'x86_64', 'disks': [ {'capacity': '8589934592', 'format': 'RAW', 'dev': 'hdd', 'allocation': '8589942784', 'alias': '/ var/lib/libvirt/images/rhel7.1.img', 'type': 'disk'}, {'alias': '/dev/sdc', 'type': 'disk', 'dev': 'hdb', 'format': 'RAW'}], 'vmId': 'd0ddc4f6-a208-4286-a665-fb9e54d14bef', 'smp': 1, 'has_snapshots': False, 'video': 'qxl', 'memSize': 1 024, 'vmName': 'rhel7.1', 'networks': [{'model': 'virtio', 'macAddr': '52:54:00:00:d7:76', 'type': 'network'}]}] > " > > FYI I added a normal disk (file based) to the VM ir order to try to understand what's happening. As expected, this disk is seen by the oVirt (without patch) > > I do reckon something in the above log : > > 'disks': [ > {'capacity': '8589934592', 'format': 'RAW', 'dev': 'hdd', 'allocation': '8589942784', 'alias': '/ var/lib/libvirt/images/rhel7.1.img', 'type': 'disk'}, > {'alias': '/dev/sdc', 'type': 'disk', 'dev': 'hdb', 'format': 'RAW'} > ], > > the "alias" line is clearly new since it did not appear before de patch! something is working ;) > > I confirm that when I created the test VM (via virt-manager), I gave directly the path to the device (/dev/sdc), so there is no "pool" to be addressed defined on libvirt! > I read https://gerrit.ovirt.org/#/c/64272/4/ and I agree with Tomas findings : Currently the conversion process is expecting a volume on a pool and it's not able to use a block device directly. > > So I'm hopping a solution can be found to this problem. We have a few hundreds VM waiting to be migrated in production and we don't have a NFS server, so this is our only option so far. > > I'm able to test any patch quickly if needed, do not hesitate. > > cordialement, regards, > Nelson LAMEIRAS > > Lyra Network > Service Projets et Processus > Tel : +33 (0) 5 32 09 09 70 > 109 rue de l’innovation > 31670 Labège - France > www.lyra-network.com > > ----- Original Message ----- > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt; > Cc: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt;, "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org > Sent: Tuesday, November 8, 2016 12:16:55 PM > Subject: Re: [ovirt-users] can't import vm from KVM host > > On 08.11.16 10:58, Nelson Lameiras wrote: > > Hi Shahar, > > > > We try to prioritise vm behaviour predictability over ressources consumption, so "thin provisionning" is not a option for us. "Preallocated" is always selected over default behaviour. > > > > Nevertheless, while trying to import a KVM vm (from another host), I get a 0 disk count on the VM, which means I do not event get to the point where I can chose allocation policy (usually next screen). > > This is true with or without patch proposed below (assuming it has not changed since sept 29). > > I assume that this line is failing: > vol = conn.storageVolLookupByPath(disk['alias']) > it may be the reason that the storage is not part of a pool. > (look at pool-xxx commands via virsh) > > If it doesn't work it may be related to a different API that we will need to > implement as Tomas suggested in the gerrit patch. > > > > > Can I give you any more information? > > > > cordialement, regards, > > Nelson LAMEIRAS > > > > Lyra Network > > Service Projets et Processus > > Tel : +33 (0) 5 32 09 09 70 > > 109 rue de l’innovation > > 31670 Labège - France > > www.lyra-network.com > > > > ----- Original Message ----- > > From: "Shahar Havivi" <shavivi(a)redhat.com&gt; > > To: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; > > Cc: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt;, users(a)ovirt.org > > Sent: Monday, November 7, 2016 1:58:34 PM > > Subject: Re: [ovirt-users] can't import vm from KVM host > > > > Hi Nelson, > > > > Sorry for the long wait (I was on PTO). > > I just test the patch with the following disk xml: > > > > <disk type='block' device='disk'> > > <driver name='qemu' type='raw' cache='none' io='native'/> > > <source dev='/dev/disk/by-path/...lun0'/> > > <target dev='vda' bus='virtio'/> > > <address type='pci' domain='0x0000' bus='0x00' > > slot='0x07' function='0x0'/> > > </disk> > > > > I did manage to see the disk with its actual size, > > I was able to import the VM as well when I choose: > > "Allocation Policy": "Preallocated" > > (not the "Thin Provision" as is the default. > > Currently we need to pre allocated kvm block disk and cannot use the thin > > provision (we may need to disable it in the UI). > > > > Did you try that? > > It may related to the reason that the dev path you have is a /dev/mapper/ and > > this is the reason that libvirt not returning the disk size? (Tomas?) > > > > Shahar. > > > > > > On 20.10.16 14:10, Nelson Lameiras wrote: > > > Hello, > > > > > > Any update on this issue? > > > Can I do anything to help? > > > > > > cordialement, regards, > > > Nelson LAMEIRAS > > > > > > Lyra Network > > > Service Projets et Processus > > > Tel : +33 (0) 5 32 09 09 70 > > > 109 rue de l’innovation > > > 31670 Labège - France > > > www.lyra-network.com > > > > > > ----- Original Message ----- > > > From: "Michal Skrivanek" <michal.skrivanek(a)redhat.com&gt; > > > To: "Tomas Golembiovsky" <tgolembi(a)redhat.com&gt; > > > Cc: "Nelson Lameiras" <nelson.lameiras(a)lyra-network.com&gt;, users(a)ovirt.org > > > Sent: Tuesday, October 11, 2016 4:11:36 PM > > > Subject: Re: [ovirt-users] can't import vm from KVM host > > > > > > > On 11 Oct 2016, at 16:01, Tomáš Golembiovský <tgolembi(a)redhat.com&gt; wrote: > > > > > > > > Hi, > > > > > > > > On Mon, 10 Oct 2016 12:21:47 +0200 (CEST) > > > > Nelson Lameiras <nelson.lameiras(a)lyra-network.com&gt; wrote: > > > > > > > >> hello michal, > > > >> > > > >> Yes, both paths are correct and working on the source host since the VM starts and works correctly on source host. Nevertheless I have checked with fdisk and mount to assure that these devices are indeed reachable. > > > >> > > > >> Please understand that my setup is the following: > > > >> source host : KVM (hosting the VM to migrate) > > > >> target host : oVirt 4.0.4 (patched) > > > >> > > > >> The error "no storage vol with matching path '/dev/sdc'" is returned by the target host script /usr/lib/python2.7/site-packages/vdsm/v2v.py. > > > >> This almost seems normal since the device /dev/sdc is not mounted on target host (where script is running) but only on source host. > > > >> > > > >> Can this be the source of the error ? > > > > > > > > To me this looks more like a bug in the engine, not in VDSM. > > > > > > > > The exception in the VDSM log is OK and it is not fatal. It is actually > > > > to be expected for block devices. VDSM is trying to find out the size of > > > > the disk, but the way it does that is tailored for volumes in storage > > > > pool. But the block device is not a volume in a storage pool and that is > > > > why libvirt complains. We can potentially skip the check for block > > > > devices to avoid error in the log, or provide some other logic of > > > > getting the size. > > > > > > > > > > > > But the real problem is this error in engine.log: > > > > > > > >> 2016-09-29 14:13:48,637 ERROR [org.ovirt.engine.core.bll.GetVmsFromExternalProviderQuery] (default task-30) [] Exception: org.ovirt.engine.core.common.errors.EngineException: EngineException: java.lang.NumberFormatException: null (Failed with error ENGINE and code 5001) > > > > > > > > We don't send the size (or rather send null value) for the disk and > > > > engine does not like that. I assume engine does not really consider all > > > > the optional VM properties as optional. > > > > > > is it really optional in schema? > > > even if it is, indeed it’s wrong:) > > > > > > the best would be to handle it on v2v.py side first to avoid libvirt err, but i don’t know if we can get that info via libvirt in any other way:/ > > > > > > > > > > > > > > > > > > > Tomas > > > > > > > > -- > > > > Tomáš Golembiovský <tgolembi(a)redhat.com&gt; > > > _______________________________________________ > > > Users mailing list > > > Users(a)ovirt.org > > > http://lists.ovirt.org/mailman/listinfo/users