Re: [ovirt-users] disk not bootable

On Mon, Jul 18, 2016 at 6:48 PM, Fernando Fuentes <ffuentes(a)darktcp.net&gt; wrote: > Nir, > > As requested: > > [root@gamma ~]# lsblk > NAME > MAJ:MIN RM > SIZE RO TYPE MOUNTPOINT > sda > 8:0 0 > 557G 0 disk > ├─sda1 > 8:1 0 > 500M 0 part /boot > └─sda2 > 8:2 0 > 556.5G 0 part > ├─vg_gamma-lv_root (dm-0) > 253:0 0 > 50G 0 lvm / > ├─vg_gamma-lv_swap (dm-1) > 253:1 0 > 4G 0 lvm [SWAP] > └─vg_gamma-lv_home (dm-2) > 253:2 0 > 502.4G 0 lvm /home > sr0 > 11:0 1 > 1024M 0 rom > sdb > 8:16 0 > 2T 0 disk > └─36589cfc000000881b9b93c2623780840 (dm-4) > 253:4 0 > 2T 0 mpath > sdc > 8:32 0 > 2T 0 disk > └─36589cfc00000050564002c7e51978316 (dm-3) > 253:3 0 > 2T 0 mpath > ├─3ccb7b67--8067--4315--9656--d68ba10975ba-metadata (dm-5) > 253:5 0 > 512M 0 lvm > ├─3ccb7b67--8067--4315--9656--d68ba10975ba-outbox (dm-6) > 253:6 0 > 128M 0 lvm > ├─3ccb7b67--8067--4315--9656--d68ba10975ba-leases (dm-7) > 253:7 0 > 2G 0 lvm > ├─3ccb7b67--8067--4315--9656--d68ba10975ba-ids (dm-8) > 253:8 0 > 128M 0 lvm > ├─3ccb7b67--8067--4315--9656--d68ba10975ba-inbox (dm-9) > 253:9 0 > 128M 0 lvm > └─3ccb7b67--8067--4315--9656--d68ba10975ba-master (dm-10) > 253:10 0 > 1G 0 lvm > sdd > 8:48 0 > 4T 0 disk > └─36589cfc00000059ccab70662b71c47ef (dm-11) > 253:11 0 > 4T 0 mpath > ├─4861322b--352f--41c6--890a--5cbf1c2c1f01-metadata (dm-12) > 253:12 0 > 512M 0 lvm > ├─4861322b--352f--41c6--890a--5cbf1c2c1f01-ids (dm-13) > 253:13 0 > 128M 0 lvm > ├─4861322b--352f--41c6--890a--5cbf1c2c1f01-leases (dm-14) > 253:14 0 > 2G 0 lvm > ├─4861322b--352f--41c6--890a--5cbf1c2c1f01-outbox (dm-15) > 253:15 0 > 128M 0 lvm > ├─4861322b--352f--41c6--890a--5cbf1c2c1f01-inbox (dm-16) > 253:16 0 > 128M 0 lvm > └─4861322b--352f--41c6--890a--5cbf1c2c1f01-master (dm-17) > 253:17 0 > 1G 0 lvm So you have 2 storage domains: - 3ccb7b67-8067-4315-9656-d68ba10975ba - 4861322b-352f-41c6-890a-5cbf1c2c1f01 But most likely both of them are not active now. Can you share the output of: iscsiadm -m session On a system connected to iscsi storage you will see something like: # iscsiadm -m session tcp: [5] 10.35.0.99:3260,1 iqn.2003-01.org.dumbo.target1 (non-flash) The special lvs (ids, leases, ...) should be active, and you should see also regular disks lvs (used for snapshot for vm disks). Here is an example from machine connected to active iscsi domain: # lvs LV VG Attr LSize 27c4c795-bca4-4d7b-9b40-cda9098790f5 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 1.00g 35be1f52-5b28-4c90-957a-710dbbb8f13f 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 8.00g 36d9b41b-4b01-4fc2-8e93-ccf79af0f766 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 128.00m 4fda3b44-27a5-4ce4-b8c3-66744aa9937b 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 2.12g c2e78f72-d499-44f0-91f5-9930a599dc87 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 128.00m d49919b4-30fc-440f-9b21-3367ddfdf396 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 4.00g f3b10280-43ed-4772-b122-18c92e098171 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 1.00g f409cc48-8248-4239-a4ea-66b0b1084416 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi------- 1.00g ids 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-ao---- 128.00m inbox 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-a----- 128.00m leases 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-a----- 2.00g master 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-a----- 1.00g metadata 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-a----- 512.00m outbox 5f35b5c0-17d7-4475-9125- e97f1cdb06f9 -wi-a----- 128.00m > [root@gamma ~]# > > Regards, > > -- > Fernando Fuentes > ffuentes(a)txweather.org > http://www.txweather.org > > On Mon, Jul 18, 2016, at 07:43 AM, Nir Soffer wrote: >> Can you share output of lsblk on this host? >> >> On Mon, Jul 18, 2016 at 3:52 AM, Fernando Fuentes >> <ffuentes(a)darktcp.net&gt; >> wrote: >> > Nir, >> > >> > That's odd. gamma is my iscsi host, its in up state and it has >> > active >> > VM's. >> > What am I missing? >> > >> > Regards, >> > >> > -- >> > Fernando Fuentes >> > ffuentes(a)txweather.org >> > http://www.txweather.org >> > >> > On Sun, Jul 17, 2016, at 07:24 PM, Nir Soffer wrote: >> >> On Sun, Jul 17, 2016 at 1:24 AM, Fernando Fuentes >> >> <ffuentes(a)darktcp.net&gt; >> >> wrote: >> >> > Nir, >> >> > >> >> > Ok I got the uuid but I am getting the same results as before. >> >> > Nothing comes up. >> >> > >> >> > [root@gamma ~]# pvscan --cache >> >> > [root@gamma ~]# lvs -o vg_name,lv_name,tags | grep >> >> > 3b7d9349-9eb1-42f8-9e04-7bbb97c02b98 >> >> > [root@gamma ~]# >> >> > >> >> > without the grep all I get is: >> >> > >> >> > [root@gamma ~]# lvs -o vg_name,lv_name,tags >> >> > VG LV LV Tags >> >> > vg_gamma lv_home >> >> > vg_gamma lv_root >> >> > vg_gamma lv_swap >> >> >> >> You are not connected to the iscsi storage domain. >> >> >> >> Please try this from a host in up state in engine. >> >> >> >> Nir >> >> >> >> > >> >> > On the other hand an fdisk shows a bunch of disks and here is >> >> > one >> >> > example: >> >> > >> >> > Disk /dev/mapper/36589cfc00000050564002c7e51978316: 2199.0 GB, >> >> > 2199023255552 bytes >> >> > 255 heads, 63 sectors/track, 267349 cylinders >> >> > Units = cylinders of 16065 * 512 = 8225280 bytes >> >> > Sector size (logical/physical): 512 bytes / 32768 bytes >> >> > I/O size (minimum/optimal): 32768 bytes / 1048576 bytes >> >> > Disk identifier: 0x00000000 >> >> > >> >> > >> >> > Disk /dev/mapper/36589cfc000000881b9b93c2623780840: 2199.0 GB, >> >> > 2199023255552 bytes >> >> > 255 heads, 63 sectors/track, 267349 cylinders >> >> > Units = cylinders of 16065 * 512 = 8225280 bytes >> >> > Sector size (logical/physical): 512 bytes / 32768 bytes >> >> > I/O size (minimum/optimal): 32768 bytes / 1048576 bytes >> >> > Disk identifier: 0x00000000 >> >> > >> >> > >> >> > Disk /dev/mapper/3ccb7b67--8067--4315--9656--d68ba10975ba- >> >> > metadata: 536 >> >> > MB, 536870912 bytes >> >> > 255 heads, 63 sectors/track, 65 cylinders >> >> > Units = cylinders of 16065 * 512 = 8225280 bytes >> >> > Sector size (logical/physical): 512 bytes / 32768 bytes >> >> > I/O size (minimum/optimal): 32768 bytes / 1048576 bytes >> >> > Disk identifier: 0x00000000 >> >> > >> >> > Disk /dev/mapper/4861322b--352f--41c6--890a--5cbf1c2c1f01- >> >> > master: 1073 >> >> > MB, 1073741824 bytes >> >> > 255 heads, 63 sectors/track, 130 cylinders >> >> > Units = cylinders of 16065 * 512 = 8225280 bytes >> >> > Sector size (logical/physical): 512 bytes / 32768 bytes >> >> > I/O size (minimum/optimal): 32768 bytes / 1048576 bytes >> >> > Disk identifier: 0x00000000 >> >> > >> >> > Regards, >> >> > >> >> > -- >> >> > Fernando Fuentes >> >> > ffuentes(a)txweather.org >> >> > http://www.txweather.org >> >> > >> >> > On Sat, Jul 16, 2016, at 04:25 PM, Fernando Fuentes wrote: >> >> >> Nir, >> >> >> >> >> >> Ok ill look for it here in a few. >> >> >> Thanks for your reply and help! >> >> >> >> >> >> -- >> >> >> Fernando Fuentes >> >> >> ffuentes(a)txweather.org >> >> >> http://www.txweather.org >> >> >> >> >> >> On Sat, Jul 16, 2016, at 04:16 PM, Nir Soffer wrote: >> >> >> > On Fri, Jul 15, 2016 at 3:50 PM, Fernando Fuentes >> >> >> > <ffuentes(a)darktcp.net&gt; >> >> >> > wrote: >> >> >> > > Nir, >> >> >> > > >> >> >> > > I try to follow your steps but I cant seem to find the ID >> >> >> > > of the >> >> >> > > template. >> >> >> > >> >> >> > The image-uuid of the template is displayed in the Disks >> >> >> > tab in engine. >> >> >> > >> >> >> > To find the volume-uuid on block storage, you can do: >> >> >> > >> >> >> > pvscan --cache >> >> >> > lvs -o vg_name,lv_name,tags | grep image-uuid >> >> >> > >> >> >> > > >> >> >> > > Regards, >> >> >> > > >> >> >> > > -- >> >> >> > > Fernando Fuentes >> >> >> > > ffuentes(a)txweather.org >> >> >> > > http://www.txweather.org >> >> >> > > >> >> >> > > On Sun, Jul 10, 2016, at 02:15 PM, Nir Soffer wrote: >> >> >> > >> On Thu, Jul 7, 2016 at 7:46 PM, Melissa Mesler >> >> >> > >> <melissa(a)justmelly.com&gt; >> >> >> > >> wrote: >> >> >> > >> > All, I did a test for Fernando in our ovirt >> >> >> > >> > environment. I created a vm >> >> >> > >> > called win7melly in the nfs domain. I then migrated it >> >> >> > >> > to the iscsi >> >> >> > >> > domain. It booted without any issue. So it has to be >> >> >> > >> > something with the >> >> >> > >> > templates. I have attached the vdsm log for the host >> >> >> > >> > the vm resides on. >> >> >> > >> >> >> >> > >> The log show a working vm, so it does not help much. >> >> >> > >> >> >> >> > >> I think that the template you copied from the nfs domain >> >> >> > >> to the block >> >> >> > >> domain is >> >> >> > >> corrupted, or the volume metadata are incorrect. >> >> >> > >> >> >> >> > >> If I understand this correctly, this started when >> >> >> > >> Fernando could not copy >> >> >> > >> the vm >> >> >> > >> disk to the block storage, and I guess the issue was >> >> >> > >> that the template >> >> >> > >> was missing >> >> >> > >> on that storage domain. I assume that he copied the >> >> >> > >> template to the >> >> >> > >> block storage >> >> >> > >> domain by opening the templates tab, selecting the >> >> >> > >> template, and choosing >> >> >> > >> copy >> >> >> > >> from the menu. >> >> >> > >> >> >> >> > >> Lets compare the template on both nfs and block storage >> >> >> > >> domain. >> >> >> > >> >> >> >> > >> 1. Find the template on the nfs storage domain, using >> >> >> > >> the image uuid in >> >> >> > >> engine. >> >> >> > >> >> >> >> > >> It should be at >> >> >> > >> >> >> >> > >> /rhev/data-center/mnt/server:_path/domain- >> >> >> > >> uuid/images/image-uuid/volume-uuid >> >> >> > >> >> >> >> > >> 2. Please share the output of: >> >> >> > >> >> >> >> > >> cat /path/to/volume.meta >> >> >> > >> qemu-img info /path/to/volume >> >> >> > >> qemu-img check /path/to/volume >> >> >> > >> >> >> >> > >> 4. Find the template on the block storage domain >> >> >> > >> >> >> >> > >> You should have an lv using the same volume uuid and the >> >> >> > >> image-uuid >> >> >> > >> should be in the lv tag. >> >> >> > >> >> >> >> > >> Find it using: >> >> >> > >> >> >> >> > >> lvs -o vg_name,lv_name,tags | grep volume-uuid >> >> >> > >> >> >> >> > >> 5. Activate the lv >> >> >> > >> >> >> >> > >> lvchange -ay vg_name/lv_name >> >> >> > >> >> >> >> > >> 6. Share the output of >> >> >> > >> >> >> >> > >> qemu-img info /dev/vg_name/lv_name >> >> >> > >> qemu-img check /dev/vg_name/lv_name >> >> >> > >> >> >> >> > >> 7. Deactivate the lv >> >> >> > >> >> >> >> > >> lvchange -an vg_name/lv_name >> >> >> > >> >> >> >> > >> 8. Find the lv metadata >> >> >> > >> >> >> >> > >> The metadata is stored in /dev/vg_name/metadata. To find >> >> >> > >> the correct >> >> >> > >> block, >> >> >> > >> find the tag named MD_N in the lv tags you found in step >> >> >> > >> 4 >> >> >> > >> >> >> >> > >> The block we need is located at offset N from start of >> >> >> > >> volume. >> >> >> > >> >> >> >> > >> 9. Share the output of: >> >> >> > >> >> >> >> > >> dd if=/dev/vg_name/metadata bs=512 skip=N count=1 >> >> >> > >> iflag=direct >> >> >> > >> >> >> >> > >> The output of this command should show the image-uuid. >> >> >> > >> >> >> >> > >> Nir >> >> >> > >> >> >> >> > >> > >> >> >> > >> > - MeLLy >> >> >> > >> > >> >> >> > >> > On Mon, Jul 4, 2016, at 11:52 PM, Fernando Fuentes >> >> >> > >> > wrote: >> >> >> > >> >> Nir, >> >> >> > >> >> >> >> >> > >> >> That's exactly how I did it Nir. >> >> >> > >> >> I will test tomorrow with a new Windows VM and report >> >> >> > >> >> back. >> >> >> > >> >> >> >> >> > >> >> Regards, >> >> >> > >> >> >> >> >> > >> >> -- >> >> >> > >> >> Fernando Fuentes >> >> >> > >> >> ffuentes(a)txweather.org >> >> >> > >> >> http://www.txweather.org >> >> >> > >> >> >> >> >> > >> >> On Mon, Jul 4, 2016, at 10:48 AM, Nir Soffer wrote: >> >> >> > >> >> > On Mon, Jul 4, 2016 at 6:43 PM, Francesco Romani >> >> >> > >> >> > <fromani(a)redhat.com&gt; >> >> >> > >> >> > wrote: >> >> >> > >> >> > > ----- Original Message ----- >> >> >> > >> >> > >> From: "Nir Soffer" <nsoffer(a)redhat.com&gt; >> >> >> > >> >> > >> To: "Fernando Fuentes" <ffuentes(a)darktcp.net&gt; >> >> >> > >> >> > >> Cc: "Francesco Romani" <fromani(a)redhat.com&gt;, >> >> >> > >> >> > >> "users" <users(a)ovirt.org&gt; >> >> >> > >> >> > >> Sent: Saturday, July 2, 2016 11:18:01 AM >> >> >> > >> >> > >> Subject: Re: [ovirt-users] disk not bootable >> >> >> > >> >> > >> >> >> >> > >> >> > >> On Sat, Jul 2, 2016 at 1:33 AM, Fernando Fuentes >> >> >> > >> >> > >> <ffuentes(a)darktcp.net&gt; >> >> >> > >> >> > >> wrote: >> >> >> > >> >> > >> > Nir, >> >> >> > >> >> > >> > >> >> >> > >> >> > >> > Ok I ran another test and this one I moved >> >> >> > >> >> > >> > from NFS domain to iSCSI and >> >> >> > >> >> > >> > stop working than I moved it back and still >> >> >> > >> >> > >> > unable to run... Windows VM >> >> >> > >> >> > >> > is saying "no available boot disk" >> >> >> > >> >> > >> > VM: Win7-Test >> >> >> > >> >> > >> > Host: Zeta >> >> >> > >> >> > >> > Info as requested: >> >> >> > >> >> > >> > http://pastebin.com/1fSi3auz >> >> >> > >> >> > >> >> >> >> > >> >> > >> We need a working xml to compare to. >> >> >> > >> >> > > >> >> >> > >> >> > > [snip expected changes] >> >> >> > >> >> > > >> >> >> > >> >> > > >> >> >> > >> >> > >> <entry name="manufacturer">oVirt</entry> >> >> >> > >> >> > >> <entry name="product">oVirt Node</entry> >> >> >> > >> >> > >> <entry name="version">6- >> >> >> > >> >> > >> 5.el6.centos.11.2</entry> >> >> >> > >> >> > >> - <entry name="serial">C938F077-55E2-3E50-A694- >> >> >> > >> >> > >> 9FCB7661FD89</entry> >> >> >> > >> >> > >> + <entry name="serial">735C7A01-1F16-3CF0-AF8C- >> >> >> > >> >> > >> A99823E95AC0</entry> >> >> >> > >> >> > >> >> >> >> > >> >> > >> Not expected - maybe this is confusing windows? >> >> >> > >> >> > >> >> >> >> > >> >> > >> Francesco, why vm serial has changed after >> >> >> > >> >> > >> moving disks from one storage >> >> >> > >> >> > >> domain >> >> >> > >> >> > >> to another? >> >> >> > >> >> > > >> >> >> > >> >> > > We put in serial either >> >> >> > >> >> > > 1. the UUID Engine send to us >> >> >> > >> >> > > 2. the host UUID as returned by our getHostUUID >> >> >> > >> >> > > utility function >> >> >> > >> >> > > >> >> >> > >> >> > > the latter is unlikely to change, even after this >> >> >> > >> >> > > disk move. >> >> >> > >> >> > >> >> >> > >> >> > Fernando, can you describe exactly how you moved >> >> >> > >> >> > the disk? >> >> >> > >> >> > >> >> >> > >> >> > I assume that you selected the vm in the virtual >> >> >> > >> >> > machines tab, then >> >> >> > >> >> > selected >> >> >> > >> >> > disks from the sub tab, then selected move, and >> >> >> > >> >> > selected the target >> >> >> > >> >> > storage domain. >> >> >> > >> >> > >> >> >> > >> >> > Also, can you reproduce this with a new vm? (create >> >> >> > >> >> > vm with disk nfs, >> >> >> > >> >> > stop vm, >> >> >> > >> >> > move disk to iscsi, start vm). >> >> >> > >> >> > >> >> >> > >> >> > > So the first suspect in line is Engine >> >> >> > >> >> > > >> >> >> > >> >> > > Arik, do you know if Engine is indeed supposed to >> >> >> > >> >> > > change the UUID in this flow? >> >> >> > >> >> > > That seems very surprising. >> >> >> > >> >> > > >> >> >> > >> >> > > Thanks and bests, >> >> >> > >> >> > > >> >> >> > >> >> > > -- >> >> >> > >> >> > > Francesco Romani >> >> >> > >> >> > > RedHat Engineering Virtualization R & D >> >> >> > >> >> > > Phone: 8261328 >> >> >> > >> >> > > IRC: fromani >> >> >> > >> >> _______________________________________________ >> >> >> > >> >> Users mailing list >> >> >> > >> >> Users(a)ovirt.org >> >> >> > >> >> http://lists.ovirt.org/mailman/listinfo/users >> >> >> > >> > >> >> >> > >> > _______________________________________________ >> >> >> > >> > Users mailing list >> >> >> > >> > Users(a)ovirt.org >> >> >> > >> > http://lists.ovirt.org/mailman/listinfo/users >> >> >> > >> > >> >> >> _______________________________________________ >> >> >> Users mailing list >> >> >> Users(a)ovirt.org >> >> >> http://lists.ovirt.org/mailman/listinfo/users >> >> > _______________________________________________ >> >> > Users mailing list >> >> > Users(a)ovirt.org >> >> > http://lists.ovirt.org/mailman/listinfo/users