- Users - oVirt List Archives

Could not associate brick
by Davide Ferrari 14 Oct '16

14 Oct '16

Hello I'm seeing several ot these warnings: 2016-10-14 10:49:23,721 WARN [org.ovirt.engine.core.vdsbroker.gluster.GlusterVolumesListReturnForXmlRpc] (DefaultQuartzScheduler7) [] Could not associate brick 'vm04.storage.billy:/gluster/ssd/data/brick' of volume '23f8f1ae-a3ac-47bf-8223-5b5f7c29e508' with correct network as no gluster network found in cluster '00000002-0002-0002-0002-000000000345' I was reading this May thread http://lists.ovirt.org/pipermail/users/2016-May/040069.html but there is no final answer about how to solve this warning. As the other poster, I created the gluster network before installing ovirt (it's an hosted-engine installation) and I used DNS names to point to the bricks. DNS resolution is working correctly (every host has everthing in /etc/hosts). I also created a non-VM network and assigned it to every host's bond1 (the gluster network) How can I fix this? TIA -- Davide Ferrari Senior Systems Engineer

2 2

Problem with backing-file, how to fix the backing-chain ?
by Claudio Soprano 14 Oct '16

14 Oct '16

Hi all, We run an ovirt environment before with engine v3.6.5 (if remember good) and now with v4.0.4 (we upgraded because we read the bug with backing-file was resolved with v4). We upgraded some of the hosts machines (but not all still) at v4.0.4 too to see if this would fix the problem, but nothing. The problem is that we have several VMs with snapshots, we do daily, weekly and monthly snapshots, keep some of them (usually the fresh ones) and remove the olds (that in the case they are weekly snapshots, they are in the middle of a series of snapshots), this in the time has produced the famous Backing file too long bug. So we upgraded the engine from 3.6.5 to 4.0.4 (latest available). We discovered this bug, when we tried to upgrade an host to v4.0.4, doing so a VM in the host didn't migrate, so we shutdown it and tried to run on another host, but never succeded for the bug. We don't know if we have more VMs in this situation because we upgraded only 2 hosts on 10. Investigating the problem we discovered that the backing file indicated in each of LVM snapshots report a path very long with /dev/storage-domain-id/../image-group-id/ with ../image-group-id/ repeated a lot of times and at the end /parentid. So to understand which was the right path that it would contain, we cloned a VM in the v4.0.4 and then we did 4 snapshots, now the backing file path is /dev/storage-domain-id/parentid Is there a way to modify the path in the backing-file or a way to recover the VM from this state ? Where do reside the informations about the backing-file path ? I attach here all the commands we run On the ovirt manager (host with the engine only) we run ovirt-shell [oVirt shell (connected)]# list disks --parent-vm-name vm1 id : 2df25a13-6958-40a8-832f-9a26ce65de0f name : vm1_Disk2 id : 8cda0aa6-9e25-4b50-ba00-b877232a1983 name : vm1_Disk1 [oVirt shell (connected)]# show disk 8cda0aa6-9e25-4b50-ba00-b877232a1983 id : 8cda0aa6-9e25-4b50-ba00-b877232a1983 name : vm1_Disk1 actual_size : 1073741824 alias : vm1_Disk1 disk_profile-id : 1731f79a-5034-4270-9a87-94d93025deac format : cow image_id : 7b354e2a-2099-4f2a-80b7-fba7d1fd13ee propagate_errors : False provisioned_size : 17179869184 shareable : False size : 17179869184 sparse : True status-state : ok storage_domains-storage_domain-id: 384f9059-ef2f-4d43-a54f-de71c5d589c8 storage_type : image wipe_after_delete : False [root@ovc1mgr ~]# su - postgres Last login: Fri Oct 14 01:02:14 CEST 2016 -bash-4.2$ psql -d engine -U postgres psql (9.2.15) Type "help" for help. engine=#\x on engine=# select * from images where image_group_id = '8cda0aa6-9e25-4b50-ba00-b877232a1983' order by creation_date; -[ RECORD 1 ]---------+------------------------------------- image_guid | 60ba7acf-58cb-475b-b9ee-15b1be99fee6 creation_date | 2016-03-29 15:12:34+02 size | 17179869184 it_guid | 00000000-0000-0000-0000-000000000000 parentid | 00000000-0000-0000-0000-000000000000 imagestatus | 4 lastmodified | 2016-04-21 11:25:59.972+02 vm_snapshot_id | 27c187cd-989f-4f7a-ac05-49c4410de6c2 volume_type | 1 volume_format | 5 image_group_id | 8cda0aa6-9e25-4b50-ba00-b877232a1983 _create_date | 2016-03-29 15:12:31.994065+02 _update_date | 2016-09-04 01:10:08.773649+02 active | f volume_classification | 1 -[ RECORD 2 ]---------+------------------------------------- image_guid | 68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e creation_date | 2016-07-03 01:01:30+02 size | 17179869184 it_guid | 00000000-0000-0000-0000-000000000000 parentid | 60ba7acf-58cb-475b-b9ee-15b1be99fee6 imagestatus | 1 lastmodified | 2016-07-04 01:03:33.732+02 vm_snapshot_id | 175c2071-a06b-4b0e-a069-5cc4bb236a34 volume_type | 2 volume_format | 4 image_group_id | 8cda0aa6-9e25-4b50-ba00-b877232a1983 _create_date | 2016-07-03 01:01:15.069585+02 _update_date | 2016-09-11 02:06:04.420965+02 active | f volume_classification | 1 -[ RECORD 3 ]---------+------------------------------------- image_guid | 37ca6494-e990-44e5-8597-28845a0a19b5 creation_date | 2016-08-07 01:06:15+02 size | 17179869184 it_guid | 00000000-0000-0000-0000-000000000000 parentid | 68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e imagestatus | 1 lastmodified | 2016-08-08 01:00:03.778+02 vm_snapshot_id | 4c0e5ac0-2ef3-4996-b3e9-7fd566d97b1a volume_type | 2 volume_format | 4 image_group_id | 8cda0aa6-9e25-4b50-ba00-b877232a1983 _create_date | 2016-08-07 01:06:01.777156+02 _update_date | 2016-09-25 01:55:54.090026+02 active | f volume_classification | 1 I removed other 10 snapshot infos from the list to easily read all On a host running v4.0.4 we run root@ovc2n06 ~]# lvchange -aey -pr 384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6 Logical volume "60ba7acf-58cb-475b-b9ee-15b1be99fee6" changed. This is the base image infact it doesn't contain a backing-file [root@ovc2n06 ~]# qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6 image: /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/60ba7acf-58cb-475b-b9ee-15b1be99fee6 file format: raw virtual size: 16G (17179869184 bytes) disk size: 0 [root@ovc2n06 ~]# lvchange -aey -pr 384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e Logical volume "68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e" changed. [root@ovc2n06 ~]# qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e qemu-img: Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6': Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6': No such file or directory To fix this problem we made a link [root@ovc2n06 ~]# lvchange -aey -pr 384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e Logical volume "68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e" changed. [root@ovc2n06 ~]# qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e qemu-img: Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6': Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6': No such file or directory [root@ovc2n06 ~]# ln -s /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8 /dev/8cda0aa6-9e25-4b50-ba00-b877232a1983 [root@ovc2n06 ~]# qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e image: /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/68d764ec-bc2e-4e1d-b8f2-b44afd9fcb2e file format: qcow2 virtual size: 16G (17179869184 bytes) disk size: 0 cluster_size: 65536 backing file: ../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6 (actual path: /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6) backing file format: raw Format specific information: compat: 0.10 refcount bits: 16 image: /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/../8cda0aa6-9e25-4b50-ba00-b877232a1983/60ba7acf-58cb-475b-b9ee-15b1be99fee6 file format: raw virtual size: 16G (17179869184 bytes) disk size: 0 We moved to the next snapshot [root@ovc2n06 ~]# lvchange -aey -pr 384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5 Logical volume "37ca6494-e990-44e5-8597-28845a0a19b5" changed. And now the error from the bug qemu-img info --backing-chain /dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5 qemu-img: Could not open '/dev/384f9059-ef2f-4d43-a54f-de71c5d589c8/37ca6494-e990-44e5-8597-28845a0a19b5': Backing file name too long Any snapshot starting from this snapshot is not usable, the VM was running ok up to we did shutdown it, it never starts again. How you can see the path of the backing-file is wrong and too long already in the previous snapshots, is there a way to fix it or to edit it manually ? Obviously we tried to clone, export, create a qcow2 image, from all the snapshot later 07-august but the operation didn't complete, we can recover only from the snapshot of 7-august that is missing 2 months of new data. Please if you have a workaround or a solution, can you write the commands we need to run with examples, we searched a lot about backing-file but only the manual of the qemu-img command was found with no examples on how to recover or change it. Thanks again Claudio Soprano -- / | / _____/ / | / _____/ | / / / | / / / / | / / / | / / / | / ___/ _____/ / / | / ___/ / | / / / | / / / / | / / / | / ______/ _/ __/ _/ _/ _/ __/ _/ _/ __/ Claudio Soprano phone: (+39)-06-9403.2349/2355 Computing Service fax: (+39)-06-9403.2649 LNF-INFN e-mail: Claudio.Soprano(a)lnf.infn.it Via Enrico Fermi, 40 www: http://www.lnf.infn.it/ I-00044 Frascati, Italy

1 0

Replication between storage domains
by Anantha Raghava 13 Oct '16

13 Oct '16

This is a multi-part message in MIME format. --------------8A5BBB261C1AC062725312E9 Content-Type: text/plain; charset=utf-8; format=flowed Content-Transfer-Encoding: 7bit Hello... I am trying to setup a DC & DR with oVirt with two sets of rack servers and two Storage Boxes. In DC, I have FC storage and in DR I have iSCSI storage. Now the question is can I replicate the VM disks and data between these dissimilar storage domains at say 15 min interval? -- Thanks & Regards, Anantha Raghava eXza Technology Consulting & Services --------------8A5BBB261C1AC062725312E9 Content-Type: text/html; charset=utf-8 Content-Transfer-Encoding: 7bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=utf-8"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hello... I am trying to setup a DC & DR with oVirt with two sets of rack servers and two Storage Boxes. In DC, I have FC storage and in DR I have iSCSI storage. Now the question is can I replicate the VM disks and data between these dissimilar storage domains at say 15 min interval? <div class="moz-signature">-- Thanks & Regards, <address style="line-height: 100%">Anantha Raghava</address> <address style="line-height: 100%">eXza Technology Consulting & Services</address> </div> </body> </html> --------------8A5BBB261C1AC062725312E9--

1 0

unable to get hosted-engine to start
by Francois Brochu 13 Oct '16

13 Oct '16

Thanks in advance for your enlightenment - somehow i may have missed a step when I try to backup - and that's when it stopped working the current situation is ALL my VM are up and running EXCEPT hostedengine - oVirt is running on a single node - ovirt-release40-4.0.4-1.noarch - Centos 7.2 hosted-engine --vm-start - did not generate error message at the console - log were complaining of lock and out of space (but now it's gone) - got rid off VM DOWN - got rid of sanlock without a process ID vdsClient does not show HostedEngine as UP (use to show DOWN - but I used poweroff) root@vmhost2 ~]# vdsClient -s 0 list table 01d78c9b-01e9-4dc3-b047-c809f82ae327 141203 GitRepo Up 5b4be6a8-2d94-440b-a221-7396f400ebf7 83717 jetbrains Up 393e28ed-6d11-4651-b367-dd0718ee40c7 126065 turing2 Up 5481665c-9a93-49c6-b7e7-b81495d00a9f 193259 vmcentosdebut Up 439c18db-9f5d-4983-9cd4-38a5ec9f57a6 79098 ecemail Up 844cfa3c-dcff-41a7-b96a-c680a83e4901 146694 vmwinchoco Up dd15556f-a290-4cc8-87d1-d311d1298caf 35974 newWds1 Up 9596a5cf-73ae-4c32-9f35-afbc4dd3cb39 5702 ipmi5 Up [root@vmhost2 ~]# sanlock client status (used to show "s hosted-engine:1:/var/run/vdsm/storage/fd533c44-09f0..." - but I used --reinitialize-lockspace [--force] ) daemon d29562f4-46e1-4838-ad70-666cbcaf1ce3.vmhost2.ec p -1 helper p -1 listener p -1 status p 141203 GitRepo p 126065 turing2 p 146694 vmwinchoco p 193259 vmcentosdebut p 5702 ipmi5 p 79098 ecemail p 35974 newWds1 p 83717 jetbrains AND when I look inside at my backup db is empty [root@vmhost2 ~]# tar -tvf /backup/oVirt.20161012/ovirtBackup drwxr-xr-x root/root 0 2016-10-11 14:52 ./ drwxr-xr-x root/root 0 2016-10-11 14:52 ./db/ -rw-r--r-- root/root 3160 2016-10-11 14:52 ./files -rw-r--r-- root/root 8 2016-10-11 14:52 ./version -rw-r--r-- root/root 8 2016-10-11 14:52 ./os_version -rw-r--r-- root/root 150 2016-10-11 14:52 ./config -rw-r--r-- root/root 168 2016-10-11 14:52 ./md5sum I also try hosted-engine --vm-start --vm-conf=/etc/ovirt-hosted-engine/hosted-engine.conf all I get is a console message "Unexpected exception" cannot find anything else What can I do to get the hosted-engine started? - and not loose any of my currently running VM -- *Francois Brochu* System Administrator/ IT Team Leader Electrical & Computer Engineering University of Alberta francois.brochu(a)ualberta.ca Tel: (780) 492-2010

1 0

VM pauses/hangs after migration
by Davide Ferrari 13 Oct '16

13 Oct '16

Hello trying to migrate a VM from one host to another, a big VM with 96GB of RAM, I found that when the migration completes, the VM goes to a paused satte and cannot be resumed. The libvirt/qemu log it gives is this: 2016-09-28T12:18:15.679176Z qemu-kvm: error while loading state section id 2(ram) 2016-09-28T12:18:15.680010Z qemu-kvm: load of migration failed: Input/output error 2016-09-28 12:18:15.872+0000: shutting down 2016-09-28 12:22:21.467+0000: starting up libvirt version: 1.2.17, package: 13.el7_2.5 (CentOS BuildSystem <http://bugs.centos.org>, 2016-06-23-14:23:27, worker1.bsys.centos.org) qemu version: 2.3.0 (qemu-kvm-ev-2.3.0-31.el7.16.1) LC_ALL=C PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin QEMU_AUDIO_DRV=spice /usr/libexec/qemu-kvm -name front04.billydomain.com -S -machine pc-i440fx-rhel7.2.0,accel=kvm,usb=off -cpu Haswell-noTSX -m size=100663296k,slots=16,maxmem=4294967296k -realtime mlock=off -smp 32,sockets=16,cores=1,threads=2 -numa node,nodeid=0,cpus=0-31,mem=98304 -uuid 4511d1c0-6607-418f-ae75-34f605b2ad68 -smbios type=1,manufacturer=oVirt,product=oVirt Node,version=7-2.1511.el7.centos.2.10,serial=4C4C4544-004A-3310-8054-B2C04F474432,uuid=4511d1c0-6607-418f-ae75-34f605b2ad68 -no-user-config -nodefaults -chardev socket,id=charmonitor,path=/var/lib/libvirt/qemu/ domain-front04.billydomain.com/monitor.sock,server,nowait -mon chardev=charmonitor,id=monitor,mode=control -rtc base=2016-09-28T14:22:21,driftfix=slew -global kvm-pit.lost_tick_policy=discard -no-hpet -no-shutdown -boot strict=on -device piix3-usb-uhci,id=usb,bus=pci.0,addr=0x1.0x2 -device virtio-scsi-pci,id=scsi0,bus=pci.0,addr=0x7 -device virtio-serial-pci,id=virtio-serial0,max_ports=16,bus=pci.0,addr=0x4 -drive if=none,id=drive-ide0-1-0,readonly=on,format=raw -device ide-cd,bus=ide.1,unit=0,drive=drive-ide0-1-0,id=ide0-1-0 -drive file=/rhev/data-center/00000001-0001-0001-0001-0000000003e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/b5b49d5c-2378-4639-9469-362e37ae7473/24fd0d3c-309b-458d-9818-4321023afacf,if=none,id=drive-virtio-disk0,format=qcow2,serial=b5b49d5c-2378-4639-9469-362e37ae7473,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x5,drive=drive-virtio-disk0,id=virtio-disk0,bootindex=1 -drive file=/rhev/data-center/00000001-0001-0001-0001-0000000003e3/ba2bd397-9222-424d-aecc-eb652c0169d9/images/f02ac1ce-52cd-4b81-8b29-f8006d0469e0/ff4e49c6-3084-4234-80a1-18a67615c527,if=none,id=drive-virtio-disk1,format=raw,serial=f02ac1ce-52cd-4b81-8b29-f8006d0469e0,cache=none,werror=stop,rerror=stop,aio=threads -device virtio-blk-pci,scsi=off,bus=pci.0,addr=0x8,drive=drive-virtio-disk1,id=virtio-disk1 -netdev tap,fd=30,id=hostnet0,vhost=on,vhostfd=31 -device virtio-net-pci,netdev=hostnet0,id=net0,mac=00:1a:4a:16:01:56,bus=pci.0,addr=0x3 -chardev socket,id=charchannel0,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.com.redhat.rhevm.vdsm,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=1,chardev=charchannel0,id=channel0,name=com.redhat.rhevm.vdsm -chardev socket,id=charchannel1,path=/var/lib/libvirt/qemu/channels/4511d1c0-6607-418f-ae75-34f605b2ad68.org.qemu.guest_agent.0,server,nowait -device virtserialport,bus=virtio-serial0.0,nr=2,chardev=charchannel1,id=channel1,name=org.qemu.guest_agent.0 -chardev spicevmc,id=charchannel2,name=vdagent -device virtserialport,bus=virtio-serial0.0,nr=3,chardev=charchannel2,id=channel2,name=com.redhat.spice.0 -vnc 192.168.10.225:1,password -k es -spice tls-port=5902,addr=192.168.10.225,x509-dir=/etc/pki/vdsm/libvirt-spice,tls-channel=default,tls-channel=main,tls-channel=display,tls-channel=inputs,tls-channel=cursor,tls-channel=playback,tls-channel=record,tls-channel=smartcard,tls-channel=usbredir,seamless-migration=on -k es -device qxl-vga,id=video0,ram_size=67108864,vram_size=8388608,vgamem_mb=16,bus=pci.0,addr=0x2 -incoming tcp:0.0.0.0:49156 -device virtio-balloon-pci,id=balloon0,bus=pci.0,addr=0x6 -msg timestamp=on Domain id=5 is tainted: hook-script red_dispatcher_loadvm_commands: KVM: entry failed, hardware error 0x8 RAX=00000000ffffffed RBX=ffff8817ba00c000 RCX=0100000000000000 RDX=0000000000000000 RSI=0000000000000000 RDI=0000000000000046 RBP=ffff8817ba00fe98 RSP=ffff8817ba00fe98 R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000 R12=0000000000000006 R13=ffff8817ba00c000 R14=ffff8817ba00c000 R15=0000000000000000 RIP=ffffffff81058e96 RFL=00010286 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00000000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =0000 0000000000000000 ffffffff 00000000 FS =0000 0000000000000000 ffffffff 00000000 GS =0000 ffff8817def80000 ffffffff 00000000 LDT=0000 0000000000000000 ffffffff 00000000 TR =0040 ffff8817def93b80 00002087 00008b00 DPL=0 TSS64-busy GDT= ffff8817def89000 0000007f IDT= ffffffffff529000 00000fff CR0=80050033 CR2=00000000ffffffff CR3=00000017b725b000 CR4=001406e0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000d01 Code=89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84 00 00 00 00 00 55 49 89 ca KVM: entry failed, hardware error 0x8 RAX=00000000ffffffed RBX=ffff8817ba008000 RCX=0100000000000000 RDX=0000000000000000 RSI=0000000000000000 RDI=0000000000000046 RBP=ffff8817ba00be98 RSP=ffff8817ba00be98 R8 =0000000000000000 R9 =0000000000000000 R10=0000000000000000 R11=0000000000000000 R12=0000000000000005 R13=ffff8817ba008000 R14=ffff8817ba008000 R15=0000000000000000 RIP=ffffffff81058e96 RFL=00010286 [--S--P-] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 0000000000000000 ffffffff 00000000 CS =0010 0000000000000000 ffffffff 00a09b00 DPL=0 CS64 [-RA] SS =0018 0000000000000000 ffffffff 00c09300 DPL=0 DS [-WA] DS =0000 0000000000000000 ffffffff 00000000 FS =0000 0000000000000000 ffffffff 00000000 GS =0000 ffff8817def40000 ffffffff 00000000 LDT=0000 0000000000000000 ffffffff 00000000 TR =0040 ffff8817def53b80 00002087 00008b00 DPL=0 TSS64-busy GDT= ffff8817def49000 0000007f IDT= ffffffffff529000 00000fff CR0=80050033 CR2=00000000ffffffff CR3=00000017b3c9a000 CR4=001406e0 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000d01 Code=89 e5 fb 5d c3 66 0f 1f 84 00 00 00 00 00 55 48 89 e5 fb f4 <5d> c3 0f 1f 84 00 00 00 00 00 55 48 89 e5 f4 5d c3 66 0f 1f 84 00 00 00 00 00 55 49 89 ca KVM: entry failed, hardware error 0x80000021 If you're running a guest on an Intel machine without unrestricted mode support, the failure can be most likely due to the guest entering an invalid state for Intel VT. For example, the guest maybe running in big real mode which is not supported on less recent Intel processors. EAX=ffffffed EBX=ba020000 ECX=00000000 EDX=00000000 ESI=00000000 EDI=00000046 EBP=ba023e98 ESP=ba023e98 EIP=81058e96 EFL=00000002 [-------] CPL=0 II=0 A20=1 SMM=0 HLT=0 ES =0000 00000000 0000ffff 00009300 DPL=0 DS [-WA] CS =f000 ffff0000 0000ffff 00009b00 DPL=0 CS16 [-RA] SS =0000 00000000 0000ffff 00009300 DPL=0 DS [-WA] DS =0000 00000000 0000ffff 00009300 DPL=0 DS [-WA] FS =0000 00000000 0000ffff 00009300 DPL=0 DS [-WA] GS =0000 00000000 0000ffff 00009300 DPL=0 DS [-WA] LDT=0000 00000000 0000ffff 00008200 DPL=0 LDT TR =0000 00000000 0000ffff 00008b00 DPL=0 TSS64-busy GDT= 0000000000000000 0000ffff IDT= 0000000000000000 0000ffff CR0=80050033 CR2=00007fd826ac20a0 CR3=000000003516c000 CR4=00140060 DR0=0000000000000000 DR1=0000000000000000 DR2=0000000000000000 DR3=0000000000000000 DR6=00000000ffff0ff0 DR7=0000000000000400 EFER=0000000000000d01 Code=?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? <??> ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? ?? Searching for errors like this I found some bug report about kernel issues but I don't think it's the case, other VMs spawned from the same image migrate without any issue. I have toi say that the original host running the VM has some RAM problem (ECC multibit fault in one DIMM). Maybe that's the problem? How can I properly read this error log? Thanks -- Davide Ferrari Senior Systems Engineer

3 8

Multi cluster question with regards to storage
by David Gossage 13 Oct '16

13 Oct '16

I've not yet found a clean concise answer and so I wanted to ask before I start trying to play with things in my test setup and waste a bunch of time maybe. If I have 2 clusters in one data center each running different cpu types (amd/intel) can they both access the same shared storage domain(gluster in my case)? I'm not interested in live migration between the clusters really, though stopping and starting on another would be nice. I've tried looking at various threads and list emails and I've found some mentions of it, but usually as an aside while discussing other matters so I could never quite say for certain it was working or not. *David Gossage* *Carousel Checks Inc. | System Administrator* *Office* 708.613.2284

2 2

unsupported configuration: Unable to find security driver for model selinux
by Николаев Алексей 13 Oct '16

13 Oct '16

4 7

vdsm ssl errors
by C. Handel 13 Oct '16

13 Oct '16

i have an ovirt 4.0.1 installation on two nodes. Hosted Engine. San storage backend. For some reason the vdsmd on the nodes is logging an error every few seconds: vdsm vds.dispatcher ERROR SSL error during reading data: unexpected eof Running tcpdump it is a connect from the node to itself. I can't figure out what is wrong. Can someone give me a hint? Greetings Christoph

9 42

[ANN] oVirt 4.0.5 Second Release Candidate is now available
by Sandro Bonazzola 13 Oct '16

13 Oct '16

The oVirt Project is pleased to announce the availability of oVirt 4.0.5 second release candidate for testing, as of October 13th, 2016. This release is available now for: * Fedora 23 (tech preview) * Red Hat Enterprise Linux 7.2 or later * CentOS Linux (or similar) 7.2 or later This release supports Hypervisor Hosts running: * Red Hat Enterprise Linux 7.2 or later * CentOS Linux (or similar) 7.2 or later * Fedora 23 (tech preview) * oVirt Next Generation Node 4.0 This is pre-release software. Please take a look at our community page[1] to know how to ask questions and interact with developers and users. All issues or bugs should be reported via oVirt Bugzilla[2]. This pre-release should not to be used in production. This update is the second release candidate of the fifth in a series of stabilization updates to the 4.0 series. 4.0.5 brings 9 enhancements and 49 bugfixes, including 24 high or urgent severity fixes, on top of oVirt 4.0 series See the release notes [3] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: * A new oVirt Live ISO is available. [4] * A new oVirt Next Generation Node is available [4] * A new oVirt Engine Appliance is available for Red Hat Enterprise Linux and CentOS Linux (or similar) * Mirrors[5] might need up to one day to synchronize. Additional Resources: * Read more about the oVirt 4.0.5 release highlights: http://www.ovirt.org/release/4.0.5/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] https://www.ovirt.org/community/ [2] https://bugzilla.redhat.com/enter_bug.cgi?classification=oVirt [3] http://www.ovirt.org/release/4.0.5/ [4] http://resources.ovirt.org/pub/ovirt-4.0-pre/iso/ [5] http://www.ovirt.org/Repository_mirrors#Current_mirrors -- Sandro Bonazzola Better technology. Faster innovation. Powered by community collaboration. See how it works at redhat.com

1 0

Failed to read hardware information
by David Pinkerton 13 Oct '16

13 Oct '16

I've spent the weekend trying to get to the bottom of this issue. Adding a Host fails: >From RHVM VDSM rhv1 command failed: Connection reset by peer Could not get hardware information for host rhv1 VDSM rhv1 command failed: Failed to read hardware information Host rhv1 installed Network changes were saved on host rhv1 Installing Host rhv1. Stage: Termination. Installing Host rhv1. Retrieving installation logs to: '/var/log/ovirt-engine/host-deploy/ovirt-host-deploy-20161010115606-192.168.21.71-24d39274.log'. Installing Host rhv1. Stage: Pre-termination. Installing Host rhv1. Starting ovirt-vmconsole-host-sshd. Installing Host rhv1. Starting vdsm. Installing Host rhv1. Stopping libvirtd. Installing Host rhv1. Stage: Closing up. Installing Host rhv1. Setting kernel arguments. Installing Host rhv1. Stage: Transaction commit. Installing Host rhv1. Enrolling serial console certificate. Installing Host rhv1. Enrolling certificate. Installing Host rhv1. Stage: Misc configuration. This was in the /var/log/vdsm/vdsm.log on the host trying to be added: jsonrpc.Executor/2::ERROR::2016-10-10 11:57:10,276::API::1340::vds::(getHardwareInfo) failed to retrieve hardware info Traceback (most recent call last): File "/usr/share/vdsm/API.py", line 1337, in getHardwareInfo hw = supervdsm.getProxy().getHardwareInfo() File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 53, in __call__ return callMethod() File "/usr/lib/python2.7/site-packages/vdsm/supervdsm.py", line 51, in <lambda> **kwargs) File "<string>", line 2, in getHardwareInfo File "/usr/lib64/python2.7/multiprocessing/managers.py", line 759, in _callmethod kind, result = conn.recv() EOFError and then VDSM fails to start. Looking at the source code... def getHardwareInfoStructure(): dmiInfo = getAllDmidecodeInfo() sysStruct = {} for k1, k2 in (('system', 'Manufacturer'), ('system', 'Product Name'), ('system', 'Version'), ('system', 'Serial Number'), ('system', 'UUID'), ('system', 'Family')): val = dmiInfo.get(k1, {}).get(k2, None) if val not in [None, 'Not Specified']: sysStruct[(k1 + k2).replace(' ', '')] = val return sysStruct Running dmidecode from command line I get.. System Information Manufacturer: Supermicro Product Name: H8DM8-2 Version: 1234567890 Serial Number: 1234567890 UUID: 00020003-0004-0005-0006-000700080009 Wake-up Type: Power Switch SKU Number: To Be Filled By O.E.M. Family: To Be Filled By O.E.M. Q: Is the string in Family the source of my problems?? Q: Any work arounds?? -- David Pinkerton Consultant Red Hat Asia Pacific Pty. Ltd. Level 11, Canberra House 40 Marcus Clarke Street Canberra 2600 ACT Mobile: +61-488-904-232 Email: david.pinkerton(a)redhat.com Web: http://apac.redhat.com/ <http://apac.redhat.com/>

5 11