- Users - oVirt List Archives

Re: Ovirt 4.3.2 VM Failover
by Strahil 26 Apr '19

26 Apr '19

I have noticed after full power failiure (all nodes simultaneously - labs don't have UPS yet) one of my VMs was in that state, and I had to restart VDSM in order the engine to notice the crash. Definitely , there is something phishy here... I guess Sahina & Sandro can take a look in your case.. Best Regards, Strahil NikolovOn Apr 26, 2019 02:56, michael(a)wanderingmad.com wrote: > > Is there something in the 4.3.x series that has changed when it comes to VM failover? I have the cluster configurated to automatically migrate all server VMs, but whenever One of the nodes goes down, all the VMs that were on that host switch to a ? status and I have to go and click "host has been rebooted" to have them failed over. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ABOBHQGF2U6JC…

1 0

Ovirt 4.3.2 VM Failover
by michael＠wanderingmad.com 26 Apr '19

26 Apr '19

Is there something in the 4.3.x series that has changed when it comes to VM failover? I have the cluster configurated to automatically migrate all server VMs, but whenever One of the nodes goes down, all the VMs that were on that host switch to a ? status and I have to go and click "host has been rebooted" to have them failed over.

2 1

All Windows vms report "Actual timezone in guest doesn't match configuration"
by michael＠wanderingmad.com 26 Apr '19

26 Apr '19

All of my windows VMs have a yellow exclamation point with the timezone error. The configuration in ovirt says eastern time, and I've tried changing the VM from eastern time to US eastern time, and error does not go away. This is with the latest ovirt guest tools for windows.

1 0

Adding network to VM - What stupid thing have I missed?
by eshwayri＠gmail.com 25 Apr '19

25 Apr '19

When creating a new VM, it looks like I connect it's nic(s) under the "Instantiate VM network interfaces by picking a vNIC profile." setting. The problem I am seeing is that the drop down only has "Empty" and "br-kvm-prod" (my production bridge). I should have two more. Under Networks and vNIC Profiles tabs, I also have a br-kvm-stor and kvm_heart-22 defined which don't appear in that list. Am I missing something I need to do for these additional profiles to also appear in the drop-down? Some of my VMs will need access to the other physical networks.

3 3

New disk creation very slow after upgrade to 4.3.3
by Steffen Luitz 25 Apr '19

25 Apr '19

This is on a 3-node hyperconverged environment with glusterfs. After upgrading to oVirt 4.3.3 (from 4.3.2) creating a new disk takes very long (hours for a 100GByte disk, making it essentially impossible to create a new disk image). In the UI the default is "preallocated" but changing it to thin provision does not make any difference. Regardless of this setting, an fallocate process gets started on the SDM host: /usr/bin/python2 /usr/libexec/vdsm/fallocate 107374182400 /rhev/data-center/mnt/glusterSD/s-vmhost2-ovir.t... Using fallocate to create a file directly on the underlying file system is fast. Using it to create a file through the glusterfs fuse mount is very slow. Thanks for any insights. Steffen

3 2

Upgrade from 4.3.2 to 4.3.3 fails on database schema update
by eshwayri＠gmail.com 24 Apr '19

24 Apr '19

Tried to upgrade to 4.3.3, and during engine-setup I get: 2019-04-20 14:24:47,041-0400 Running upgrade sql script '/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql'... 2019-04-20 14:24:47,043-0400 dbfunc_psql_die --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql ********* QUERY ********** SELECT fn_db_create_constraint('image_transfers', 'fk_image_transfers_command_enitites', 'FOREIGN KEY (command_id) REFERENCES command_entities(command_id) ON DELETE CASCADE'); ************************** 2019-04-20 14:24:47,060-0400 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.executeRaw:863 execute-result: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20190420142153-s5heq0.log', '-c', 'apply'], rc=1 2019-04-20 14:24:47,060-0400 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:921 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20190420142153-s5heq0.log', '-c', 'apply'] stdout: 2019-04-20 14:24:47,061-0400 DEBUG otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema plugin.execute:926 execute-output: ['/usr/share/ovirt-engine/dbscripts/schema.sh', '-s', 'localhost', '-p', '5432', '-u', 'engine', '-d', 'engine', '-l', '/var/log/ovirt-engine/setup/ovirt-engine-setup-20190420142153-s5heq0.log', '-c', 'apply'] stderr: psql:/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql:3: ERROR: insert or update on table "image_transfers" violates foreign key constraint "fk_image_transfers_command_enitites" DETAIL: Key (command_id)=(7d68555a-71ee-41f5-9f12-0b26b6b9d449) is not present in table "command_entities". CONTEXT: SQL statement "ALTER TABLE image_transfers ADD CONSTRAINT fk_image_transfers_command_enitites FOREIGN KEY (command_id) REFERENCES command_entities(command_id) ON DELETE CASCADE" PL/pgSQL function fn_db_create_constraint(character varying,character varying,text) line 9 at EXECUTE FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql 2019-04-20 14:24:47,061-0400 ERROR otopi.plugins.ovirt_engine_setup.ovirt_engine.db.schema schema._misc:432 schema.sh: FATAL: Cannot execute sql command: --file=/usr/share/ovirt-engine/dbscripts/upgrade/04_03_0830_add_foreign_key_to_image_transfers.sql 2019-04-20 14:24:47,061-0400 DEBUG otopi.context context._executeMethod:145 method exception Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/otopi/context.py", line 132, in _executeMethod method['method']() File "/usr/share/ovirt-engine/setup/bin/../plugins/ovirt-engine-setup/ovirt-engine/db/schema.py", line 434, in _misc raise RuntimeError(_('Engine schema refresh failed')) RuntimeError: Engine schema refresh failed 2019-04-20 14:24:47,064-0400 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Misc configuration': Engine schema refresh failed 2019-04-20 14:24:47,065-0400 DEBUG otopi.transaction transaction.abort:119 aborting 'Yum Transaction' 2019-04-20 14:24:47,065-0400 INFO otopi.plugins.otopi.packagers.yumpackager yumpackager.info:80 Yum Performing yum transaction rollback

2 2

Re: Upgrading from 4.2.8 to 4.3.3 broke Node NG GlusterFS
by Strahil 24 Apr '19

24 Apr '19

Fix those disconnectes node and run find against a node that has successfully mounted the volume. Best Regards, Strahil NikolovOn Apr 24, 2019 15:31, Andreas Elvers <andreas.elvers+ovirtforum(a)solutions.work> wrote: > > The file handle is stale so find will display: > > "find: '/rhev/data-center/mnt/glusterSD/node01.infra.solutions.work:_vmstore': Transport endpoint is not connected" > > "stat /rhev/data-center/mnt/glusterSD/node01.infra.solutions.work:_vmstore" will output > stat: cannot stat '/rhev/data-center/mnt/glusterSD/node01.infra.solutions.work:_vmstore': Transport endpoint is not connected > > All Nodes are peering with the other nodes: > ----- > Saiph:~ andreas$ ssh node01 gluster peer status > Number of Peers: 2 > > Hostname: node02.infra.solutions.work > Uuid: 87fab40a-2395-41ce-857d-0b846e078cdb > State: Peer in Cluster (Connected) > > Hostname: node03.infra.solutions.work > Uuid: 49025f81-e7c1-4760-be03-f36e0f403d26 > State: Peer in Cluster (Connected) > ---- > Saiph:~ andreas$ ssh node02 gluster peer status > Number of Peers: 2 > > Hostname: node03.infra.solutions.work > Uuid: 49025f81-e7c1-4760-be03-f36e0f403d26 > State: Peer in Cluster (Disconnected) > > Hostname: node01.infra.solutions.work > Uuid: f25e6bff-e5e2-465f-a33e-9148bef94633 > State: Peer in Cluster (Connected) > ---- > ssh node03 gluster peer status > Number of Peers: 2 > > Hostname: node02.infra.solutions.work > Uuid: 87fab40a-2395-41ce-857d-0b846e078cdb > State: Peer in Cluster (Connected) > > Hostname: node01.infra.solutions.work > Uuid: f25e6bff-e5e2-465f-a33e-9148bef94633 > State: Peer in Cluster (Connected) > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DI6AWTLIQIPWN…

2 1

error while setup
by W3SERVICES 24 Apr '19

24 Apr '19

PLAY [gluster_servers] ********************************************************* TASK [Run a shell script] ****************************************************** changed: [localhost.localdomain] => (item=/usr/share/gdeploy/scripts/grafton-sanity-check.sh -d sdb -h localhost.localdomain) PLAY RECAP ********************************************************************* localhost.localdomain : ok=1 changed=1 unreachable=0 failed=0 PLAY [gluster_servers] ********************************************************* TASK [Enable or disable services] ********************************************** ok: [localhost.localdomain] => (item=chronyd) PLAY RECAP ********************************************************************* localhost.localdomain : ok=1 changed=0 unreachable=0 failed=0 PLAY [gluster_servers] ********************************************************* TASK [start/stop/restart/reload services] ************************************** changed: [localhost.localdomain] => (item=chronyd) PLAY RECAP ********************************************************************* localhost.localdomain : ok=1 changed=1 unreachable=0 failed=0 PLAY [gluster_servers] ********************************************************* TASK [Run a command in the shell] ********************************************** changed: [localhost.localdomain] => (item=vdsm-tool configure --force) PLAY RECAP ********************************************************************* localhost.localdomain : ok=1 changed=1 unreachable=0 failed=0 PLAY [gluster_servers] ********************************************************* TASK [Run a shell script] ****************************************************** changed: [localhost.localdomain] => (item=/usr/share/gdeploy/scripts/blacklist_all_disks.sh) PLAY RECAP ********************************************************************* localhost.localdomain : ok=1 changed=1 unreachable=0 failed=0 PLAY [gluster_servers] ********************************************************* TASK [Clean up filesystem signature] ******************************************* skipping: [localhost.localdomain] => (item=/dev/sdb) TASK [Create Physical Volume] ************************************************** failed: [localhost.localdomain] (item=/dev/sdb) => {"changed": false, "failed_when_result": true, "item": "/dev/sdb", "msg": " Device /dev/sdb not found.\n", "rc": 5} to retry, use: --limit @/tmp/tmpQtig6r/pvcreate.retry PLAY RECAP ********************************************************************* localhost.localdomain : ok=0 changed=0 unreachable=0 failed=1 From *Sunil Kumar | 09967222267Tech Head and Cloud Solution Architect* *-----------------------------------------------------------------------* *Website : * *https://w3services.net <https://www.w3services.net/>* https://w3services.com (Tech Consultancy) Office Tel. +91 09300670068 | Write A Google Review ( Help Us ) <https://goo.gl/tKGa79> *-----------------------------------------------------------------------*

1 0

[ANN] oVirt 4.3.3 second async update is now available
by Sandro Bonazzola 24 Apr '19

24 Apr '19

The oVirt Team has just released a new version of the following packages: - ovirt-engine-4.3.3.6 The async release addresses the following bugs: - Bug 1701205 <https://bugzilla.redhat.com/show_bug.cgi?id=1701205> - Creating a new VM over the not defaulted cluster fails with "CPU Profile doesn't match provided Cluster" error. - Bug 1700759 <https://bugzilla.redhat.com/show_bug.cgi?id=1700759> - engine failed schema refresh v4.3.2 -> v4.3.3 Thanks, -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

1 0

Arbiter brick disk performance
by Leo David 24 Apr '19

24 Apr '19

Hello Everyone, I need to look into adding some enterprise grade sas disks ( both ssd and spinning ), and since the prices are not too low, I would like to benefit of replica 3 arbitrated. Therefore, I intend to buy some smaller disks for use them as arbiter brick. My question is, what performance ( regarding iops, througput ) the arbiter disks need to be. Should they be at least the same as the real data disks ? Knowing that they only keep metadata, I am thinking that will not be so much pressure on the arbiters. Any thoughts? Thank you ! -- Best regards, Leo David

1 0

Upgrading from 4.2.8 to 4.3.3 broke Node NG GlusterFS
by Andreas Elvers 24 Apr '19

24 Apr '19

Hi, I am currently upgrading my oVirt setup from 4.2.8 to 4.3.3.1. The setup consists of: Datacenter/Cluster Default: [fully upgraded to 4.3.3.1] 2 nodes (node04,node05)- NFS storage domain with self hosted engine Datacenter Luise: Cluster1: 3 nodes (node01,node02,node03) - Node NG with GlusterFS - Ceph Cinder storage domain [Node1 and Node3 are upgraded to 4.3.3.1, Node2 is on 4.2.8] Cluster2: 1 node (node06) - only Ceph Cinder storage domain [fully upgraded to 4.3.3.1] Problems started when upgrading Luise/Cluster1 with GlusterFS: (I always waited for GlusterFS to be fully synced before proceeding to the next step) - Upgrade node01 to 4.3.3 -> OK - Upgrade node03 to 4.3.3.1 -> OK - Upgrade node01 to 4.3.3.1 -> GlusterFS became unstable. I now get the error message: VDSM node03.infra.solutions.work command ConnectStoragePoolVDS failed: Cannot find master domain: u'spUUID=f3218bf7-6158-4b2b-b272-51cdc3280376, msdUUID=02a32017-cbe6-4407-b825-4e558b784157' And on node03 there is a problem with Gluster: node03#: ls -l /rhev/data-center/mnt/glusterSD/node01.infra.solutions.work:_vmstore ls: cannot access /rhev/data-center/mnt/glusterSD/node01.infra.solutions.work:_vmstore: Transport endpoint is not connected The directory is available on node01 and node02. The engine is reporting the brick on node03 as down. Node03 and Node06 are shown as NonOperational, because they are not able to access the gluster storage domain. A “gluster peer status” on node1, node2, and node3 shows all peers connected. “gluster volume heal vmstore info” shows for all nodes: gluster volume heal vmstore info Brick node01.infra.solutions.work:/gluster_bricks/vmstore/vmstore Status: Transport endpoint is not connected Number of entries: - Brick node02.infra.solutions.work:/gluster_bricks/vmstore/vmstore <gfid:0bcb7825-e649-4178-a899-c5cc04c95286> <gfid:71ec8035-f5a5-4e61-bb34-5ad9db28c0eb> <gfid:16d5961e-c3bb-4493-a51d-bf83074c4cc7> /02a32017-cbe6-4407-b825-4e558b784157/dom_md/ids /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.66 <gfid:5fe350e4-1eb5-4b6f-a3fb-42c98b7b2f8d> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.60 /02a32017-cbe6-4407-b825-4e558b784157/images/a3a10398-9698-4b73-84d9-9735448e3534/6161e310-4ad6-42d9-8117-5a89c5b2b4b6 <gfid:8eb9fd30-fdb9-442b-9c54-8ba256d7981b> <gfid:c72001be-e7d3-4b34-bac5-9ab50b609eea> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.96 <gfid:447ec09b-336e-4d2b-8338-f31329ee7a55> <gfid:9d7db516-d6fb-43d8-a069-dcbc1d72e62a> /.shard/d66880de-3fa1-4362-8c43-574a173c5f7d.133 <gfid:2412d449-d3ed-40ef-b7eb-d81bdf7c5c05> <gfid:0fae358b-2cdd-4064-b63c-7f31a35bc35a> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.38 /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.67 /__DIRECT_IO_TEST__ <gfid:a7945526-9ff3-40fe-b1e2-3921117ef738> <gfid:e78e9c1f-ce6b-4871-b5bf-9bde34685b99> /02a32017-cbe6-4407-b825-4e558b784157/images/493188b2-c137-4440-99ee-43a753842a7d/9aa2d139-e3bd-406b-8fe0-b189123eaa73 <gfid:3aed3fb6-044a-4371-9302-e0bd54cbd794> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.64 /.shard/d66880de-3fa1-4362-8c43-574a173c5f7d.132 <gfid:f7631be7-2ab5-4985-904d-69174c0e1267> <gfid:43001625-1aad-4032-a76e-4cc2a51de2b3> <gfid:6ae3fe7f-15c9-4103-960c-faba0ba59cb3> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.44 /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.9 /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.69 <gfid:f20cc6f7-9391-4260-9238-3e1d0cabbfa3> /02a32017-cbe6-4407-b825-4e558b784157/images/12e647fb-20aa-4957-b659-05fa75a9215e/f7e4b2a3-ab84-4eb5-a4e7-7208ddad8156 <gfid:c540368a-4431-4405-9a59-e11a217d0ea6> <gfid:4e698a74-39dc-40a3-ac9c-14456420ab66> <gfid:afd48e71-ff23-42d7-aef4-b2e2167b75e8> <gfid:194589b3-0760-4150-80ef-d87376813835> <gfid:6e17ead1-88cc-4e3e-84fa-7495c4fc3a0e> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.35 /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.32 <gfid:982d071f-2081-4371-b007-bc48d8167e7c> <gfid:e2285905-a8da-44f5-8c56-1f8a4d6326a8> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.39 <gfid:956c7d9c-2f96-42d1-bf6e-57bc9e534f84> <gfid:16162bc7-201a-4842-a41a-af2cc4fb8a9e> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.34 /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.68 Status: Connected Number of entries: 47 Brick node03.infra.solutions.work:/gluster_bricks/vmstore/vmstore /02a32017-cbe6-4407-b825-4e558b784157/images/12e647fb-20aa-4957-b659-05fa75a9215e/f7e4b2a3-ab84-4eb5-a4e7-7208ddad8156 <gfid:c540368a-4431-4405-9a59-e11a217d0ea6> <gfid:4e698a74-39dc-40a3-ac9c-14456420ab66> <gfid:afd48e71-ff23-42d7-aef4-b2e2167b75e8> <gfid:194589b3-0760-4150-80ef-d87376813835> <gfid:6e17ead1-88cc-4e3e-84fa-7495c4fc3a0e> <gfid:099284a6-9538-4f9a-928a-d9b704fe0735> <gfid:75d3c8f7-d67a-49a4-9cd4-7fff202df40d> <gfid:982d071f-2081-4371-b007-bc48d8167e7c> <gfid:e2285905-a8da-44f5-8c56-1f8a4d6326a8> <gfid:447ec09b-336e-4d2b-8338-f31329ee7a55> <gfid:956c7d9c-2f96-42d1-bf6e-57bc9e534f84> /.shard/d66880de-3fa1-4362-8c43-574a173c5f7d.133 <gfid:43001625-1aad-4032-a76e-4cc2a51de2b3> <gfid:6ae3fe7f-15c9-4103-960c-faba0ba59cb3> <gfid:1a0b2737-9172-4c51-aa77-e93e9671840c> <gfid:eb471e13-6749-4f62-b1f5-15a44f8990c2> <gfid:a7945526-9ff3-40fe-b1e2-3921117ef738> <gfid:e78e9c1f-ce6b-4871-b5bf-9bde34685b99> /02a32017-cbe6-4407-b825-4e558b784157/images/493188b2-c137-4440-99ee-43a753842a7d/9aa2d139-e3bd-406b-8fe0-b189123eaa73 <gfid:6b418e80-9f61-4d6e-ba77-8a1969d9a99b> <gfid:914c72d2-e45e-48f2-b7ef-5846b13f7a91> <gfid:2bd28bdb-1dc6-41d5-96be-c696f452e3f2> <gfid:9d7db516-d6fb-43d8-a069-dcbc1d72e62a> <gfid:f7631be7-2ab5-4985-904d-69174c0e1267> <gfid:16162bc7-201a-4842-a41a-af2cc4fb8a9e> /.shard/40948f85-2212-47f9-bd5e-102a8dd632b8.44 <gfid:afc6d611-528d-441b-b74e-c5fae6672088> <gfid:707308e9-e8e5-487a-b0f1-a816720c4243> <gfid:5f1a81a7-7c42-4226-9142-1b5b35c2b1e9> <gfid:f20cc6f7-9391-4260-9238-3e1d0cabbfa3> <gfid:0bcb7825-e649-4178-a899-c5cc04c95286> <gfid:71ec8035-f5a5-4e61-bb34-5ad9db28c0eb> <gfid:16d5961e-c3bb-4493-a51d-bf83074c4cc7> /02a32017-cbe6-4407-b825-4e558b784157/dom_md/ids <gfid:7661180b-1917-4a7b-9749-5dfb826c4449> <gfid:5fe350e4-1eb5-4b6f-a3fb-42c98b7b2f8d> <gfid:a6197593-7e09-4d3f-b538-9cd1ebadd6c9> /02a32017-cbe6-4407-b825-4e558b784157/images/a3a10398-9698-4b73-84d9-9735448e3534/6161e310-4ad6-42d9-8117-5a89c5b2b4b6 <gfid:8eb9fd30-fdb9-442b-9c54-8ba256d7981b> <gfid:c72001be-e7d3-4b34-bac5-9ab50b609eea> <gfid:3aed3fb6-044a-4371-9302-e0bd54cbd794> /.shard/d66880de-3fa1-4362-8c43-574a173c5f7d.132 <gfid:2412d449-d3ed-40ef-b7eb-d81bdf7c5c05> <gfid:0fae358b-2cdd-4064-b63c-7f31a35bc35a> <gfid:c0ca2784-a8af-44b3-9091-a1eaf4c8676f> /__DIRECT_IO_TEST__ Status: Connected Number of entries: 47 On Node03 there are several self healing processes, that seem to be doing nothing. Oh well.. What now? Best regards, - Andreas

2 4

Unable to use MAC address starting with reserved value 0xFE
by Ricardo Alonso 24 Apr '19

24 Apr '19

Is there a way to use a mac starting with FE? The machine has a license requirement attached to the mac address, and when I try to start it, it fails with the message: VM is down with error. Exit message: unsupported configuration: Unable to use MAC address starting with reserved value 0xFE - 'fe:XX:XX:XX:XX:XX'

1 0

Upgrade Initiation fails without obvious reason
by Vrgotic, Marko 24 Apr '19

24 Apr '19

Dear oVIrt, Short platform summary: * 5 Hosts * 3/5 in Shared DC/Cluster with NetApp NFS storage * 1/5 with Local Storage * 1/5 with Local Storage * oVirt version 4.3.2 , recently upgraded from 4.2.8 Host related: ovirt-staging-hv-04 Host is moved to Maintenance. Selected Upgrade, following log lines are written: 2019-04-23 10:58:32,639Z INFO [org.ovirt.engine.core.bll.hostdeploy.UpgradeHostCommand] (default task-247) [11381573-01c0-4e48-8634-c577988a8f79] Running command: UpgradeHostCommand internal: false. Entities affected : ID: 9462386e-74d1-4000-96b0-0e081ec7d069 Type: VDSAction group EDIT_HOST_CONFIGURATION with role type ADMIN 2019-04-23 10:58:32,648Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-247) [11381573-01c0-4e48-8634-c577988a8f79] EVENT_ID: HOST_UPGRADE_STARTED(840), Host ovirt-staging-hv-04 upgrade was started (User: mvrgotic@ictv.com@ictv.com-authz<mailto:mvrgotic@ictv.com@ictv.com-authz>). However, the host is still in Maintenance mode, expected is Installing: [cid:image001.png@01D4F9D5.35D3A4C0] Tail YUM logs also shows the procedure did not start and packages are not being updated. Kindly awaiting you reply. Marko Vrgotic

3 2

Prevent 2 different VMs from running on the same host
by Paulo Silva 24 Apr '19

24 Apr '19

Hi, I have a cluster of 6 hosts using ovirt 4.3 and I want to make sure that 2 VMs are always started on different hosts. Is it possible to prevent 2 different VMs from running on the same physical host without specifying manually a different set of hosts where each VM can start running? Thanks -- Paulo Silva <paulojjs(a)gmail.com>

2 1

Scale out ovirt 4.3 (from 3 to 6 or 9 nodes) with hyperconverged setup and Gluster
by adrianquintero＠gmail.com 24 Apr '19

24 Apr '19

Hello, I have a 3 node Hyperconverged setup with gluster and added 3 new nodes to the cluster for a total of 6 servers. I am now taking advantage of more compute power but cant scale out my storage volumes. Current Hyperconverged setup: - host1.mydomain.com ---> Bricks: engine data1 vmstore1 - host2.mydomain.com ---> Bricks: engine data1 vmstore1 - host3.mydomain.com ---> Bricks: engine data1 vmstore1 From these 3 servers we get the following Volumes: - engine(host1:engine, host2:engine, host3:engine) - data1 (host1:data1, host2:data1, host3:data1) - vmstore1 (host1:vmstore1, host2:vmstore1, host3:vmstore11) The following are the newly added servers to the cluster, however as you can see there are no gluster bricks - host4.mydomain.com - host5.mydomain.com - host6.mydomain.com I know that the bricks must be added in sets of 3 and per the first 3 hosts that is how it was deployed thru the web UI. Questions: -how can i extend the gluster volumes engine, data1 and vmstore using host4, host5 and host6? -Do I need to configure gluster volumes manually through the OS CLI in order for them to span amongst all 6 servers? -If I configure the fail storage scenario manually will oVirt know about it?, will it still be hyperconverged? I have only seen 3 host hyperconverged setup examples with gluster, but have not found examples for 6, 9 or 12 host cluster with gluster. I know it might be a lack of understanding from my end on how ovirt and gluster integrate with one another. If you can point me in the right direction would be great. thanks,

4 6

Gluster and few iSCSI Datastores in one Data Center
by toslavik＠yandex.ru 23 Apr '19

23 Apr '19

I decided to add another cluster to the existing data center (Enable Virt Service + Enable Gluster Service). Three nodes. But after installing the nodes (without errors) Ovirt Engine loses them cyclically. From the logs you can see that this happens when you try to interrogate the connected block devices. These block devices are LVM for VM. I use several Datastores submitted via iSCSI. About 2000 virtual machines and more than 2300 LVMs. All in the neighbor cluster (one data center). Logs in https://drive.google.com/open?id=1ja7Usxx5YCFDgjD2X51z9tzn_ycPoC2g Why is this happening and what can be done in this case?

2 2

ovirt with kvm stand alone
by igalvarez＠gmail.com 23 Apr '19

23 Apr '19

I have 3 KVM stand alone servers with no external storage. My question is, can I add these 3 KVM servers to ovirt without impact on VMs already running on KVM side? Thanks.

3 3

Re: 4.3.x upgrade and issues with OVN
by Weber, Charles (NIH/NIA/IRP) [E] 23 Apr '19

23 Apr '19

Configuration with versions Next email will have log files. 2 sites First site: Bayview 4 nodes BL460 gen9 with 4 x 10G nics Node 1-3 have not been changed since 4.3.2 upgrade. These nodes have the network sync issue and cannot migrate VMs. OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: CentOS Linux 7 (Core) Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.11-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 0, RETP: 1 VNC Encryption: Disabled I evacuated node 4 and did update to 4.3.3, still had issues so tried ISO OVNode install. It does not have network sync issue. However I cannot upgrade all the nodes without scheduling cluster downtime. OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.3.1 Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.13-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 0, RETP: 1, SSBD: 3 VNC Encryption: Disabled 1 engine, DL360g10. 4.3.3.5-1.el7 CentOS 7 patched up to today Storage: Dell Unity ISCSI What I see in the engine is that the first 3 nodes all have the public network listed as out of sync with the DC. I can migrate form node 4 to the other 3 nodes but cannot migrate off the other 3 nodes. I also cannot sync the network on the other three nodes. There have been no recent network changes. After the 4.3.3 upgrade I initially found some curious OVN errors in the logfile on nodes 3 and 4. Nodes 1 and 2 do not have these errors. However the engine did have 2 extra OVN ports defined. ovs-vsctl show be10cd3d-85fe-4985-9635-f447bfbc5e25 Bridge br-int fail_mode: secure Port "ovn-877214-0" Interface "ovn-877214-0" type: geneve options: {csum="true", key=flow, remote_ip="137.187.160.14"} error: "could not add network device ovn-877214-0 to ofproto (File exists)" Port "ovn-48e040-0" Interface "ovn-48e040-0" type: geneve options: {csum="true", key=flow, remote_ip="137.187.160.18"} Port "ovn-d6eaa1-0" Interface "ovn-d6eaa1-0" type: geneve options: {csum="true", key=flow, remote_ip="137.187.160.13"} error: "could not add network device ovn-d6eaa1-0 to ofproto (File exists)" Port br-int Interface br-int type: internal Port "ovn-f0f789-0" Interface "ovn-f0f789-0" type: geneve options: {csum="true", key=flow, remote_ip="137.187.160.13"} Second site:Harbor I upgraded to 4.3.2 and stopped any upgrades until I could deal with the migration and restarting VDSMD issues. Nodes are 3 supermicro 1U with 2x10G nics All 3 nodes are the same and from the 4.3.2 update. OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: CentOS Linux 7 (Core) Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.11-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 GlusterFS Version: [N/A] CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 0, RETP: 1 VNC Encryption: Disabled 1 engine DL360 g10 with 2x10G nics running 4.3.2.1-1.el7 Storage: Dell Unity ISCSI The only issue with Harbor site while running 4.3.2 is that when I migrate a node it never finishes in the engine until I restart VDSMD on the original host. It did not exhibit this issue with 4.2.x

1 0

Undeploy oVirt Metrics Store
by toslavik＠yandex.ru 23 Apr '19

23 Apr '19

Deployed MetricStore on the engine and hosts as per the instructions. But using some time I realized that for me the functionality is redundant, enough data collected by DWH. https://www.ovirt.org/develop/release-management/features/metrics/metrics-s… Is there an instruction on how undeploy oVirt MetricStore?

2 4

Re: Issues post-upgrade from 4.2.8 to 4.3.2
by Vrgotic, Marko 23 Apr '19

23 Apr '19

Hey Shani, Thank you for the reply. This morning I have tried to re-provision HA Hosts. Managed successfully with HA host 2 and 3, about to do it with HA Host 1. If that goes well, I will repeat the exercises with Migration and Hosts updates to see if the issue is gone or still present. Upon those, I intend to provide full step-by-step of the Upgrade Procedure and after with what was done. Thank you for the patience. Marko Vrgotic From: Shani Leviim <sleviim(a)redhat.com> Date: Sunday, 21 April 2019 at 09:44 To: "Vrgotic, Marko" <M.Vrgotic(a)activevideo.com> Subject: Re: [ovirt-users] Issues post-upgrade from 4.2.8 to 4.3.2 Hi Marko, I've noticed that the timestamp of the error from your engine.log is one hour delayed of the one from your ui log. Is this one intentioned? Also, can you please attach a screenshot? Thanks Regards, Shani Leviim On Sun, Apr 21, 2019 at 10:12 AM Vrgotic, Marko <M.Vrgotic(a)activevideo.com<mailto:M.Vrgotic@activevideo.com>> wrote: Dear oVirt team, We have 3 Hosts HA Cluster with SHE engine. Storage is NetApp. Recently we have executed an upgrade from 4.2.8 to 4.3.2. Since then we are seeing strange behavior when trying to put Hosts to Maintenance or manually Migrate VMs between Hosts. The most repeated WARN/ERROR I am observing is following: The UI is constantly throwing an exception when trying to put HA host to maintenance or increase the SPM level. The repeated log line from engine.log I see, with regards or failing to migrate all hosts is: 2019-04-18 13:05:30,760Z WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [63d33dd1] Validation of action 'MigrateVm' failed for user SYSTEM. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL But there is more than enough. Also from UI.log: 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Permutation name: 0D2DB7A91B469CC36C64386E5632FAC5 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Uncaught exception: com.google.gwt.event.shared.Umbrell aException: Exception caught: (TypeError) : oab(...) is null at java.lang.Throwable.Throwable(Throwable.java:70) [rt.jar:1.8.0_201] at java.lang.RuntimeException.RuntimeException(RuntimeException.java:32) [rt.jar:1.8.0_201] at com.google.web.bindery.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:64) [gwt-servlet.jar:] at com.google.gwt.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:25) [gwt-servlet.jar:] at com.google.gwt.event.shared.HandlerManager.$fireEvent(HandlerManager.java:117) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.event.dom.client.DomEvent.fireNativeEvent(DomEvent.java:110) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.DOM.dispatchEvent(DOM.java:1415) [gwt-servlet.jar:] at com.google.gwt.user.client.impl.DOMImplStandard.dispatchEvent(DOMImplStandard.java:312) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.apply(Impl.java:236) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:275) [gwt-servlet.jar:] at Unknown.Su/<(https://ovirt-staging-engine.avinity.tv/ovirt-engine/webadmin/?locale=en_US… line 9 > scriptElement) at Unknown.anonymous(Unknown) Caused by: com.google.gwt.core.client.JavaScriptException: (TypeError) : oab(...) is null at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.$onAddHost(ClusterGuideModel.java:533) at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.executeCommand(ClusterGuideModel.java:617) at org.ovirt.engine.ui.uicommonweb.UICommand.$execute(UICommand.java:163) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget.$lambda$4(AbstractModelBoundPopupPresenterWidget.java:306) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget$lambda$4$Type.onClick(AbstractModelBoundPopupPresenterWidget.java:306) at com.google.gwt.event.dom.client.ClickEvent.dispatch(ClickEvent.java:55) [gwt-servlet.jar:] at com.google.gwt.event.shared.GwtEvent.dispatch(GwtEvent.java:76) [gwt-servlet.jar:] at com.google.web.bindery.event.shared.SimpleEventBus.$doFire(SimpleEventBus.java:173) [gwt-servlet.jar:] ... 12 more Can you give me some advice on how to proceed? This started happening since upgrade. Kindly awaiting your reply, Marko Vrgotic _______________________________________________ Users mailing list -- users(a)ovirt.org<mailto:users@ovirt.org> To unsubscribe send an email to users-leave(a)ovirt.org<mailto:users-leave@ovirt.org> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/7SVM3XWFREH3W…

1 0

Re: Migrate HE between ISCSI storages
by Strahil 23 Apr '19

23 Apr '19

Hi, I think you can go to the disk section in UI and migrate the disk from 1 storage domain to another. Best Regards, Strahil Nikolov

3 2

Is there a way to tell whether a permission set for a VM is inherited using REST API?
by Anton E 23 Apr '19

23 Apr '19

Hi everyone! My question is in the subject for the most part. I'm trying to backup a vm parameters, including the linked attributes, namely the permissions. But I don't want the inherited permissions to be included into the backup, I only need the attributes specific to the vm. It looks like there is no difference between the inherited and the vm-specific permissions in the output from /ovirt-engine/api/vms/<vm-id>/permissions service. What is the best way to distinguish the specific permissions from the inherited ones? It seems I can follow the link to the particular permission and see whether the vm attribute is set on it and it points to the VM I need, but that would require numerous REST requests to be done. Is there a more elegant way? Maybe I can somehow filter the output of /ovirt-engine/api/vms/<vm-id>/permissions to only include the required info? Thanks in advance. Best regards, Anton.

4 5

oVirt 4.3.3 gluster option cluster.choose-local added to 'Optimize for Virt Store'
by Strahil Nikolov 22 Apr '19

22 Apr '19

Hello Community, can someone tell me why the option cluster.choose-local was added to "Optimize for Virt Store" ?Testing my cluster after the patching showed reduced performance , due to relying constantly on network instead of using local NVME brick.I have checked my older volumes and they did not have that option. Is there any benefit to not use the local brick ?Any issues to reset that option and use local brick for reading ? Best Regards,Strahil Nikolov

1 0

Re: Is there a way to tell whether a permission set for a VM is inherited using REST API?
by Tentaclius 22 Apr '19

22 Apr '19

Hi Shani, I'm aware of this attribute, but my understanding is it only applies for a vm to template or template to vm operations. I don't see how it can be used in order to get the list of vm-specific permissions for an existing VM. Could you please provide an example of that? Thanks and regards, Anton. On Mon, Apr 22, 2019 at 2:22 PM Shani Leviim <sleviim(a)redhat.com> wrote: > Hi Anton, > In that case, how about using this one: > http://ovirt.github.io/ovirt-engine-api-model/master/#services/vms/methods/… > > > *Regards,* > > *Shani Leviim* > > > On Mon, Apr 22, 2019 at 1:35 PM Tentaclius <tentaclius(a)gmail.com> wrote: > >> Hi Shani, >> >> thanks for your reply. >> >> My question is not about assigning permissions, but rather about querying >> permissions assigned to a given VM. >> I need to get a list of permissions, assigned to the VM without the >> permissions this VM inherits from parent objects. Unluckily for me the >> /ovirt-engine/api/vms/<vm-id>/permissions service returns the list >> containing both VM-specific and inherited permissions, which cannot be >> distinguished. >> >> Thanks and regards, >> Anton. >> >> On Mon, Apr 22, 2019 at 1:29 PM Shani Leviim <sleviim(a)redhat.com> wrote: >> >>> Hi Anton, >>> You can assign permissions to the VM by using the following syntax: >>> >>> POST /ovirt-engine/api/vms/ yyy/permissions >>> >>> <permission> >>> <role> >>> <name> username</name> >>> </role> >>> <user id=" xxx"/> >>> </permission> >>> >>> For further information, you can visit here: >>> http://ovirt.github.io/ovirt-engine-api-model/master/#services/assigned_per… >>> >>> Hope it helps! >>> >>> >>> *Regards,* >>> >>> *Shani Leviim* >>> >>> >>> On Mon, Apr 22, 2019 at 1:04 PM Anton E <tentaclius(a)gmail.com> wrote: >>> >>>> Hi everyone! >>>> >>>> My question is in the subject for the most part. >>>> I'm trying to backup a vm parameters, including the linked attributes, >>>> namely the permissions. But I don't want the inherited permissions to be >>>> included into the backup, I only need the attributes specific to the vm. It >>>> looks like there is no difference between the inherited and the vm-specific >>>> permissions in the output from /ovirt-engine/api/vms/<vm-id>/permissions >>>> service. >>>> >>>> What is the best way to distinguish the specific permissions from the >>>> inherited ones? It seems I can follow the link to the particular permission >>>> and see whether the vm attribute is set on it and it points to the VM I >>>> need, but that would require numerous REST requests to be done. Is there a >>>> more elegant way? Maybe I can somehow filter the output of >>>> /ovirt-engine/api/vms/<vm-id>/permissions to only include the required info? >>>> >>>> Thanks in advance. >>>> >>>> Best regards, >>>> Anton. >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/AVHAC4SSJWVYY… >>>> >>>

1 0

Import of VMs failing - 0% progress on qemu-img
by Callum Smith 22 Apr '19

22 Apr '19

Dear All, We've exported some VMs from our old oVirt infrastructure and want to import them into the new one, but qemu-img appears to be failing. We have mounted an export domain populated from the old oVirt in the new hosted engine and are using the GUI to import the VM. Manually running the command sits at 16% CPU, 0% network usage and no progress. It appears to lock the NFS mount and ls and lsof both hang. sudo -u vdsm /usr/bin/qemu-img convert -p -t none -T none -f raw <path to source> -O raw <path to dest> Conversely a simple cp will work (ruling out file permissions errors): sudo -u vddm cp <path to source> <path to test> What might we be doing wrong? Regards, Callum -- Callum Smith Research Computing Core Wellcome Trust Centre for Human Genetics University of Oxford e. callum(a)well.ox.ac.uk<mailto:callum@well.ox.ac.uk>

2 12

Can't create pool after update to 4.3.3
by kiv＠intercom.pro 22 Apr '19

22 Apr '19

Hi all. After update my oVirt to 4.3.3 i can't create new pool. I make new template and trying to create new pool - but can't select any template, selection field is empty. I have this error: Uncaught exception occurred. Please try reloading the page. Details: Exception caught: undefined Please have your administrator check the UI logs

3 4

4.3.x upgrade and issues with OVN
by Charles Weber 22 Apr '19

22 Apr '19

Hi everyone, We have had a pair of Ovirt clusters at work, starting with the 3.x. I replaced last year with 4.x cluster on new machines. 4.2 worked great, but when I upgraded to 4.3.2 and now .3 I immediately ran into host networking issues resulting in hung or unable totally to migrate VMs. Configuration notes: 1. I use commercial star.x.x.x SSL cert on engine installed per Ovirt.org instructions 2. I have 2 untagged NICs assigned to ovirtmgmt and Public, both on the same IP range network. I also have another set of VLANs tagged on various NICs. 1 cluster uses HP blades with lots of NICs, the other uses supermicro 1Us with only 2 NICs. 3. I upgraded both engines and hosts to 4.3.2 from 4.2.8. I started seeing hung migrations cleared by restarting vdsm. 4. All computers involved run current CentOS 7.6 5. Both clusters use iscsi on dedicated tagged storage VLAN, seems fine. 6. I upgraded 1 engine and 1 host to 4.3.3 and things got worse. I have not updated any hosts or the second engine since. 7. The 4 hosts on the 4.3.3 engine now have out of sync error that refuses to clear for my Public network. 8. 2 OV 4.3.2 nodes had the following errors with 2 ovn ports. Here is example, perhaps related to the genev_sys_6081 error. ovn-d6eaa1-0: attempting to add tunnel port with same config as port 'ovn-f0f789-0' (::->137.187.160.13 ovn-877214-0: attempting to add tunnel port with same config as port 'ovn-483528-0' 9. I deleted and uninstalled all ovirt related rpms on one node, then did clean node install using latest 4.3 release. Same errors. 10. I downloaded latest node iso, installed on same host, upgraded to to 4.3.3.1 node and joined cluster. The node installation has no errors and can migrate to my other hosts. All networks are in sync. The migration hangs. Restarting VDSMD clears the hung migration. 11. None of the other hosts can migrate VMs to the new node. Here are excerpts from 3.2 node on 3.3 engine log file. Apr 17, 2019, 2:43:04 PM Check for available updates on host BRCVN3 was completed successfully with message 'ovirt-host, cockpit-ovirt-dashboard, vdsm-client, ovirt-release43, ovirt-host, ovirt-hosted-engine-setup, vdsm-api, vdsm-jsonrpc, ovirt-ansible-hosted-engine-setup, ovirt-host-dependencies and 12 others. To see all packages check engine.log.'. oVirt Apr 17, 2019, 2:43:04 PM Host BRCVN3 has available updates: ovirt-host, cockpit-ovirt-dashboard, vdsm-client, ovirt-release43, ovirt-host, ovirt-hosted-engine-setup, vdsm-api, vdsm-jsonrpc, ovirt-ansible-hosted-engine-setup, ovirt-host-dependencies and 12 others. To see all packages check engine.log.. oVirt Apr 17, 2019, 2:27:39 PM Host BRCVN3's following network(s) are not synchronized with their Logical Network configuration: Public. oVirt Apr 16, 2019, 5:53:05 PM Failed to sync all host BRCVN3 networks c1ba631c-7be5-4be0-abbb-a37b6bb7d26d oVirt Apr 16, 2019, 5:53:05 PM (1/1): Failed to apply changes on host BRCVN3. (User: admin@internal-authz) 455a260f oVirt Apr 16, 2019, 5:53:05 PM VDSM BRCVN3 command HostSetupNetworksVDS failed: Bridge Public has interfaces set([u'vnet19', u'vnet12', u'vnet10', u'vnet11', u'vnet16', u'vnet17', u'vnet14', u'vnet15', u'vnet0', u'vnet2', u'vnet3', u'vnet4', u'vnet7', u'vnet8', u'vnet9']) connected oVirt Apr 16, 2019, 5:52:59 PM (1/1): Applying network's changes on host BRCVN3. (User: admin@internal-authz) 455a260f oVirt Apr 16, 2019, 5:32:07 PM Check for available updates on host BRCVN3 was completed successfully with message 'no updates found.'. oVirt Apr 16, 2019, 5:16:40 PM Host BRCVN3's following network(s) are not synchronized with their Logical Network configuration: Public. oVirt Apr 16, 2019, 3:33:36 PM Migration failed (VM: Hwebdev, Source: BRCVN3, Destination: BRCVN4). bd21f294-0170-47d1-845e-c64c52abbab4 oVirt Apr 16, 2019, 3:33:36 PM Migration started (VM: Hwebdev, Source: BRCVN3, Destination: BRCVN4, User: admin@internal-authz). bd21f294-0170-47d1-845e-c64c52abbab4 oVirt Apr 16, 2019, 2:49:36 PM Migration failed (VM: Hwebdev, Source: BRCVN3, Destination: BRCVN4). 7c04f7a6-8f88-4c64-99f9-e621724fc7ff oVirt Apr 16, 2019, 2:49:36 PM Migration started (VM: Hwebdev, Source: BRCVN3, Destination: BRCVN4, User: admin@internal-authz). 7c04f7a6-8f88-4c64-99f9-e621724fc7ff oVirt Apr 16, 2019, 2:22:03 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:11:02 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:10:02 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:05:46 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:03:16 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:02:16 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 2:00:16 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 1:59:01 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 1:57:31 PM VDSM BRCVN3 command Get Host Statistics failed: Internal JSON-RPC error: {'reason': '[Errno 19] genev_sys_6081 is not present in the system'} oVirt Apr 16, 2019, 1:57:05 PM

2 1

Re: Help with 4.3.3 re-setup
by Strahil 21 Apr '19

21 Apr '19

Hi Michael, I know how you feel abiut VDO. I wass in the same boat until I starrted using systemd's mount units.Since then I do not have any issues. For now try the following: Boot 1 host via rescue dvd , chroot into the system and install the vdo and kvdo packages (maybe reinstall is more correct) . Once then, try to run the vdo service (still in chroot) and try to make it work. Then, just rebuild the initramfs (dracut -f -k kernel-version) and reboot. Check if your VDO, bricks and Gluster are operational again. If yes, do the other nodes. Once done - run 'gluster volume heal my-volume full' (replace my-volume with actual name). This will force gluster to sync all bricks. P.S.: Now, I'm using gluster snapshot for recovery between engine upgrades (HostedEngine is stopped while makking the snapahot). Best Regards, Strahil NikolovOn Apr 21, 2019 20:44, michael(a)wanderingmad.com wrote: > > I updated my ovirt from 4.3.2 to 4.3.3 and everything went bad, my gluster volumes were constantly dropping out and getting unsynced and everything was just running slower than molasses. So I thought I would reload the hosts from ovirt node to Centos and re-do the storage, thinking that may help. I tried using centos, but I was never able to add centos hosts back to the ovirt engine due to a gluster error in the logs, so I went and downloaded ovirt node 4.3.3, and now after spending an hour reloading all the servers with ovirt node, One of the nodes I can't access the cockpit on, and the other two have broken VDO modules, the modules are installed but don't exist in the kernel, and when I removed the module, I lost all network configuration, and when I re-installed it, I get a scriptlet error on compilation and there is still a missing VDO module. This is now the 3rd time i've hit this VDO issue and i'm really frustrated right now, should I just go back to 4.3.2? I just redownloa > ded the latest 4.3.3 ISO today. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QA2SMYRZCXQYR…

1 0

Help with 4.3.3 re-setup
by michael＠wanderingmad.com 21 Apr '19

21 Apr '19

I updated my ovirt from 4.3.2 to 4.3.3 and everything went bad, my gluster volumes were constantly dropping out and getting unsynced and everything was just running slower than molasses. So I thought I would reload the hosts from ovirt node to Centos and re-do the storage, thinking that may help. I tried using centos, but I was never able to add centos hosts back to the ovirt engine due to a gluster error in the logs, so I went and downloaded ovirt node 4.3.3, and now after spending an hour reloading all the servers with ovirt node, One of the nodes I can't access the cockpit on, and the other two have broken VDO modules, the modules are installed but don't exist in the kernel, and when I removed the module, I lost all network configuration, and when I re-installed it, I get a scriptlet error on compilation and there is still a missing VDO module. This is now the 3rd time i've hit this VDO issue and i'm really frustrated right now, should I just go back to 4.3.2? I just redownloa ded the latest 4.3.3 ISO today.

1 0

Issues post-upgrade from 4.2.8 to 4.3.2
by Vrgotic, Marko 21 Apr '19

21 Apr '19

Dear oVirt team, We have 3 Hosts HA Cluster with SHE engine. Storage is NetApp. Recently we have executed an upgrade from 4.2.8 to 4.3.2. Since then we are seeing strange behavior when trying to put Hosts to Maintenance or manually Migrate VMs between Hosts. The most repeated WARN/ERROR I am observing is following: The UI is constantly throwing an exception when trying to put HA host to maintenance or increase the SPM level. The repeated log line from engine.log I see, with regards or failing to migrate all hosts is: 2019-04-18 13:05:30,760Z WARN [org.ovirt.engine.core.bll.MigrateVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-83) [63d33dd1] Validation of action 'MigrateVm' failed for user SYSTEM. Reasons: VAR__ACTION__MIGRATE,VAR__TYPE__VM,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,SCHEDULING_ALL_HOSTS_FILTERED_OUT,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-02,$filterName Memory,$availableMem 3824,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL,VAR__FILTERTYPE__INTERNAL,$hostName ovirt-staging-hv-01,$filterName Memory,$availableMem 0,VAR__DETAIL__NOT_ENOUGH_MEMORY,SCHEDULING_HOST_FILTERED_REASON_WITH_DETAIL But there is more than enough. Also from UI.log: 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Permutation name: 0D2DB7A91B469CC36C64386E5632FAC5 2019-04-18 14:05:09,007Z ERROR [org.ovirt.engine.ui.frontend.server.gwt.OvirtRemoteLoggingService] (default task-114) [] Uncaught exception: com.google.gwt.event.shared.Umbrell aException: Exception caught: (TypeError) : oab(...) is null at java.lang.Throwable.Throwable(Throwable.java:70) [rt.jar:1.8.0_201] at java.lang.RuntimeException.RuntimeException(RuntimeException.java:32) [rt.jar:1.8.0_201] at com.google.web.bindery.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:64) [gwt-servlet.jar:] at com.google.gwt.event.shared.UmbrellaException.UmbrellaException(UmbrellaException.java:25) [gwt-servlet.jar:] at com.google.gwt.event.shared.HandlerManager.$fireEvent(HandlerManager.java:117) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.fireEvent(Widget.java:127) [gwt-servlet.jar:] at com.google.gwt.event.dom.client.DomEvent.fireNativeEvent(DomEvent.java:110) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.$onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.ui.Widget.onBrowserEvent(Widget.java:163) [gwt-servlet.jar:] at com.google.gwt.user.client.DOM.dispatchEvent(DOM.java:1415) [gwt-servlet.jar:] at com.google.gwt.user.client.impl.DOMImplStandard.dispatchEvent(DOMImplStandard.java:312) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.apply(Impl.java:236) [gwt-servlet.jar:] at com.google.gwt.core.client.impl.Impl.entry0(Impl.java:275) [gwt-servlet.jar:] at Unknown.Su/<(https://ovirt-staging-engine.avinity.tv/ovirt-engine/webadmin/?locale=en_US… line 9 > scriptElement) at Unknown.anonymous(Unknown) Caused by: com.google.gwt.core.client.JavaScriptException: (TypeError) : oab(...) is null at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.$onAddHost(ClusterGuideModel.java:533) at org.ovirt.engine.ui.uicommonweb.models.clusters.ClusterGuideModel.executeCommand(ClusterGuideModel.java:617) at org.ovirt.engine.ui.uicommonweb.UICommand.$execute(UICommand.java:163) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget.$lambda$4(AbstractModelBoundPopupPresenterWidget.java:306) at org.ovirt.engine.ui.common.presenter.AbstractModelBoundPopupPresenterWidget$lambda$4$Type.onClick(AbstractModelBoundPopupPresenterWidget.java:306) at com.google.gwt.event.dom.client.ClickEvent.dispatch(ClickEvent.java:55) [gwt-servlet.jar:] at com.google.gwt.event.shared.GwtEvent.dispatch(GwtEvent.java:76) [gwt-servlet.jar:] at com.google.web.bindery.event.shared.SimpleEventBus.$doFire(SimpleEventBus.java:173) [gwt-servlet.jar:] ... 12 more Can you give me some advice on how to proceed? This started happening since upgrade. Kindly awaiting your reply, Marko Vrgotic

1 0

VM Snapshots not erasable and not bootable
by Jonathan Baecker 21 Apr '19

21 Apr '19

Hello, I make automatically backups of my VMs and last night there was making some new one. But somehow ovirt could not delete the snapshots anymore, in the log it show that it tried the hole day to delete them but they had to wait until the merge command was done. In the evening the host was totally crashed and started again. Now I can not delete the snapshots manually and I can also not start the VMs anymore. In the web interface I get the message: VM timetrack is down with error. Exit message: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'serial': 'fd3b80fd-49ad-44ac-9efd-1328300582cd', 'index': 0, 'iface': 'scsi', 'apparentsize': '1572864', 'specParams': {}, 'cache': 'none', 'imageID': 'fd3b80fd-49ad-44ac-9efd-1328300582cd', 'truesize': '229888', 'type': 'disk', 'domainID': '9c3f06cf-7475-448e-819b-f4f52fa7d782', 'reqsize': '0', 'format': 'cow', 'poolID': '59ef3a18-002f-02d1-0220-000000000124', 'device': 'disk', 'path': '/rhev/data-center/59ef3a18-002f-02d1-0220-000000000124/9c3f06cf-7475-448e-819b-f4f52fa7d782/images/fd3b80fd-49ad-44ac-9efd-1328300582cd/47c0f42e-8bda-4e3f-8337-870899238788', 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': '47c0f42e-8bda-4e3f-8337-870899238788', 'diskType': 'file', 'alias': 'ua-fd3b80fd-49ad-44ac-9efd-1328300582cd', 'discard': False}. When I check the path permission is correct and there are also files in it. Is there any ways to fix that? Or to prevent this issue in the future? In the attachment I send also the engine.log Regards Jonathan

2 10

Re: Ovirt gluster arbiter within hosted VM
by Strahil 20 Apr '19

20 Apr '19

With GlusterD2 , you can use thin arbiter - which can be deployed in the cloud. Do not try to setup regular arbiter far away from the data bicks or yoir performance will be awful. Best Regards, Strahil NikolovOn Apr 19, 2019 13:28, Alex K <rightkicktech(a)gmail.com> wrote: > > > > On Fri, Apr 19, 2019 at 1:14 PM Scott Worthington <scott.c.worthington(a)gmail.com> wrote: >> >> And where, magically, is that node's storage going to live? > > In the disk of the VM. Let me know if I need to clarify further. >> >> >> You can't fake a proper setup of gluster. > > Not trying to fake it. Trying to find a solution with the available options. >> >> >> On Fri, Apr 19, 2019, 5:00 AM Alex K <rightkicktech(a)gmail.com> wrote: >>> >>> Hi all, >>> >>> I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. >>> >>> What do you think about this setup in regards to stability and performance? >>> I am running ovirt 4.2. >>> >>> Thanx, >>> Alex >>> _______________________________________________ >>> Users mailing list -- users(a)ovirt.org >>> To unsubscribe send an email to users-leave(a)ovirt.org >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>> oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ >>> List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/LTDOAMBKVE5BP…

2 1

Re: Ovirt gluster arbiter within hosted VM
by Strahil 20 Apr '19

20 Apr '19

As long as the VM is not hosted by those 2 hyperconverged nodes - there should be no problem. Another option is a small machine with a single SSD. I'm using Lenovo Tiny M-series as an arbiter. Best Regards, Strahil NikolovOn Apr 19, 2019 12:59, Alex K <rightkicktech(a)gmail.com> wrote: > > Hi all, > > I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. > > What do you think about this setup in regards to stability and performance? > I am running ovirt 4.2. > > Thanx, > Alex

1 0

Replica 3 distribute-replicated - data placement and fault tolerance
by Leo David 19 Apr '19

19 Apr '19

Hello Everyone, I did some fio performance tests on a particular vm and I have noticed things that I do not understand regarding how data is placed along the bricks. I am sure this is a lack of knowledge, but I would really appreciate any help in understanding this, I did a bit of research on the internet, but just could't find something relevant. I have one replica 3 distributed-replicated arbitrated volume, across 18 bricks ( 9 nodes, 2 jbods per node ). The volume was created as: node1 - brick1, node-2, brick1,.....node9-brick1, node1 - brick2, node-2, brick2,.....node9-brick2 As far as i've understood, under the hood there are sets of 3 times replicated data ( subvolumes ) which are assigned to the first 3 bricks, next set of replicated data to the next set of 3 bricks, and so on.. Now, I have this testing vm runign one node one. When I've started the fio test, i've noticed incresed gluster traffic from node 1 to node2,4 and 5 So I assumed that the vm disk is data resides on a subvolume allocated to bricks from these hosts. Then I have migrated the vm on node 2, and did the same test. Now the increased treffic is generated from node2 to node1, 4, and 5.. What I do not understand is: - why gluster client ( ovirt host ) is sending data to 3 bricks if the volume is arbitrated - shouldn't it send only to 2 of them ? - why are there 4 brick implicated in this subvolume - what would it be the fault tolerance level in this setup ,ie: how many hosts can I take down and still having the volume serving IO requests; can they be random ? I am sorry for my lack of knowledge, I am just trying to understand what is happening so I can deploy a decent proper setup of an hci environment. Thank you, Leo -- Best regards, Leo David

1 0

Re: Ovirt gluster arbiter within hosted VM
by Alex K 19 Apr '19

19 Apr '19

On Fri, Apr 19, 2019 at 1:14 PM Scott Worthington < scott.c.worthington(a)gmail.com> wrote: > And where, magically, is that node's storage going to live? > In the disk of the VM. Let me know if I need to clarify further. > > You can't fake a proper setup of gluster. > Not trying to fake it. Trying to find a solution with the available options. > > On Fri, Apr 19, 2019, 5:00 AM Alex K <rightkicktech(a)gmail.com> wrote: > >> Hi all, >> >> I have a two node hyper-converged setup which are causing me split-brains >> when network issues are encountered. Since I cannot add a third hardware >> node, I was thinking to add a dedicated guest VM hosted in same >> hyper-converged cluster which would do the arbiter for the volumes. >> >> What do you think about this setup in regards to stability and >> performance? >> I am running ovirt 4.2. >> >> Thanx, >> Alex >> _______________________________________________ >> Users mailing list -- users(a)ovirt.org >> To unsubscribe send an email to users-leave(a)ovirt.org >> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >> oVirt Code of Conduct: >> https://www.ovirt.org/community/about/community-guidelines/ >> List Archives: >> https://lists.ovirt.org/archives/list/users@ovirt.org/message/LTDOAMBKVE5BP… >> >

1 0

Ovirt gluster arbiter within hosted VM
by Alex K 19 Apr '19

19 Apr '19

Hi all, I have a two node hyper-converged setup which are causing me split-brains when network issues are encountered. Since I cannot add a third hardware node, I was thinking to add a dedicated guest VM hosted in same hyper-converged cluster which would do the arbiter for the volumes. What do you think about this setup in regards to stability and performance? I am running ovirt 4.2. Thanx, Alex

1 0

Ovirt-engine self hostd
by Dennis Perfors 19 Apr '19

19 Apr '19

Hi, We have build Ovirt-engine self hosted. We have created 2 vlans 1 services and 1 student. This works good. Services vlan 192.168.60.0/24 gateway ip 192.168.60.1 on this card Student vlan 192.168.10.0/24 gateway ip 192.168.10.1 on this network card Everything seems to be oke, but the probem is on the vlan. On the node 192.168.60.3, there is no internet connection when we do yum update -y But there is no internet connection on the host. I cant do yum update -y or do anything else. Here more information: ^Chttp://mirror.cj2.nl/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#56 - "Callback aborted" Trying other mirror. ^Chttp://mirror.sitbv.nl/centos/7.6.1810/extras/x86_64/repodata/repomd.xml: [Errno 14] curl#56 - "Callback aborted" Trying other mirror. http://mirror.serverius.net/centos/7.6.1810/extras/x86_64/repodata/repomd.x…: [Errno 14] curl#7 - "Failed to connect to 2a03:3f40:1::15: Network is unreachable" [root@bastion ~]# ping nu.nl PING nu.nl (52.85.140.48) 56(84) bytes of data. 64 bytes from server-52-85-140-48.man50.r.cloudfront.net (52.85.140.48): icmp_seq=1 ttl=241 time=12.8 ms ^C [root@bastion ~]# ping alibabia.com PING alibabia.com (137.175.26.243) 56(84) bytes of data. 64 bytes from 137.175.26.243 (137.175.26.243): icmp_seq=1 ttl=115 time=144 ms 64 bytes from 137.175.26.243 (137.175.26.243): icmp_seq=2 ttl=115 time=144 ms ^C ipv4.method: manual ipv4.dns: 1.1.1.1 ipv4.dns-search: -- ipv4.dns-options: "" ipv4.dns-priority: 0 ipv4.addresses: 192.168.60.3/24 ipv4.gateway: 192.168.60.1 ipv4.routes: -- ipv4.route-metric: -1 ipv4.route-table: 0 (unspec) ipv4.ignore-auto-routes: no ipv4.ignore-auto-dns: no public (active) target: default icmp-block-inversion: no interfaces: ovirtmgmt sources: services: ssh dhcpv6-client ovirt-postgres ovirt-https ovn-central-firewall-service ovirt-fence-kdump-listener ovirt-imageio-proxy ovirt-websocket-proxy ovirt-http ovirt-vmconsole-proxy ovirt-provider-ovn https http cockpit libvirt-tls snmp vdsm ovirt-imageio ovirt-vmconsole ports: 9986/tcp 22/tcp 6081/udp 6100/tcp protocols: masquerade: yes forward-ports: port=1337:proto=tcp:toport=22:toaddr=192.168.60.3 port=80:proto=tcp:toport=80:toaddr=192.168.60.5 source-ports: icmp-blocks: rich rules: rule family="ipv4" source address="0.0.0.0/24" accept [root@athena ~]# Portfowarding also works to the host 192.168.60.3. so that is no poiint. But only traffic from the insidehost to outside is not working. Thanks

1 0

Rebrandind Problems
by siovelrm＠gmail.com 19 Apr '19

19 Apr '19

hello I made a rebranding of my ovirt 4.3.2, but something went wrong and I saved it from the original incorrectly without realizing it. Please I need the original "ovirt.brand" and "ovirt" directories when ovirt 4.3.2 is installed. Where can I get them to be able to restore them? Greetings

2 1

Re: Expand existing gluster storage in ovirt 4.2/4.3
by Strahil 18 Apr '19

18 Apr '19

Recently , it was discussed in the mailing lists and dev mentioned that distributed replicated volumes are not officially supported, but some users use them. Even if not supported, it still should work without issues.If you think not to go this way, you can create a new 3 node cluster which will be fully suppported. Otherwise, if you go towards replicated distributed volumes , you just need to provide another set of 3 bricks and once added you can rebalance in order to distribute the files accross the sets. Here is an old thread that describes it for replica 2 volume types: https://lists.gluster.org/pipermail/gluster-users/2011-February/006599.html I guess I have confused you with my last e-mail, but that was not intentionable. Best Regards, Strahil NikolovOn Apr 17, 2019 17:13, adrianquintero(a)gmail.com wrote: > > Hi Strahil, > I had a 3 node Hyperconverged setup and added 3 new nodes to the cluster for a total of 6 servers. I am now taking advantage of more compute power, however the gluster storage part is what gets me. > > Current Hyperconverged setup: > - host1.mydomain.com > Bricks: > engine > data1 > vmstore1 > - host2.mydomain.com > Bricks: > engine > data1 > vmstore1 > - host3.mydomain.com > Bricks: > engine > data1 > vmstore1 > > - host4.mydomain.com > Bricks: > > - host5.mydomain.com > Bricks: > > - host6.mydomain.com > Bricks: > > > As you can see from the above, the original first 3 servers are the only ones that contain the gluster storage bricks, so storage redundancy is not set across all 6 nodes. I think it is a lack of understanding from my end on how ovirt and gluster integrate with one another so have a few questions: > > How would I go about achieving storage redundancy across all nodes? > Do I need to configure gluster volumes manually through the OS CLI? > If I configure the fail storage scenario manually will oVirt know about it? > > Again I know that the bricks must be added in sets of 3 and per the first 3 nodes my gluster setup looks like this (all done by hyperconverged seup in ovirt): > engine volume: host1:brick1, host2:brick1, host3:brick1 > data1 volume: host1:brick2, host2:brick2, host3:brick2 > vmstore1 volume: host1:brick3, host2:brick3, host3:brick3 > > So after adding the 3 new servers I dont know if I need to do something similar to the example in https://medium.com/@tumballi/scale-your-gluster-cluster-1-node-at-a-time-62…, if I do a similar change will oVirt know about it? will it be able to handle it as hyperconverged? > > As I mentioned before I normally see 3 node hyperconverged setup examples with gluster but have not found one for 6, 9 or 12 node cluster. > > Thanks again. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/U5T7TCSP4HFB2…

1 0

Ovirt Host Replacement/Rebuild
by Jim Kusznir 18 Apr '19

18 Apr '19

Hi all: I had an unplanned power outage (generator failed to start, power failure lasted 3 min longer than UPS batteries). One node didn't survive the unplanned power outage. By that, I mean it kernel panic's on boot, and I haven't been able to capture the KP or the first part of it (just the end), and so I don't truely know what the root cause is. I have validated the hardware is just fine, so its got to be an OS corruption. Based on this, I was thinking that perhaps the easiest way to recover would simply be to delete the host from the cluster, reformat and reinstall this host, and then add it back to the cluster as a new host. Is this in fact a good idea? Are there any references to how to do this (the detailed steps so I don't mess it up)? My cluster is (was) a 3 node hyperconverged cluster with gluster used for the management node. I also have a gluster share for VMs, but I use an NFS share from a NAS for that (which I will ask about in another post). Thanks for the help! --Jim

2 1

Importing existing GlusterFS
by Zryty ADHD 18 Apr '19

18 Apr '19

Hi, I have a questiion about that. I Install Ovirt 4.3.3 on Rhel 7.6 and want to import my existing GlusterFs cluster to it but i can't find option to do that. Can anyone explain me how to do that or its not possible in this version ?

3 5

oVirt and NetApp NFS storage
by klaasdemter＠gmail.com 18 Apr '19

18 Apr '19

Hi, I got a question regarding oVirt and the support of NetApp NFS storage. We have a MetroCluster for our virtual machine disks but a HA-Failover of that (active IP gets assigned to another node) seems to produce outages too long for sanlock to handle - that affects all VMs that have storage leases. NetApp says a "worst case" takeover time is 120 seconds. That would mean sanlock has already killed all VMs. Is anyone familiar with how we could setup oVirt to allow such storage outages? Do I need to use another type of storage for my oVirt VMs because that NFS implementation is unsuitable for oVirt? Greetings Klaas

2 4

what is the best solution for gluster?
by Edoardo Mazza 18 Apr '19

18 Apr '19

Hi all, I have 4 nodes with ovirt and gluster and I must create a new gluster volume, but I would know which is the best solution to have a high avaibility, best performance without wasting much space in the disk. Thanks for suggestions Edoardo

1 0

Changing from thin provisioned to preallocated?
by Wesley Stewart 18 Apr '19

18 Apr '19

I am currently running a ZFS server (Running RaidZ2) and I have been experimenting with NFS and shares to host my guests. I am currently running oVirt 4.2.8 and using a RaidZ2 NFS mount for the guests. ZFS definitely is definitely pretty awful (At least in my experience so far) for hosting VMs. I believe this is due to the synchronous writes being performed. However, I think running an iSCSI target with Synchronization disabled over a 10Gb connection might do the trick. (I have a couple of mirroed SSD drives for performance if I need it, but the RaidZ2 crawls for disk speed). When I tried to migrate a thin provisioned guest to iSCSI, I keep getting an "Out of disk space error" which I am pretty sure is due to the block style storage on the iSCSI target. Is there a way to switch from Thin to Preallocated? Or is my best bet to try and take a snapshot and clone this into a pre-allocated disk?

3 3

[ANN] ovirt 4.3.3 async update is now available
by Sandro Bonazzola 17 Apr '19

17 Apr '19

The oVirt Team has just released a new version of the following packages: - cockpit-ovirt-0.12.8 - ovirt-release43-4.3.3.1 - oVirt Node The async release addresses the following bugs: Bug 1697965 <https://bugzilla.redhat.com/show_bug.cgi?id=1697965> - [gluster-ansible] Order of the hosts are not preserved, when creating the volume This fix is relevant only on new deployments. -- SANDRO BONAZZOLA MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://red.ht/sig>

1 0

Re: Expand existing gluster storage in ovirt 4.2/4.3
by Strahil 17 Apr '19

17 Apr '19

You can have hyperconverged setup, but the number of bricks should be multiplied by 3. Best Regards, Strahil NikolovOn Apr 11, 2019 14:52, adrianquintero(a)gmail.com wrote: > > Thank you for the explanation, so from my understanding we can't have a hyperconverged setup if more that 3 nodes are required? or is it possible if gluster is setup separate of the oVirt node installs? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/77URQGHY7LBF4…

2 1

Problem trying to detach ISO datastore
by Eduardo Mayoral 17 Apr '19

17 Apr '19

Hi. When trying to detach an ISO datastore I get the following error. Also fails if I try to switch the datastore from maintenance to active. 2019-04-17 10:07:23,100+0000 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call StoragePool.connectStorageServer succeeded in 0.01 seconds (__init__:312) 2019-04-17 10:07:23,104+0000 INFO (jsonrpc/4) [vdsm.api] START detachStorageDomain(sdUUID=u'2722942e-0887-439c-81f0-70b60d860060', spUUID=u'6901a281-c414-42c6-8b41-3cf6e6bcd788', msdUUID=u'00000000-0000-0000-0000-000000000000', masterVersion=14, options=None) from=::fff f:10.5.72.63,58618, flow_id=0420c8d0-fb37-4b22-9fea-9ee8bac7d41d, task_id=9301405a-eaf6-4f1c-a1b6-6537740e2daf (api:48) 2019-04-17 10:07:23,105+0000 INFO (jsonrpc/4) [storage.StoragePool] sdUUID=2722942e-0887-439c-81f0-70b60d860060 spUUID=6901a281-c414-42c6-8b41-3cf6e6bcd788 (sp:1048) 2019-04-17 10:07:23,316+0000 INFO (itmap/0) [IOProcessClient] (/pre1-svm-templates.por-ngcs.lan:_ds__pre1__templates__iso01_ovirt3) Starting client (__init__:308) 2019-04-17 10:07:23,324+0000 INFO (itmap/1) [IOProcessClient] (/pre1-svm-templates.por-ngcs.lan:_ds__pre1__templates__iso01_ovirt) Starting client (__init__:308) 2019-04-17 10:07:23,327+0000 INFO (ioprocess/74639) [IOProcess] (/pre1-svm-templates.por-ngcs.lan:_ds__pre1__templates__iso01_ovirt3) Starting ioprocess (__init__:434) 2019-04-17 10:07:23,333+0000 INFO (ioprocess/74645) [IOProcess] (/pre1-svm-templates.por-ngcs.lan:_ds__pre1__templates__iso01_ovirt) Starting ioprocess (__init__:434) 2019-04-17 10:07:23,335+0000 INFO (jsonrpc/4) [IOProcessClient] (2722942e-0887-439c-81f0-70b60d860060) Starting client (__init__:308) 2019-04-17 10:07:23,344+0000 INFO (ioprocess/74653) [IOProcess] (2722942e-0887-439c-81f0-70b60d860060) Starting ioprocess (__init__:434) 2019-04-17 10:07:23,350+0000 INFO (jsonrpc/4) [storage.StorageDomain] Removing remnants of deleted images [] (fileSD:740) 2019-04-17 10:07:23,352+0000 INFO (jsonrpc/4) [vdsm.api] FINISH detachStorageDomain error=Storage domain not in pool: u'domain=2722942e-0887-439c-81f0-70b60d860060, pool=6901a281-c414-42c6-8b41-3cf6e6bcd788' from=::ffff:10.5.72.63,58618, flow_id=0420c8d0-fb37-4b22-9fea- 9ee8bac7d41d, task_id=9301405a-eaf6-4f1c-a1b6-6537740e2daf (api:52) 2019-04-17 10:07:23,352+0000 ERROR (jsonrpc/4) [storage.TaskManager.Task] (Task='9301405a-eaf6-4f1c-a1b6-6537740e2daf') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in detachStorageDomain File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 50, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 891, in detachStorageDomain pool.detachSD(sdUUID) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 1053, in detachSD self.validateAttachedDomain(dom) File "/usr/lib/python2.7/site-packages/vdsm/storage/securable.py", line 79, in wrapper return method(self, *args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/sp.py", line 551, in validateAttachedDomain raise se.StorageDomainNotInPool(self.spUUID, dom.sdUUID) StorageDomainNotInPool: Storage domain not in pool: u'domain=2722942e-0887-439c-81f0-70b60d860060, pool=6901a281-c414-42c6-8b41-3cf6e6bcd788' 2019-04-17 10:07:23,352+0000 INFO (jsonrpc/4) [storage.TaskManager.Task] (Task='9301405a-eaf6-4f1c-a1b6-6537740e2daf') aborting: Task is aborted: "Storage domain not in pool: u'domain=2722942e-0887-439c-81f0-70b60d860060, pool=6901a281-c414-42c6-8b41-3cf6e6bcd788'" - co de 353 (task:1181) 2019-04-17 10:07:23,353+0000 ERROR (jsonrpc/4) [storage.Dispatcher] FINISH detachStorageDomain error=Storage domain not in pool: u'domain=2722942e-0887-439c-81f0-70b60d860060, pool=6901a281-c414-42c6-8b41-3cf6e6bcd788' (dispatcher:83) 2019-04-17 10:07:23,353+0000 INFO (jsonrpc/4) [jsonrpc.JsonRpcServer] RPC call StorageDomain.detach failed (error 353) in 0.25 seconds (__init__:312) 2019-04-17 10:07:26,275+0000 INFO (jsonrpc/2) [api.host] START getAllVmStats() from=::ffff:10.5.72.63,58604 (api:48) 2019-04-17 10:07:26,275+0000 INFO (jsonrpc/2) [api.host] FINISH getAllVmStats return={'status': {'message': 'Done', 'code': 0}, 'statsList': (suppressed)} from=::ffff:10.5.72.63,58604 (api:54) 2019-04-17 10:07:26,275+0000 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Host.getAllVmStats succeeded in 0.01 seconds (__init__:312) 2019-04-17 10:07:29,675+0000 INFO (jsonrpc/7) [api.host] START getStats() from=::ffff:10.5.72.63,58604 (api:48) 2019-04-17 10:07:29,739+0000 INFO (jsonrpc/7) [vdsm.api] START repoStats(domains=()) from=::ffff:10.5.72.63,58604, task_id=3ec94f9f-7c98-4c49-8710-ffbfd9910ff2 (api:48) 2019-04-17 10:07:29,739+0000 INFO (jsonrpc/7) [vdsm.api] FINISH repoStats return={u'a97bbda5-33e4-4e78-890c-f7d2a572179e': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000505524', 'lastCheck': '3.6', 'valid': True}, u'4121f681-45c3-413a-9c6e-6 9d154710ace': {'code': 0, 'actual': True, 'version': 4, 'acquired': True, 'delay': '0.000790301', 'lastCheck': '2.7', 'valid': True}} from=::ffff:10.5.72.63,58604, task_id=3ec94f9f-7c98-4c49-8710-ffbfd9910ff2 (api:54) 2019-04-17 10:07:29,740+0000 INFO (jsonrpc/7) [vdsm.api] START multipath_health() from=::ffff:10.5.72.63,58604, task_id=f567e03a-8cf4-49dd-84eb-628acb10e1d3 (api:48) 2019-04-17 10:07:29,740+0000 INFO (jsonrpc/7) [vdsm.api] FINISH multipath_health return={} from=::ffff:10.5.72.63,58604, task_id=f567e03a-8cf4-49dd-84eb-628acb10e1d3 (api:54) Interestingly, I have 2 datacenters, I can attach, detach, set to maintenance or activate this same datastore on the second datacenter, but not on this one. So far I have tried updating OVF, setting master datastore and hosts to maintenance and re-activating them. Started happening on 4.3.2, updating to 4.3.3 makes no difference. This is what I see on the database. Looks OK to me: engine=# select * from storage_pool; id | name | description | storage_pool_type | storage_pool_format_type | status | master_domain_version | spm_vds_id | compatibility_version | _create_date | _update_date | quota_enforcement_type | free_text_comment | is_local --------------------------------------+-------------------+-------------+-------------------+--------------------------+--------+-----------------------+--------------------------------------+-----------------------+-------------------------------+----------------------- --------+------------------------+-------------------+---------- 039aef77-77c8-4f04-8d17-22f4b655df28 | dc-pre1-vc03-cl02 | | | 4 | 1 | 1 | 42c969f3-a2a6-4784-8506-e6fbc731da9d | 4.3 | 2019-03-14 12:13:24.411468+00 | 2019-04-17 08:38:05.53 8042+00 | 0 | | f 6901a281-c414-42c6-8b41-3cf6e6bcd788 | dc-pre1-vc03-cl01 | | | 4 | 1 | 15 | 089cdde4-ecbf-4054-82f9-c106b046d73a | 4.3 | 2019-03-14 12:13:24.365529+00 | 2019-04-17 11:10:46.42 2305+00 | 0 | | f (2 rows) engine=# select * from storage_pool_iso_map ; storage_id | storage_pool_id | status --------------------------------------+--------------------------------------+-------- 2722942e-0887-439c-81f0-70b60d860060 | 039aef77-77c8-4f04-8d17-22f4b655df28 | 6 2722942e-0887-439c-81f0-70b60d860060 | 6901a281-c414-42c6-8b41-3cf6e6bcd788 | 6 5d390949-24ec-45e2-bb6d-43b3f30bd286 | 039aef77-77c8-4f04-8d17-22f4b655df28 | 3 94cb1a4f-a267-479c-88e7-ea9c32e3fd24 | 039aef77-77c8-4f04-8d17-22f4b655df28 | 3 4121f681-45c3-413a-9c6e-69d154710ace | 6901a281-c414-42c6-8b41-3cf6e6bcd788 | 3 a97bbda5-33e4-4e78-890c-f7d2a572179e | 6901a281-c414-42c6-8b41-3cf6e6bcd788 | 3 (6 rows) Apart from root cause. I would be interested in recovering from the error, detaching the ISO storage, even if it is manually. This datacenter is not in production yet, so I have options on the things I can try. -- Eduardo Mayoral Jimeno Systems engineer, platform department. Arsys Internet. emayoral(a)arsys.es - +34 941 620 105 - ext 2153

1 1

Re: restore from dump
by Yedidyah Bar David 17 Apr '19

17 Apr '19

On Tue, Apr 16, 2019 at 10:10 PM Florian Rädler <Florian.Raedler(a)deutschebahn.com> wrote: > > Hi, > > > > is it possible to restore an engine from dump file? Define "restore an engine" and "dump file". Assuming "restore an engine" means "Install a fresh OS+engine and make it behave like an old one", and "dump file" being a pg_dump dump file, the answer is "not easily, and not fully". You will have to do something like: 1. Install OS+engine 2. Manually provision PG (init, create user/db, etc) 3. Restore your dump (pg_restore or psql, depending on format). Use the user you created, not postgres, for restoring - this will have much better chances to get permissions right 4. Run engine-setup and provide it the credentials of the database you restored It should mostly work, and I heard about people doing that, although didn't try this myself. Notably, you will have to reinstall all your hosts (move to maintenance, reinstall). If you have also a backup of /etc, that's a different case. Then, you can selectively restore files there before engine-setup. To see a list of the files you should restore, check the source of the engine-backup script - it has an internal variable called 'BACKUP_PATHS'. Good luck and best regards, -- Didi

2 1

VM migrations failing after upgrade 4.3.2 -> 4.3.3
by Eduardo Mayoral 17 Apr '19

17 Apr '19

Hi, After upgrade to 4.3.3 from 4.3.2, VM migrations are failing with "No available host to migrate VMs to" Interestingly, this happens in one of our clusters, not on the other. Both on the same datacenter, both in 4.3 compatibility version. This are the relevant lines from /var/log/ovirt-engine/engine.log 2019-04-17 08:58:41,816Z ERROR [org.ovirt.engine.core.bll.GetValidHostsForVmsQuery] (default task-7) [51191ffb-0ae1-4414-8916-9e5012d86289] Query 'GetValidHostsForVmsQuery' failed: null 2019-04-17 08:58:41,816Z ERROR [org.ovirt.engine.core.bll.GetValidHostsForVmsQuery] (default task-7) [51191ffb-0ae1-4414-8916-9e5012d86289] Exception: java.lang.NullPointerException at org.ovirt.engine.core.bll.scheduling.SchedulingManager.subtractRunningVmResources(SchedulingManager.java:923) [bll.jar:] at org.ovirt.engine.core.bll.scheduling.SchedulingManager.canSchedule(SchedulingManager.java:616) [bll.jar:] at org.ovirt.engine.core.bll.GetValidHostsForVmsQuery.lambda$getValidHosts$0(GetValidHostsForVmsQuery.java:56) [bll.jar:] at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193) [rt.jar:1.8.0_201] at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382) [rt.jar:1.8.0_201] at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481) [rt.jar:1.8.0_201] at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471) [rt.jar:1.8.0_201] at java.util.stream.ReduceOps$ReduceOp.evaluateSequential(ReduceOps.java:708) [rt.jar:1.8.0_201] at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234) [rt.jar:1.8.0_201] at java.util.stream.ReferencePipeline.collect(ReferencePipeline.java:499) [rt.jar:1.8.0_201] at org.ovirt.engine.core.bll.GetValidHostsForVmsQuery.getValidHosts(GetValidHostsForVmsQuery.java:59) [bll.jar:] at org.ovirt.engine.core.bll.GetValidHostsForVmsQuery.executeQueryCommand(GetValidHostsForVmsQuery.java:36) [bll.jar:] at org.ovirt.engine.core.bll.QueriesCommandBase.executeCommand(QueriesCommandBase.java:106) [bll.jar:] at org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) [dal.jar:] at org.ovirt.engine.core.bll.executor.DefaultBackendQueryExecutor.execute(DefaultBackendQueryExecutor.java:14) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQueryImpl(Backend.java:521) [bll.jar:] at org.ovirt.engine.core.bll.Backend.runQuery(Backend.java:490) [bll.jar:] at sun.reflect.GeneratedMethodAccessor168.invoke(Unknown Source) [:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_201] at org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) [wildfly-weld-ejb-15.0.1.Final.jar:15.0.1.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.interceptorChainCompleted(WeldInvocationContextImpl.java:107) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:126) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) [common.jar:] at sun.reflect.GeneratedMethodAccessor66.invoke(Unknown Source) [:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_201] at org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.invokeNext(WeldInvocationContextImpl.java:92) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:124) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:105) [weld-core-impl-3.0.5.Final.jar:3.0.5.Final] at org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) [wildfly-weld-ejb-15.0.1.Final.jar:15.0.1.Final] at org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:77) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89) at org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102) at org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) [bll.jar:] at sun.reflect.GeneratedMethodAccessor167.invoke(Unknown Source) [:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_201] at org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) [wildfly-ee-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:216) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:418) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:148) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) [weld-ejb-3.0.5.Final.jar:3.0.5.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-15.0.1.Final.jar:15.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:618) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at org.ovirt.engine.core.common.interfaces.BackendLocal$$$view2.runQuery(Unknown Source) [common.jar:] at org.ovirt.engine.api.restapi.resource.BackendResource.runQuery(BackendResource.java:82) at org.ovirt.engine.api.restapi.resource.BackendResource.getBackendCollection(BackendResource.java:152) at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.getBackendCollection(AbstractBackendCollectionResource.java:68) at org.ovirt.engine.api.restapi.resource.BackendHostsResource.listAll(BackendHostsResource.java:83) at org.ovirt.engine.api.restapi.resource.BackendHostsResource.list(BackendHostsResource.java:54) at org.ovirt.engine.api.resource.HostsResource.doList(HostsResource.java:130) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_201] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_201] at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:509) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:399) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:363) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:355) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:365) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:337) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:137) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:100) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:439) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:355) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at javax.servlet.http.HttpServlet.service(HttpServlet.java:791) [jboss-servlet-api_4.0_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209) at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:223) at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:149) at io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:178) at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117) at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111) at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102) at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:283) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] 2019-04-17 08:58:41,831Z ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-7) [] Operation Failed: 2019-04-17 08:58:41,831Z ERROR [org.ovirt.engine.api.restapi.resource.AbstractBackendResource] (default task-7) [] Exception: org.ovirt.engine.api.restapi.resource.BaseBackendResource$BackendFailureException: at org.ovirt.engine.api.restapi.resource.BackendResource.backendFailure(BackendResource.java:412) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.resource.BackendResource.getBackendCollection(BackendResource.java:155) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.getBackendCollection(AbstractBackendCollectionResource.java:68) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.resource.BackendHostsResource.listAll(BackendHostsResource.java:83) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.resource.BackendHostsResource.list(BackendHostsResource.java:54) [restapi-jaxrs.jar:] at org.ovirt.engine.api.resource.HostsResource.doList(HostsResource.java:130) [restapi-definition.jar:] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) [rt.jar:1.8.0_201] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) [rt.jar:1.8.0_201] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_201] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_201] at org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:139) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:509) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:399) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:363) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:355) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:365) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:337) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:137) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:100) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:439) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:355) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) [resteasy-jaxrs-3.6.2.Final.jar:3.6.2.Final] at javax.servlet.http.HttpServlet.service(HttpServlet.java:791) [jboss-servlet-api_4.0_spec-1.0.0.Final.jar:1.0.0.Final] at io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:274) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:209) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:223) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:149) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:178) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:98) [restapi-jaxrs.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:117) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:72) [restapi-jaxrs.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:47) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) [aaa.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:111) [restapi-jaxrs.jar:] at org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:102) [restapi-jaxrs.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:283) [utils.jar:] at io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:292) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:81) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:138) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:135) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1502) at io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:272) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:81) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:104) [undertow-servlet-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.Connectors.executeRootHandler(Connectors.java:360) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) [undertow-core-2.0.15.Final.jar:2.0.15.Final] at org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1985) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1487) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1378) at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_201] -- Eduardo Mayoral Jimeno Systems engineer, platform department. Arsys Internet. emayoral(a)arsys.es - +34 941 620 105 - ext 2153

2 2

Inconsistencies beteween WebUI, API read verbs and API write verbs
by Baptiste Agasse 17 Apr '19

17 Apr '19

Hi All, As I stated in a previous thread, we use instances types and wanted to automate some parts to manage it [1]. Thansk to ovirt-orb/lago and pointers given in the last thread, I'm working on ansible an module to manage ovirt instance types, but I'm facing some troubles for some of instance type attributes. From the API, I don't have access to all fields provided via WebUI (eg: Custom Emulated Machine), and it seems that some fields can be added/updated via the API but are not part of the response when you read an instance type (eg: VirtIO-SCSI Enabled). Provided examples are not exhaustive. I tried to find my way in the oVirt engine code-base to see if the fix will be easy for a non java developer like me. I find, I think, that it seems to be some API definitions for instance type [2] that define, if I understand it well, all verbs that can be applied to each resource and what field can be put when adding an instance type, but nothing about fields to be included when you read a resource (and I'm not sure if its related to v4 API). I find also some java code [3] [4] that seems related to this but As I'm not a java developer, its a little bit difficult for me to find my path in it. For me this inconsistencies are a bug, but as you stated that instance types are in deep maintenance mode, do you think that bug will get some attention ? My objective is to have a ansible module that manage the hardware/bios related stuff (memory, boot sequence, scsi controller, bios type, almost all the things) and HA for instance type. Have a nice day. Cheers. [1]: https://lists.ovirt.org/archives/list/users@ovirt.org/message/TJAXYBMSIAAAY… [2]: https://github.com/oVirt/ovirt-engine/blob/master/backend/manager/modules/r… [3]: https://github.com/oVirt/ovirt-engine/blob/ede62008318d924556bc9dfc5710d90e… [4]: https://github.com/oVirt/ovirt-engine/blob/ede62008318d924556bc9dfc5710d90e… [5]: https://ovirt.github.io/ovirt-engine-sdk/4.1/types.m.html#ovirtsdk4.types.I… -- Baptiste

2 2