December 2020 - Users - oVirt List Archives

Failed upgrade from SHE 4.3.10 to 4.4.3 - Host set to Non-Operational - missing networks
by Roberto Nunin 07 Dec '20

07 Dec '20

We are following both oVirt upgrade guide [1] and RHV 4.4 upgrade guide [2]. aps-te62-mng.corporate.it ---> host resinatlled with oVirt Node 4.4.3 aps-te61-mng.corporate.it ---> host where previous ovirt-engine 4.3.10 VM was running when backup was taken. hosted-engine --deploy --restore-from-file=<path to file> fails with following errors in ovirt-hosted-engine-setup: 2020-12-01 15:53:37,534+0100 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 fatal: [localhost]: FAILED! => {"changed": false, "msg": "The host has been set in non_operational status, please check engine logs, more info can be found in the engine logs, fix accordingly and re-deploy."} 2020-12-01 15:56:30,414+0100 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 fatal: [localhost]: FAILED! => {"changed": false, "msg": "The system may not be provisioned according to the playbook results: please check the logs for the issue, fix accordingly or re-deploy from scratch.\n"} 2020-12-01 15:56:33,731+0100 ERROR otopi.context context._executeMethod:154 Failed to execute stage 'Closing up': Failed executing ansible-playbook 2020-12-01 15:57:08,663+0100 ERROR otopi.ovirt_hosted_engine_setup.ansible_utils ansible_utils._process_output:109 fatal: [localhost]: UNREACHABLE! => {"changed": false, "msg": "Failed to connect to the host via ssh: ssh: connect to host itte1lv51-mng.comifar.it port 22: Connection timed out", "skip_reason": "Host localhost is unreachable", "unreachable": true} 2020-12-01 15:58:22,179+0100 ERROR otopi.plugins.gr_he_common.core.misc misc._terminate:167 Hosted Engine deployment failed: please check the logs for the issue, fix accordingly or re-deploy from scratch. while within the HostedEngineLocal engine.log: 2020-12-01 15:52:42,161+01 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (EE-ManagedThreadFactory-engine-Thread-96) [11f50ce0] Host ' aps-te62-mng.corporate.it' is set to Non-Operational, it is missing the following networks: 'migration,traffic_11,traffic_202,traffic_5,traffic_555,traffic_9' 2020-12-01 15:52:48,474+01 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (EE-ManagedScheduledExecutorService-engineScheduledTh readPool-Thread-12) [41688fc7] Host 'aps-te62-mng.corporate.it' is set to Non-Operational, it is missing the following networks: 'migration,traffic_11,traffic_202,traffic_5,traffic_555,traffic_9' 2020-12-01 15:52:53,734+01 ERROR [org.ovirt.engine.core.bll.SetNonOperationalVdsCommand] (EE-ManagedScheduledExecutorService-engineScheduledThreadPool-Thread-6) [5fc7257] Host 'aps-te62-mng.corporate.it' is set to Non-Operational, it is missing the following networks: 'migration,traffic_11,traffic_202,traffic_5,traffic_555,traffic_9' 2020-12-01 15:52:54,567+01 ERROR [org.ovirt.engine.core.vdsbroker.monitoring.VmsMonitoring] (ForkJoinPool-1-worker-13) [] Rerun VM 'f9249e06-237e-412c-91e9-7b0fa0b6ec2a'. Called from VDS ' aps-te62-mng.corprorate.it' 2020-12-01 15:52:54,676+01 ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engine-Thread-361) [] EVENT_ID: VM_MIGRATION_TO_SERVER_FAILED(120), Migration failed (VM: external-HostedEngineLocal, Source: aps-te62-mng.corporate.it, Destination: aps-te61-mng.corporate.it). Why is the playbook trying to migrate HostedEngineLocal from reinstalled 4.4.3 oVirt node to an existing one that is still running oVirt Node 4.3.x ? How can we manage this issue and proceed with the upgrade ? [1] https://www.ovirt.org/documentation/upgrade_guide/#SHE_Upgrading_from_4-3 [2] https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.4/ht… Thanks is advance for support. Best regards -- Roberto Nunin

4 3

strange dns request since 4.4.3
by Adam Xu 06 Dec '20

06 Dec '20

Hi ovirt Recently, my samba internal dns has been getting a lot of errors, like: [2020/12/02 15:24:31.484073, 1] ../../source4/dns_server/dns_query.c:1141(dns_server_process_query_got_auth) dns_server_process_query_got_auth: Failed to add SOA record: WERR_DNS_ERROR_RCODE_FORMAT_ERROR I found that my ovirt nodes was sending a lot of strange requests, I use tcpdump and got some like: tcpdump -i ovirtmgmt udp port 53 17:00:23.875038 IP ovirt1.adagene.cn.53862 > dc1.mydomain.com.domain: 35727+ [1au] NS? . (40) 17:00:24.436466 IP 192.168.49.195.domain > ovirt1.mydomain.com.33625: 8887 FormErr-$ 0/0/1 (40) about one request in a second. And I noticed that some nodes which version is 4.4.2 did't send dns requests like above. therefore no error occured. Were there any recent changes to ovirt node that caused these errors?

2 2

oVirt 4.4.3 or 4.4.4 rc4 Hosted Engine deploy
by jorgevisentini＠gmail.com 06 Dec '20

06 Dec '20

Hi all. I'm trying to deploy the engine, but when I get the LUNs from the hypervisor, the deploy doesn't find it. Message: "Retrieval of fiber channel LUNs failed." At the hypervisor/Linux level, the LUNs are there, both through /dev /mapper and multipath -ll. [root@kcmi1r02kvm07 ~]# ls -lh /dev/mapper/ total 0 lrwxrwxrwx. 1 root root 8 Dec 5 14:13 360002ac0000000000000002f0001f5d4 -> ../dm-12 lrwxrwxrwx. 1 root root 8 Dec 5 14:07 3600d023100094404020e199d773aa248 -> ../dm-20 lrwxrwxrwx. 1 root root 8 Dec 5 14:13 3600d02310009440404ebf5c67a1af728 -> ../dm-37 lrwxrwxrwx. 1 root root 8 Dec 5 14:07 3600d02310009440404ef2f75443b031c -> ../dm-51 lrwxrwxrwx. 1 root root 8 Dec 5 14:13 3600d02310009440406f5362702fd465f -> ../dm-22 [root@kcmi1r02kvm07 ~]# multipath -ll 3600d02310009440416067733055f176d dm-18 IFT,GS 5200 Series size=5.0T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 7:0:2:13 sdbn 68:16 active ready running | `- 8:0:0:13 sdbd 67:112 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 9:0:0:13 sday 67:32 active ready running `- 10:0:1:13 sdbi 67:192 active ready running 3600d02310009440404ebf5c67a1af728 dm-37 IFT,GS 5200 Series size=4.0T features='1 queue_if_no_path' hwhandler='1 alua' wp=rw |-+- policy='round-robin 0' prio=50 status=active | |- 9:0:0:30 sddp 71:112 active ready running | `- 10:0:1:30 sddx 71:240 active ready running `-+- policy='round-robin 0' prio=10 status=enabled |- 7:0:2:30 sdee 128:96 active ready running `- 8:0:0:30 sddr 71:144 active ready running Any tips? Thank you all!!

2 3

no disk /dev/sdb/glusters_vg_sdb/gluster_lv_engine etc...
by garcialiang.anne＠gmail.com 06 Dec '20

06 Dec '20

Hello, I've the problem with the disk /dev/sdb. After reboot my server, I losed my partition in /dev/sdb/glusters_vg_sdb/gluster_lv_engine. I've only 362cea7f05157a3002725415707fe65ec 253:0 0 9.8T 0 mpath when I look gluster volume info, I've all the volume. Why I've not sdb /dev/sdb/glusters_vg_sdb/gluster_lv_engine /dev/sdb/glusters_vg_sdb/gluster_lv_data /dev/sdb/glusters_vg_sdb/gluster_lv_vmstore How I can mount the volume ? Could you help me? Best regards Anne

2 1

new hyperconverged setup
by dofranklin＠port-orange.org 06 Dec '20

06 Dec '20

I am about to setup a new hyperconverged setup with the following 3 host setup 2 x nvme 240gb hd for OS on each host (raid) 3 x nvme 7 tb hd for vm storage on each host (no raid card) 500gb RAM per each host I am confused during the gluster setup portion using the web ui setup wizard. When asked for the LV size do I input the maximum size of the hard drive on a single host or do I combine the total capacity of the matching hard drives on all 3 host? Example that I would be putting in is hard drive: /dev/nvme1p1 capacity 7tb or should it be 21tb (combined capacity of the matching single HD on the other host). Or is there a better method you recommend? And since I am using nvme hard drive would you recommend using dedup and compression or no? Thanks

5 10

Re: Error in ovirt 4.4: RedefineVmCheckpointsVDS: Property xml is required
by Eyal Shenitzky 06 Dec '20

06 Dec '20

Hi Lukasz, Please see my comments below On Fri, 4 Dec 2020 at 17:55, Łukasz Kołaciński <l.kolacinski(a)storware.eu> wrote: > Hello, > Could you tell me more about this Error:* Failed to > RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is > required',)*. It happened on creating incremental backup using ovirt sdk. > This error means that you have a checkpoint that was created during one of your previous backups that missing the checkpoint XML in the DB, this XML is needed in order to redefine the checkpoint on the host after the VM powered-off. This situation shouldn't happen if the the backup that created checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e' was successful. Did you enable the incremental backup feature before you took the first backup for that VM? # engine-config -s "IsIncrementalBackupSupported=true" # systemctl restart ovirt-engine Do you have the logs for the backup that created checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e'? Can you please provide the content of the checkpoint from the DB? *select * from vm_checkpoints where checkpoint_id='6a569f20-f1e4-469a-8982-7f8cbfbbfc9e'* > How can I fix it? I know that is not because of wrong checkpointId because > I tried to use wrong Id on other environment and logs are different. > In case of a problem when trying to redefine one of the checkpoints, a full backup should be taken for the VM. > > Logs from this event: > > 2020-12-03 08:07:27,020Z INFO > [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: > StartVmBackupCommand internal: false. Entities affected : ID: > fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with > role type ADMIN, ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction > group BACKUP_DISK with role type ADMIN, ID: > 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK > with role type ADMIN > 2020-12-03 08:07:27,020Z INFO > [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Creating VmBackup entity for VM > 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' > 2020-12-03 08:07:27,034Z INFO > [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Created VmBackup entity > 'abb3161b-13c5-444c-b656-11e79a474342' > 2020-12-03 08:07:27,034Z INFO > [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine previous VM checkpoints for > VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' > 2020-12-03 08:07:27,045Z INFO > [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: > RedefineVmCheckpointCommand internal: true. Entities affected : ID: > fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with > role type ADMIN, ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction > group BACKUP_DISK with role type ADMIN, ID: > 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK > with role type ADMIN > 2020-12-03 08:07:27,048Z INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, > ListVmCheckpointsVDSCommand(HostName = > prd-ovirt-host-07.live.rot01.kwebbl.cloud, > VdsAndVmIDVDSParametersBase:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', > vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 55174e1c > 2020-12-03 08:07:27,052Z INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] FINISH, > ListVmCheckpointsVDSCommand, return: > [870b213b-4190-48fe-9306-5f6cb759b05a], log id: 55174e1c > 2020-12-03 08:07:27,052Z INFO > [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine VM > 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' checkpoint > '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e' > 2020-12-03 08:07:27,060Z INFO > [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, > RedefineVmCheckpointsVDSCommand(HostName = > prd-ovirt-host-07.live.rot01.kwebbl.cloud, > VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', > vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 76a75d56 > 2020-12-03 08:07:27,069Z ERROR > [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Failed in > 'RedefineVmCheckpointsVDS' method > 2020-12-03 08:07:27,077Z ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: > VDS_BROKER_COMMAND_FAILURE(10,802), VDSM > prd-ovirt-host-07.live.rot01.kwebbl.cloud command RedefineVmCheckpointsVDS > failed: General Exception: ('Property xml is required',) > 2020-12-03 08:07:27,077Z ERROR > [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command > 'RedefineVmCheckpointsVDSCommand(HostName = > prd-ovirt-host-07.live.rot01.kwebbl.cloud, > VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', > vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'})' execution failed: > VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, > error = General Exception: ('Property xml is required',), code = 100 > [...] > > 2020-12-03 08:07:27,080Z ERROR > [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command > 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand' failed: > EngineException: > org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: > VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, > error = General Exception: ('Property xml is required',), code = 100 > (Failed with error GeneralException and code 100) > 2020-12-03 08:07:27,089Z ERROR > [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Transaction rolled-back for command > 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand'. > 2020-12-03 08:07:27,090Z INFO > [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) > [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Lock freed to object > 'EngineLock:{exclusiveLocks='[64fd8dd9-d16f-4359-a4b9-8435a5abd8ca=DISK, > 794ffad6-1d1a-406a-a65e-c0600abb44f1=DISK]', > sharedLocks='[fafb9627-a20b-47a0-aece-3b4a09ce70dc=VM]'}' > 2020-12-03 08:07:27,101Z ERROR > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: > VM_INCREMENTAL_BACKUP_FAILED_FULL_VM_BACKUP_NEEDED(10,791), Incremental > backup abb3161b-13c5-444c-b656-11e79a474342 for VM prd-event-syslog-test2 > failed. Full backup is now needed (User: admin@internal-authz). > Here you can see that the engine reports the checkpoint redefinition failed and a full backup should be taken. > 2020-12-03 08:07:27,120Z INFO > [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] > (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: > VM_BACKUP_STARTED(10,790), Backup abb3161b-13c5-444c-b656-11e79a474342 for > VM prd-event-syslog-test2 started (User: admin@internal-authz). > > > Pozdrawiam, > > Łukasz Kołaciński > > Junior Java Developer > > e-mail: l.kolacinski(a)storware.eu > <m.helbert(a)storware.eu> > > > > > *[image: STORWARE]* <http://www.storware.eu/> > > > > *ul. Leszno 8/44 01-192 Warszawa www.storware.eu > <https://www.storware.eu/>* > > *[image: facebook]* <https://www.facebook.com/storware> > > *[image: twitter]* <https://twitter.com/storware> > > *[image: linkedin]* <https://www.linkedin.com/company/storware> > > *[image: Storware_Stopka_09]* > <https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> > > > > *Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa > 000510131* *, NIP 5213672602.** Wiadomość ta jest przeznaczona jedynie > dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne > i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, > przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub > podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub > podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez > pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie > tej wiadomości z wszelkich komputerów. **This message is intended only > for the person or entity to which it is addressed and may contain > confidential and/or privileged material. Any review, retransmission, > dissemination or other use of, or taking of any action in reliance upon, > this information by persons or entities other than the intended recipient > is prohibited. If you have received this message in error, please contact > the sender and remove the material from all of your computer systems.* > > -- Regards, Eyal Shenitzky

1 0

[urgent] oVirt 4.3 -> 4.4 production upgrade: OpenStack Provider regression
by Konstantin Shalygin 05 Dec '20

05 Dec '20

Create ticket for this: https://bugzilla.redhat.com/show_bug.cgi?id=1904669

1 0

[urgent] oVirt 4.3 -> 4.4 production upgrade: OpenStack Provider regression
by Konstantin Shalygin 05 Dec '20

05 Dec '20

Hello. I was upgrade our ovirt-engine from 4.3 to latest 4.4.3.12. Seems all flawless, except our storage domain (Cinder). Currently our clusters can't start VM, create Disks, resize disks, etc. Only migration works. The root cause: ovirt missing project_id in API call: ovirt 4.3 call: 17910:2020-12-02 20:43:32.087 1949 INFO eventlet.wsgi.server [req-7267b4fd-9659-4380-9297-4582ece3fe23 - - - - -] 192.168.101.10 "POST /v2/07f5bf3f6dc64b85988c3779654e175e/volumes/e3df2f84-2206-4165-9001-bcace8613315/action HTTP/1.1" status: 200 len: 777 time: 0.4810839 ovirt 4.4 call: 2020-12-05 14:09:24.155 2031 INFO eventlet.wsgi.server [req-71b47b34-8ed7-410a-812f-8f662f9f4037 - - - - -] 192.168.101.10 "POST /v2/volumes/e3df2f84-2206-4165-9001-bcace8613315/action HTTP/1.1" status: 404 len: 333 time: 0.2505009 provider configuration in engine database: engine=# select * from providers where name='cinder_ceph_backened'; -[ RECORD 1 ]---------+--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- id | 9c5800e6-5a88-403a-9c57-a501714fe816 name | cinder_ceph_backened description | OpenStack Cinder with Ceph Backend url | http://192.168.101.20:8776 provider_type | OPENSTACK_VOLUME auth_required | t auth_username | admin auth_password | OUDZtXsI4eOT69UKYI2DNqTFmN3c08XNAbbi3PQHq2Np319yURcIjhOJ81lKUo+T+pa/e6d5XUbPZmwulCK21fU5UrY2uJBSg8GXaVH23os7BmZzx+7V0V82LLBFYWUAAACeXY0hu9UGgQiMd0L7wPS0hU23iSib/BWnCcxY6h4ooQ0/pfKNZ10so5tKin/mAgMHNmX2YtqiYaQgZTYpDcIf9JnfqsiJKUW3xekPzTJQCIUEDbX/1Jpp5sJCW5aFHDSiy1I9CU/etAcqrzf6JMN8Mfn6X4VZjXqrg4YQ+QD6TiTwOAS7u7oJwCYopdRHvGNspc2YbPykN62NgFWwmg== _create_date | 2017-04-05 19:30:57.045699+07 _update_date | 2020-12-05 13:27:31.320646+07 custom_properties | tenant_name | admin plugin_type | auth_url | http://192.168.101.20:5000/v2.0 additional_properties | read_only | f is_unmanaged | f auto_sync | f user_domain_name | project_name | project_domain_name | Now we miss some field or bug in code where project_id (admin = 07f5bf3f6dc64b85988c3779654e175e) is not concat to uri? Thanks, k

1 0

Error in ovirt 4.4: RedefineVmCheckpointsVDS: Property xml is required
by Łukasz Kołaciński 04 Dec '20

04 Dec '20

Hello, Could you tell me more about this Error: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',). It happened on creating incremental backup using ovirt sdk. How can I fix it? I know that is not because of wrong checkpointId because I tried to use wrong Id on other environment and logs are different. Logs from this event: 2020-12-03 08:07:27,020Z INFO [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: StartVmBackupCommand internal: false. Entities affected : ID: fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with role type ADMIN, ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction group BACKUP_DISK with role type ADMIN, ID: 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK with role type ADMIN 2020-12-03 08:07:27,020Z INFO [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Creating VmBackup entity for VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' 2020-12-03 08:07:27,034Z INFO [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Created VmBackup entity 'abb3161b-13c5-444c-b656-11e79a474342' 2020-12-03 08:07:27,034Z INFO [org.ovirt.engine.core.bll.StartVmBackupCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine previous VM checkpoints for VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' 2020-12-03 08:07:27,045Z INFO [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Running command: RedefineVmCheckpointCommand internal: true. Entities affected : ID: fafb9627-a20b-47a0-aece-3b4a09ce70dc Type: VMAction group BACKUP_DISK with role type ADMIN, ID: 64fd8dd9-d16f-4359-a4b9-8435a5abd8ca Type: DiskAction group BACKUP_DISK with role type ADMIN, ID: 794ffad6-1d1a-406a-a65e-c0600abb44f1 Type: DiskAction group BACKUP_DISK with role type ADMIN 2020-12-03 08:07:27,048Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, ListVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VdsAndVmIDVDSParametersBase:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 55174e1c 2020-12-03 08:07:27,052Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.ListVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] FINISH, ListVmCheckpointsVDSCommand, return: [870b213b-4190-48fe-9306-5f6cb759b05a], log id: 55174e1c 2020-12-03 08:07:27,052Z INFO [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Redefine VM 'fafb9627-a20b-47a0-aece-3b4a09ce70dc' checkpoint '6a569f20-f1e4-469a-8982-7f8cbfbbfc9e' 2020-12-03 08:07:27,060Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] START, RedefineVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'}), log id: 76a75d56 2020-12-03 08:07:27,069Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Failed in 'RedefineVmCheckpointsVDS' method 2020-12-03 08:07:27,077Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM prd-ovirt-host-07.live.rot01.kwebbl.cloud command RedefineVmCheckpointsVDS failed: General Exception: ('Property xml is required',) 2020-12-03 08:07:27,077Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command 'RedefineVmCheckpointsVDSCommand(HostName = prd-ovirt-host-07.live.rot01.kwebbl.cloud, VmCheckpointsVDSParameters:{hostId='22cf5917-95b1-42e2-81c5-e8b4ddb8fd53', vmId='fafb9627-a20b-47a0-aece-3b4a09ce70dc'})' execution failed: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100 2020-12-03 08:07:27,077Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] FINISH, RedefineVmCheckpointsVDSCommand, return: , log id: 76a75d56 2020-12-03 08:07:27,077Z ERROR [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Failed to execute 'RedefineVmCheckpoints': {}: org.ovirt.engine.core.common.errors.EngineException: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100 (Failed with error GeneralException and code 100) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VdsHandler.handleVdsResult(VdsHandler.java:114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.VDSBrokerFrontendImpl.runVdsCommand(VDSBrokerFrontendImpl.java:33) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runVdsCommand(CommandBase.java:2114) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.RedefineVmCheckpointCommand.performVmCheckpointsOperation(RedefineVmCheckpointCommand.java:113) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.RedefineVmCheckpointCommand.executeCommand(RedefineVmCheckpointCommand.java:71) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1327) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:181) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:113) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1387) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:419) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:442) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:424) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runInternalAction(Backend.java:630) at jdk.internal.reflect.GeneratedMethodAccessor788.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:79) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:228) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:430) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:160) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:72) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-private@1.11.4.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view4.runInternalAction(Unknown Source) at jdk.internal.reflect.GeneratedMethodAccessor787.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:410) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:134) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.module.ejb.InjectionPointPropagatingEnterpriseTargetBeanInstance.invoke(InjectionPointPropagatingEnterpriseTargetBeanInstance.java:68) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.runInternalAction(Unknown Source) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInternalAction(CommandBase.java:2375) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.StartVmBackupCommand.executeCommand(StartVmBackupCommand.java:170) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeWithoutTransaction(CommandBase.java:1169) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeActionInTransactionScope(CommandBase.java:1327) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInSuppressed(TransactionSupport.java:140) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:79) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.execute(CommandBase.java:1387) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.CommandBase.executeAction(CommandBase.java:419) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.executor.DefaultBackendActionExecutor.execute(DefaultBackendActionExecutor.java:13) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:442) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runActionImpl(Backend.java:424) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.Backend.runAction(Backend.java:379) at jdk.internal.reflect.GeneratedMethodAccessor493.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ManagedReferenceMethodInterceptor.processInvocation(ManagedReferenceMethodInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.as.weld.ejb@19.1.0.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:92) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.interceptorChainCompleted(WeldInvocationContextImpl.java:107) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:126) at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) at jdk.internal.reflect.GeneratedMethodAccessor65.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.invokeNext(WeldInvocationContextImpl.java:92) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.WeldInvocationContextImpl.proceed(WeldInvocationContextImpl.java:124) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.bean.InterceptorImpl.intercept(InterceptorImpl.java:105) at org.jboss.as.weld.ejb@19.1.0.Final//org.jboss.as.weld.ejb.DelegatingInterceptorInvocationContext.proceed(DelegatingInterceptorInvocationContext.java:82) at org.jboss.as.weld.ejb@19.1.0.Final//org.jboss.as.weld.interceptors.EjbComponentInterceptorSupport.delegateInterception(EjbComponentInterceptorSupport.java:60) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.delegateInterception(Jsr299BindingsInterceptor.java:77) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.doMethodInterception(Jsr299BindingsInterceptor.java:89) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.interceptors.Jsr299BindingsInterceptor.processInvocation(Jsr299BindingsInterceptor.java:102) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.interceptors.UserInterceptorFactory$1.processInvocation(UserInterceptorFactory.java:63) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.bll.interceptors.CorrelationIdTrackerInterceptor.aroundInvoke(CorrelationIdTrackerInterceptor.java:13) at jdk.internal.reflect.GeneratedMethodAccessor70.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ManagedReferenceLifecycleMethodInterceptor.processInvocation(ManagedReferenceLifecycleMethodInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.invocationmetrics.ExecutionTimeInterceptor.processInvocation(ExecutionTimeInterceptor.java:43) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.concurrent.ConcurrentContextInterceptor.processInvocation(ConcurrentContextInterceptor.java:45) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InitialInterceptor.processInvocation(InitialInterceptor.java:40) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.interceptors.ComponentDispatcherInterceptor.processInvocation(ComponentDispatcherInterceptor.java:52) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.singleton.SingletonComponentInstanceAssociationInterceptor.processInvocation(SingletonComponentInstanceAssociationInterceptor.java:53) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInNoTx(CMTTxInterceptor.java:228) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:430) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:160) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) at org.jboss.as.weld.common@19.1.0.Final//org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3@19.1.0.Final//org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.elytron-private@1.11.4.Final//org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:627) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation@1.5.2.Final//org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee@19.1.0.Final//org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at deployment.engine.ear.bll.jar//org.ovirt.engine.core.common.interfaces.BackendLocal$$$view3.runAction(Unknown Source) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.BackendResource.doAction(BackendResource.java:251) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.doCreateEntity(AbstractBackendCollectionResource.java:244) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:144) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:134) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.AbstractBackendCollectionResource.performCreate(AbstractBackendCollectionResource.java:153) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.resource.BackendVmBackupsResource.add(BackendVmBackupsResource.java:55) at org.ovirt.engine.api.restapi-definition//org.ovirt.engine.api.resource.VmBackupsResource.doAdd(VmBackupsResource.java:80) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.MethodInjectorImpl.invoke(MethodInjectorImpl.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceMethodInvoker.internalInvokeOnTarget(ResourceMethodInvoker.java:535) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTargetAfterFilter(ResourceMethodInvoker.java:424) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceMethodInvoker.lambda$invokeOnTarget$0(ResourceMethodInvoker.java:385) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invokeOnTarget(ResourceMethodInvoker.java:387) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:150) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:110) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:141) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:110) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invokeOnTargetObject(ResourceLocatorInvoker.java:141) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.ResourceLocatorInvoker.invoke(ResourceLocatorInvoker.java:104) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:440) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$invoke$4(SynchronousDispatcher.java:229) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.SynchronousDispatcher.lambda$preprocess$0(SynchronousDispatcher.java:135) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.interception.PreMatchContainerRequestContext.filter(PreMatchContainerRequestContext.java:356) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.SynchronousDispatcher.preprocess(SynchronousDispatcher.java:138) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.core.SynchronousDispatcher.invoke(SynchronousDispatcher.java:215) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.service(ServletContainerDispatcher.java:227) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:56) at org.jboss.resteasy.resteasy-jaxrs@3.11.2.Final//org.jboss.resteasy.plugins.server.servlet.HttpServletDispatcher.service(HttpServletDispatcher.java:51) at javax.servlet.api@2.0.0.Final//javax.servlet.http.HttpServlet.service(HttpServlet.java:590) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletHandler.handleRequest(ServletHandler.java:74) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:81) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:251) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchToPath(ServletInitialHandler.java:186) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.spec.RequestDispatcherImpl.forwardImpl(RequestDispatcherImpl.java:227) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.spec.RequestDispatcherImpl.forwardImplSetup(RequestDispatcherImpl.java:149) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.spec.RequestDispatcherImpl.forward(RequestDispatcherImpl.java:111) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:167) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.VersionFilter.doFilter(VersionFilter.java:87) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:106) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.invocation.CurrentFilter.doFilter(CurrentFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.RestApiSessionMgmtFilter.doFilter(RestApiSessionMgmtFilter.java:78) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.EnforceAuthFilter.doFilter(EnforceAuthFilter.java:42) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SsoRestApiNegotiationFilter.doFilter(SsoRestApiNegotiationFilter.java:84) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SsoRestApiAuthFilter.doFilter(SsoRestApiAuthFilter.java:46) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.SessionValidationFilter.doFilter(SessionValidationFilter.java:59) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.aaa//org.ovirt.engine.core.aaa.filters.RestApiSessionValidationFilter.doFilter(RestApiSessionValidationFilter.java:35) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:100) at org.ovirt.engine.api.restapi-jaxrs//org.ovirt.engine.api.restapi.security.CSRFProtectionFilter.doFilter(CSRFProtectionFilter.java:91) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at org.ovirt.engine.core.utils//org.ovirt.engine.core.utils.servlet.CORSSupportFilter.doFilter(CORSSupportFilter.java:272) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ManagedFilter.doFilter(ManagedFilter.java:61) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler$FilterChainImpl.doFilter(FilterHandler.java:131) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.FilterHandler.handleRequest(FilterHandler.java:84) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.ServletSecurityRoleHandler.handleRequest(ServletSecurityRoleHandler.java:62) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletChain$1.handleRequest(ServletChain.java:68) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletDispatchingHandler.handleRequest(ServletDispatchingHandler.java:36) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.security.SecurityContextAssociationHandler.handleRequest(SecurityContextAssociationHandler.java:78) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.RedirectDirHandler.handleRequest(RedirectDirHandler.java:68) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.SSLInformationAssociationHandler.handleRequest(SSLInformationAssociationHandler.java:132) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.ServletAuthenticationCallHandler.handleRequest(ServletAuthenticationCallHandler.java:57) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.1.0.Final//io.undertow.security.handlers.AuthenticationConstraintHandler.handleRequest(AuthenticationConstraintHandler.java:53) at io.undertow.core@2.1.0.Final//io.undertow.security.handlers.AbstractConfidentialityHandler.handleRequest(AbstractConfidentialityHandler.java:46) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.ServletConfidentialityConstraintHandler.handleRequest(ServletConfidentialityConstraintHandler.java:64) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.ServletSecurityConstraintHandler.handleRequest(ServletSecurityConstraintHandler.java:59) at io.undertow.core@2.1.0.Final//io.undertow.security.handlers.AuthenticationMechanismsHandler.handleRequest(AuthenticationMechanismsHandler.java:60) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.security.CachedAuthenticatedSessionHandler.handleRequest(CachedAuthenticatedSessionHandler.java:77) at io.undertow.core@2.1.0.Final//io.undertow.security.handlers.NotificationReceiverHandler.handleRequest(NotificationReceiverHandler.java:50) at io.undertow.core@2.1.0.Final//io.undertow.security.handlers.AbstractSecurityContextAssociationHandler.handleRequest(AbstractSecurityContextAssociationHandler.java:43) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.security.jacc.JACCContextIdHandler.handleRequest(JACCContextIdHandler.java:61) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.core@2.1.0.Final//io.undertow.server.handlers.PredicateHandler.handleRequest(PredicateHandler.java:43) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.handleFirstRequest(ServletInitialHandler.java:269) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$100(ServletInitialHandler.java:78) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:133) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler$2.call(ServletInitialHandler.java:130) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ServletRequestContextThreadSetupAction$1.call(ServletRequestContextThreadSetupAction.java:48) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.core.ContextClassLoaderSetupAction$1.call(ContextClassLoaderSetupAction.java:43) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.security.SecurityContextThreadSetupAction.lambda$create$0(SecurityContextThreadSetupAction.java:105) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541) at org.wildfly.extension.undertow@19.1.0.Final//org.wildfly.extension.undertow.deployment.UndertowDeploymentInfoService$UndertowThreadSetupAction.lambda$create$0(UndertowDeploymentInfoService.java:1541) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.dispatchRequest(ServletInitialHandler.java:249) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler.access$000(ServletInitialHandler.java:78) at io.undertow.servlet@2.1.0.Final//io.undertow.servlet.handlers.ServletInitialHandler$1.handleRequest(ServletInitialHandler.java:99) at io.undertow.core@2.1.0.Final//io.undertow.server.Connectors.executeRootHandler(Connectors.java:370) at io.undertow.core@2.1.0.Final//io.undertow.server.HttpServerExchange$1.run(HttpServerExchange.java:830) at org.jboss.threads@2.3.3.Final//org.jboss.threads.ContextClassLoaderSavingRunnable.run(ContextClassLoaderSavingRunnable.java:35) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor.safeRun(EnhancedQueueExecutor.java:1982) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.doRunTask(EnhancedQueueExecutor.java:1486) at org.jboss.threads@2.3.3.Final//org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1377) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100 at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.createDefaultConcreteException(VdsBrokerCommand.java:81) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.createException(BrokerCommandBase.java:224) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.BrokerCommandBase.proceedProxyReturnValue(BrokerCommandBase.java:194) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.RedefineVmCheckpointsVDSCommand.executeVdsBrokerCommand(RedefineVmCheckpointsVDSCommand.java:20) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVdsCommandWithNetworkEvent(VdsBrokerCommand.java:123) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.VdsBrokerCommand.executeVDSCommand(VdsBrokerCommand.java:111) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.VDSCommandBase.executeCommand(VDSCommandBase.java:65) at org.ovirt.engine.core.dal//org.ovirt.engine.core.dal.VdcCommandBase.execute(VdcCommandBase.java:31) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.vdsbroker.DefaultVdsCommandExecutor.execute(DefaultVdsCommandExecutor.java:14) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager.runVdsCommand(ResourceManager.java:398) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand$$super(Unknown Source) at jdk.internal.reflect.GeneratedMethodAccessor81.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.TerminalAroundInvokeInvocationContext.proceedInternal(TerminalAroundInvokeInvocationContext.java:51) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.AroundInvokeInvocationContext.proceed(AroundInvokeInvocationContext.java:78) at org.ovirt.engine.core.common//org.ovirt.engine.core.common.di.interceptor.LoggingInterceptor.apply(LoggingInterceptor.java:12) at jdk.internal.reflect.GeneratedMethodAccessor65.invoke(Unknown Source) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:566) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.reader.SimpleInterceptorInvocation$SimpleMethodInvocation.invoke(SimpleInterceptorInvocation.java:73) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeAroundInvoke(InterceptorMethodHandler.java:84) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.executeInterception(InterceptorMethodHandler.java:72) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.interceptor.proxy.InterceptorMethodHandler.invoke(InterceptorMethodHandler.java:56) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:79) at org.jboss.weld.core@3.1.3.Final//org.jboss.weld.bean.proxy.CombinedInterceptorAndDecoratorStackMethodHandler.invoke(CombinedInterceptorAndDecoratorStackMethodHandler.java:68) at deployment.engine.ear//org.ovirt.engine.core.vdsbroker.ResourceManager$Proxy$_$$_WeldSubclass.runVdsCommand(Unknown Source) ... 291 more 2020-12-03 08:07:27,080Z ERROR [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Command 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand' failed: EngineException: org.ovirt.engine.core.vdsbroker.vdsbroker.VDSErrorException: VDSGenericException: VDSErrorException: Failed to RedefineVmCheckpointsVDS, error = General Exception: ('Property xml is required',), code = 100 (Failed with error GeneralException and code 100) 2020-12-03 08:07:27,089Z ERROR [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Transaction rolled-back for command 'org.ovirt.engine.core.bll.RedefineVmCheckpointCommand'. 2020-12-03 08:07:27,090Z INFO [org.ovirt.engine.core.bll.RedefineVmCheckpointCommand] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] Lock freed to object 'EngineLock:{exclusiveLocks='[64fd8dd9-d16f-4359-a4b9-8435a5abd8ca=DISK, 794ffad6-1d1a-406a-a65e-c0600abb44f1=DISK]', sharedLocks='[fafb9627-a20b-47a0-aece-3b4a09ce70dc=VM]'}' 2020-12-03 08:07:27,101Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VM_INCREMENTAL_BACKUP_FAILED_FULL_VM_BACKUP_NEEDED(10,791), Incremental backup abb3161b-13c5-444c-b656-11e79a474342 for VM prd-event-syslog-test2 failed. Full backup is now needed (User: admin@internal-authz). 2020-12-03 08:07:27,120Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-506) [e3b57751-64d4-4ec1-8dc2-a9586c41826a] EVENT_ID: VM_BACKUP_STARTED(10,790), Backup abb3161b-13c5-444c-b656-11e79a474342 for VM prd-event-syslog-test2 started (User: admin@internal-authz). Pozdrawiam, Łukasz Kołaciński Junior Java Developer e-mail: l.kolacinski(a)storware.eu<mailto:l.kolacinski@storware.eu> <mailto:m.helbert@storware.eu> [STORWARE]<http://www.storware.eu/> ul. Leszno 8/44 01-192 Warszawa www.storware.eu <https://www.storware.eu/> [facebook]<https://www.facebook.com/storware> [twitter]<https://twitter.com/storware> [linkedin]<https://www.linkedin.com/company/storware> [Storware_Stopka_09]<https://www.youtube.com/channel/UCKvLitYPyAplBctXibFWrkw> Storware Spółka z o.o. nr wpisu do ewidencji KRS dla M.St. Warszawa 000510131 , NIP 5213672602. Wiadomość ta jest przeznaczona jedynie dla osoby lub podmiotu, który jest jej adresatem i może zawierać poufne i/lub uprzywilejowane informacje. Zakazane jest jakiekolwiek przeglądanie, przesyłanie, rozpowszechnianie lub inne wykorzystanie tych informacji lub podjęcie jakichkolwiek działań odnośnie tych informacji przez osoby lub podmioty inne niż zamierzony adresat. Jeżeli Państwo otrzymali przez pomyłkę tę informację prosimy o poinformowanie o tym nadawcy i usunięcie tej wiadomości z wszelkich komputerów. This message is intended only for the person or entity to which it is addressed and may contain confidential and/or privileged material. Any review, retransmission, dissemination or other use of, or taking of any action in reliance upon, this information by persons or entities other than the intended recipient is prohibited. If you have received this message in error, please contact the sender and remove the material from all of your computer systems.

1 0

edit /var/lib/ovirt-hosted-engine-ha/broker.conf
by jb 04 Dec '20

04 Dec '20

Hello, I have try to edit the file */var/lib/ovirt-hosted-engine-ha/broker.conf*, but when I restart the service *ovirt-ha-broker*, the old settings are reverted. Is there a way to change the settings in this file? Best regard Jonathan

1 1