- Users - oVirt List Archives

Re: oVirt Metrics Store Installation - Error during SSO authentication
by Strahil 15 Oct '19

15 Oct '19

This one worked for me: curl --cacert ~/API/ca-cert -X GET -H "Accept: application/xml" -u 'admin@internal' 'https://engine.localdomain/ovirt-engine/api/URL-more' So, try without the password... Best Regards, Strahil NikolovOn Oct 15, 2019 10:54, Markus Schaufler <markus.schaufler(a)digit-all.at> wrote: > > I also cannot authenticate e.g. with > > curl -u admin@internal:pass https://xxxx.at/ovirt-engine/api > > access_denied: Cannot authenticate user 'ADMIN@INTERNAL': No valid profile found in credentials. > > How can I specify the profile? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FRUQZ3S63EZCJ…

2 1

disk locked after export as OVA
by adam_xu＠adagene.com.cn 15 Oct '19

15 Oct '19

Hello, everyone. I tried to export a vm as OVA. I got a error in webui: VDSM ovirt1.ntbaobei.com command HSMGetAllTasksStatusesVDS failed: Volume Group not big enough: (u'Not enough free extents for extending LV d9f2378f-92f0-4bc1-96a8-20a0f2c575cb/c515a3f9-0590-4ebe-81c5-9d0993f1fec9 (free=848, needed=872)',) seems no enough space in the storage domain. I deleted the vm which i wanted to export as OVA before, but I saw a disk with id 8140d2e7-6908-438e-a646-23e58a33913e left in the storage and the status is locked. here's some log in the engine.log: 2018-09-17 14:27:03,467+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-107849) [a407f6a9-445d-4ccc-b998-f9dfc6dc67ec] START, CopyImageVDSCommand( CopyImageVDSCommandParameters:{storagePoolId='79432500-ad45-11e8-98f3-00163e188641', ignoreFailoverLimit='false', storageDomainId='d9f2378f-92f0-4bc1-96a8-20a0f2c575cb', imageGroupId='ee55bbe7-8001-4c82-b1a6-0cac7710704b', imageId='701a7472-8c00-4b9f-aa87-4837eb128695', dstImageGroupId='8140d2e7-6908-438e-a646-23e58a33913e', vmId='b3aa623e-3072-430e-87d6-b81a3b07f466', dstImageId='c515a3f9-0590-4ebe-81c5-9d0993f1fec9', imageDescription='', dstStorageDomainId='d9f2378f-92f0-4bc1-96a8-20a0f2c575cb', copyVolumeType='LeafVol', volumeFormat='COW', preallocate='Sparse', postZero='false', discard='false', force='true'}), log id: 164e3046 2018-09-17 14:27:03,467+08 INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CopyImageVDSCommand] (EE-ManagedThreadFactory-engine-Thread-107849) [a407f6a9-445d-4ccc-b998-f9dfc6dc67ec] ++ dstImageGUID=8140d2e7-6908-438e-a646-23e58a33913e I think maybe some command did not been excuted after I export the vm as OVA. I can find the similar case here: https://access.redhat.com/solutions/1298893 I have used the tool unlock_entity.sh but cannot find any disk that is locked. So what should I do to delete the disk that is been locked? yours Adam

2 1

Re: Clean up engine database?
by Strahil 15 Oct '19

15 Oct '19

Can someonw clarify whether the DB will grow forever , if we never do a vacuum (either with these commands or via the engine-upgrade)? Best Regards, Strahil NikolovOn Oct 14, 2019 17:03, Lucie Leistnerova <lleistne(a)redhat.com> wrote: > > > On 10/14/19 3:36 PM, Chris Adams wrote: > > Once upon a time, Lucie Leistnerova <lleistne(a)redhat.com> said: > >> Ah, you are right. dwh-vacuum was added in 4.2. > > Just for interest, I tried this on my 4.3.6 dev setup, and it just > > gives: > > > > [root@engine ~]# dwh-vacuum > > /usr/bin/dwh-vacuum: line 47: vacuumdb: command not found > > > > Looks like it needs the SCL enablement that engine-vacuum has. > > My mistake, I thought it was fixed already but it was only for > engine-vacuum. > I've created new BZ https://bugzilla.redhat.com/show_bug.cgi?id=1761494 > > Thanks for catching this! > > -- > Lucie Leistnerova > Senior Quality Engineer, QE Cloud, RHVM > Red Hat EMEA > > IRC: lleistne @ #rhev-qe > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/5WGD552G3GKGS…

2 1

Clean up engine database?
by Chris Adams 14 Oct '19

14 Oct '19

On my oVirt 4.1.9 cluster, the hosted engine filled up its disk last night (it was deployed from the engine appliance image and had a 20G root and 5G swap). I gained a little space by trimming down swap, but I need to clean it up. It looks like the problem is the Postgres database - /var/lib/pgsql/data is using 16G. Is there some cleanup I can do (and then some way to keep it from happening again)? Also, is there a way to extend the hosted engine disk? It's on iSCSI storage, and the LUN has free space. -- Chris Adams <cma(a)cmadams.net>

4 7

Bridge issue
by con_louca＠hotmail.com 14 Oct '19

14 Oct '19

Hello all, I am trying to setup a bridge. I have 3 virtual machines. The 1st acting as router, the second acting as bridge and the 3rd is the client, like the following diagram ROUTER BRIDGE CLIENT internet(ovirtmng network) ----- [eth0 router --eth1 ] -------[eth0 --[br0]-- eth1] -------[--eth Windows 10] The problem is that windows 10 is not able to connect the internet although is able to allocate ip. I use ovirt external network to connect the VMs to each other. It seems that Windows 10 sends ARP to BRIDGE, the BRIDGE sends the ARP to ROUTER but the replies are not passing the br0 when it comes a reply from ROUTER to Windows 10. I use oVirt 4.3.6. Do you know what should be the issue ? Is there a chance that ovirt blocking the bridge? Thanks

4 7

New Certificate -> Image-IO-Proxy Errors
by Markus Schaufler 14 Oct '19

14 Oct '19

Hi, I've changed the cert to an official cert using the howto at https://www.ovirt.org/documentation/admin-guide/appe-oVirt_and_SSL.html Got two problems/questions now: 1. The image-io-proxy doesn't work now 2. I tried to upload an ISO image, which is stuck and locked now @1: Which logs should I watch and are there any troubleshooting tips at front? @2: I've already tried the "unlock_entity.sh" script, which didn't help. So I tried to login to the CLI to have a look, if there's some kind of "upload" job still active - but there's no more CLI. How can I check actual running or stuck jobs? Many thanks in advance

2 1

Re: change connection string in db
by Strahil 14 Oct '19

14 Oct '19

I think that there is a special file on the shared storage (something like OVA) that stores the necessary data. Have you tried with: hosted-engine --set-shared-config storage <server1>:/engine hosted-engine --set-shared-config mnt_options backup-volfile-servers=<server2>:<server3> Best Regards, Strahil NikolovOn Sep 30, 2019 19:14, olaf.buitelaar(a)gmail.com wrote: > > Dear oVirt users, > > one thing i still cannot find out, is where the engine gathers the storage= value from in the /etc/ovirt-hosted-engine/hosted-engine.conf > I suppose it's somewhere in a answers file, but i cannot find it. > Any points are appreciated. hopfully this is the last place where the old connection string lives. > > Thanks Olaf > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GPPKRN5YLUMEJ…

2 1

Import OVA Error - HE version Version 4.3.5.4-1.el7
by paul.christian.suba＠cevalogistics.com 14 Oct '19

14 Oct '19

Hi, We have 2 oVirt Lab environments. Trying to copy the VM templates or OVA files to Lab2 from Lab1. I have copied the ovas in the NFS share of LAB1 to the NFS ISO Domain of Lab2. The NFS ISO Domain of Lab2 can be seen mounted on all hosts via df -h. If I do a template import with OVA as source and the /rhev/data-center/mnt/XX_XX defined path of the ISP domain, I either get nothing or an error. I get nothing if I do not include the file name. If i include the file name, i get "Uncaught exception occurred. Please try reloading the page. Details: (TYpeError): Cannot read property 'ob' of null. Please have your administrator check the UI logs" I created an NFS ExportDomain in lab2. Copied the files again from the NFS appliance. Did a repeat of the steps, but selected export domain. Same results.

1 0

Re: Delete snapshots task hung
by Strahil 13 Oct '19

13 Oct '19

Nah... It's done directly on the DB and I wouldn't recommend such action for Production Cluster. I've done it only once and it was based on some old mailing lists. Maybe someone from the dev can assist?On Oct 10, 2019 13:31, Leo David <leoalex(a)gmail.com> wrote: > > Thank you Strahil, > Could you tell me what do you mean by changing status ? Is this something to be done in the UI ? > > Thanks, > > Leo > > On Thu, Oct 10, 2019, 09:55 Strahil <hunter86_bg(a)yahoo.com> wrote: >> >> Maybe you can change the status of the VM in order the engine to know that it has to blockcommit the snapshots. >> >> Best Regards, >> Strahil Nikolov >> >> On Oct 9, 2019 09:02, Leo David <leoalex(a)gmail.com> wrote: >>> >>> Hi Everyone, >>> Please let me know if any thoughts or recommandations that could help me solve this issue.. >>> The real bad luck in this outage is that these 5 vms are part on an Openshift deployment, and now we are not able to start it up... >>> Before trying to sort this at ocp platform level by replacing the failed nodes with new vms, I would rather prefer to do it at the oVirt level and have the vms starting since the disks are still present on gluster. >>> Thank you so much ! >>> >>> >>> Leo

2 1

Change self-hosted engine storage domain
by Strahil 12 Oct '19

12 Oct '19

Hello All, Recently I wanted to enable nested virtualization on my cluster and thus I had to redeploy the first host. Sadly, some time ago I have taken the shortcut and changed the values in /etc/ovirt-hosted-engine/hosted-engine.conf locally in order to replace the gluster host and mount options. Thus, after the redeploy, I had to restore the old config in order to be able to mount the shared storage. Now I want to make the proper changes and according to an old thread I need to run the following commands , but they fail. Can some one advise what is the correct syntax ? [root@ovirt1 ~]# hosted-engine --set-shared-config storage gluster1:/engine Duplicate key storage, please specify the key type. Available types are: ['he_local', 'he_shared'] [root@ovirt1 ~]# hosted-engine --set-shared-config mnt_options backup-volfile-servers=gluster2:ovirt3 Duplicate key mnt_options, please specify the key type. Available types are: ['he_local', 'he_shared'] Best Regards, Strahil Nikolov

2 1

Damaged hard disk in volume replica gluster
by matteo fedeli 12 Oct '19

12 Oct '19

Hi, I have in my ovirt HCI a volume that not work properly because one of three hdd fail. I bought a new hdd, I recreate old lvm partitioning and mount point. Now, how can I attach this empty new brick?

1 0

Move Hosted Engine to external oVirt or VMware
by Markus Schaufler 11 Oct '19

11 Oct '19

Hi, I've got a oVirt Infrastructure with 4 nodes and a self-hosted engine. The hosted engine also manages additional 4 nodes in 4 datacenters. Two tasks: - Re installation of our infrastructure without interfering or interrupting the external nodes. - Separate the external hosts from our infrastructure by either leaving them in the old hosted engine or setting up a new hosted engine on VMware or the new Ovirt Infrastructure We've already drained 2 of our 4 nodes, re-installed them with ovirt node and a new self-hosted engine and moved our VMs onto it. On the two remaining nodes only runs the old self-hosted engine which still manages the 4 external hosts - which prevents me from reinstalling that remaining hosts. I'd prefer to move the old HE to the new oVirt Installation or to our VMware infrastructure. Is that possible and what pitfalls do I have to care of? Many thanks for any hint on this! Markus

2 1

FOSDEM 2020 Virtualization & IaaS Devroom CfP
by Piotr Kliczewski 11 Oct '19

11 Oct '19

We are excited to announce that the call for proposals is now open for the Virtualization & IaaS devroom at the upcoming FOSDEM 2020, to be hosted on February 1st 2020. This year will mark FOSDEM’s 20th anniversary as one of the longest-running free and open source software developer events, attracting thousands of developers and users from all over the world. FOSDEM will be held once again in Brussels, Belgium, on February 1st & 2nd, 2020. This devroom is a collaborative effort, and is organized by dedicated folks from projects such as OpenStack, Xen Project, oVirt, QEMU, KVM, and Foreman. We would like to invite all those who are involved in these fields to submit your proposals by December 1st, 2019. About the Devroom The Virtualization & IaaS devroom will feature session topics such as open source hypervisors and virtual machine managers such as Xen Project, KVM, bhyve, and VirtualBox, and Infrastructure-as-a-Service projects such as KubeVirt, Apache CloudStack, OpenStack, oVirt, QEMU and OpenNebula. This devroom will host presentations that focus on topics of shared interest, such as KVM; libvirt; shared storage; virtualized networking; cloud security; clustering and high availability; interfacing with multiple hypervisors; hyperconverged deployments; and scaling across hundreds or thousands of servers. Presentations in this devroom will be aimed at developers working on these platforms who are looking to collaborate and improve shared infrastructure or solve common problems. We seek topics that encourage dialog between projects and continued work post-FOSDEM. Important Dates Submission deadline: 1 December 2019 Acceptance notifications: 10 December 2019 Final schedule announcement: 15th December 2019 Devroom: 1st February 2020 Submit Your Proposal All submissions must be made via the Pentabarf event planning site[1]. If you have not used Pentabarf before, you will need to create an account. If you submitted proposals for FOSDEM in previous years, you can use your existing account. After creating the account, select Create Event to start the submission process. Make sure to select Virtualization and IaaS devroom from the Track list. Please fill out all the required fields, and provide a meaningful abstract and description of your proposed session. Submission Guidelines We expect more proposals than we can possibly accept, so it is vitally important that you submit your proposal on or before the deadline. Late submissions are unlikely to be considered. All presentation slots are 30 minutes, with 20 minutes planned for presentations, and 10 minutes for Q&A. All presentations will be recorded and made available under Creative Commons licenses. In the Submission notes field, please indicate that you agree that your presentation will be licensed under the CC-By-SA-4.0 or CC-By-4.0 license and that you agree to have your presentation recorded. For example: "If my presentation is accepted for FOSDEM, I hereby agree to license all recordings, slides, and other associated materials under the Creative Commons Attribution Share-Alike 4.0 International License. Sincerely, <NAME>." In the Submission notes field, please also confirm that if your talk is accepted, you will be able to attend FOSDEM and deliver your presentation. We will not consider proposals from prospective speakers who are unsure whether they will be able to secure funds for travel and lodging to attend FOSDEM. (Sadly, we are not able to offer travel funding for prospective speakers.) Submission Guidelines Mentored presentations will have 25-minute slots, where 20 minutes will include the presentation and 5 minutes will be reserved for questions. The number of newcomer session slots is limited, so we will probably not be able to accept all applications. You must submit your talk and abstract to apply for the mentoring program, our mentors are volunteering their time and will happily provide feedback but won't write your presentation for you! If you are experiencing problems with Pentabarf, the proposal submission interface, or have other questions, you can email our devroom mailing list[2] and we will try to help you. How to Apply In addition to agreeing to video recording and confirming that you can attend FOSDEM in case your session is accepted, please write "speaker mentoring program application" in the "Submission notes" field, and list any prior speaking experience or other relevant information for your application. Code of Conduct Following the release of the updated code of conduct for FOSDEM, we'd like to remind all speakers and attendees that all of the presentations and discussions in our devroom are held under the guidelines set in the CoC and we expect attendees, speakers, and volunteers to follow the CoC at all times. If you submit a proposal and it is accepted, you will be required to confirm that you accept the FOSDEM CoC. If you have any questions about the CoC or wish to have one of the devroom organizers review your presentation slides or any other content for CoC compliance, please email us and we will do our best to assist you. Call for Volunteers We are also looking for volunteers to help run the devroom. We need assistance watching time for the speakers, and helping with video for the devroom. Please contact devroom mailing list [2] for more information. Questions? If you have any questions about this devroom, please send your questions to our devroom mailing list. You can also subscribe to the list to receive updates about important dates, session announcements, and to connect with other attendees. See you all at FOSDEM! [1] <https://penta.fosdem.org/submission/FOSDEM17> https://penta.fosdem.org/submission/FOSDEM20 [2] iaas-virt-devroom at lists.fosdem.org

1 0

Re: [ovirt-devel] Creating a custom NON-VM Network using the Ovirt SDK
by Dominik Holler 11 Oct '19

11 Oct '19

On Thu, 10 Oct 2019 14:11:29 -0000 "Jamie Holohan" <james.holohan(a)oracle.com> wrote: > If i'm creating a a custom network on OVM using the UI, one of the options in the pop up is a check box labelled "VM network" for determining if I want a VM network. > "VM network" is modeled as usage in the sdk, so an empty attribute "usages" does the trick: system_service = connection.system_service() networks_service = system_service.networks_service() networks_service.add( types.Network( name='custom_network', data_center=types.DataCenter( name='Default' ), usages=[] ) ) > When using the SDK I am building network with a NetworkBuilder, but I cannot see any methods to toggle the VM network option. How do I create a custom Network with not VM role using the SDK? > > Can it be done with a NetworkBuilder? > _______________________________________________ > Devel mailing list -- devel(a)ovirt.org > To unsubscribe send an email to devel-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/devel@ovirt.org/message/UPR2B3IOGB33R…

1 0

Ovirt 4.3.5/6 automated install fails
by adrianquintero＠gmail.com 11 Oct '19

11 Oct '19

I am having issues while trying to deploy an automated HC install ================================ ansible 2.8.2 config file = /etc/ansible/ansible.cfg configured module search path = [u'/root/.ansible/plugins/modules', u'/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python2.7/site-packages/ansible executable location = /usr/bin/ansible python version = 2.7.5 (default, Jun 20 2019, 20:27:34) [GCC 4.8.5 20150623 (Red Hat 4.8.5-36)] Ovirt node = 4.3.5 or 4.3.5 1.- Followed the procedure listed in the following link: https://access.redhat.com/documentation/en-us/red_hat_hyperconverged_infras… 2.-Ran the playbook within Path: /etc/ansible/roles/gluster.ansible/playbooks/hc-ansible-deployment: ansible-playbook -i gluster_inventory.yml hc_deployment.yml --extra-vars='@he_gluster_vars.json' 3.-Json file [root@host1 hc-ansible-deployment]# cat he_gluster_vars.json { "he_appliance_password": "changeme", "he_admin_password": "changeme", "he_domain_type": "glusterfs", "he_fqdn": "ovirt-engine.example.com", "he_vm_mac_addr": "00:16:fe:05:e1:ee", "he_default_gateway": "10.10.10.1", "he_mgmt_network": "ovirtmgmt", "he_ansible_host_name": "host1.example.com", "he_storage_domain_name": "HostedEngine", "he_storage_domain_path": "/engine", "he_storage_domain_addr": "vmm10.virt.aid3p", "he_mount_options": "backup-volfile-servers=host2.example.com:host3.example.com", "he_bridge_if": "eno49", "he_enable_hc_gluster_service": true, "he_mem_size_MB": "32768", "he_cluster": "Default", } Error: task path: /usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml:8 fatal: [localhost]: FAILED! => { "msg": "The task includes an option with an undefined variable. The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'he_host_ip'\n\nThe error appears to be in '/usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml': line 8, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n timeout: 180\n - name: Add an entry for this host on /etc/hosts on the local VM\n ^ here\n" } task path: /usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml:8 fatal: [localhost]: FAILED! => { "msg": "The task includes an option with an undefined variable. The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'he_host_ip'\n\nThe error appears to be in '/usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml': line 8, column 5, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n timeout: 180\n - name: Add an entry for this host on /etc/hosts on the local VM\n ^ here\n" } task path: /usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml:57 fatal: [localhost]: FAILED! => { "msg": "The task includes an option with an undefined variable. The error was: 'ansible.vars.hostvars.HostVarsVars object' has no attribute 'he_local_vm_dir'\n\nThe error appears to be in '/usr/share/ansible/roles/ovirt.hosted_engine_setup/tasks/bootstrap_local_vm/03_engine_initial_tasks.yml': line 57, column 7, but may\nbe elsewhere in the file depending on the exact syntax problem.\n\nThe offending line appears to be:\n\n delegate_to: \"{{ he_ansible_host_name }}\"\n - name: Get local VM dir path\n ^ here\n" } PLAY RECAP ****************************************************************************************************************************************************************************** localhost : ok=184 changed=56 unreachable=0 failed=2 skipped=65 rescued=0 ignored=0 host1.example.com : ok=34 changed=17 unreachable=0 failed=0 skipped=70 rescued=0 ignored=0 host2.example.com : ok=37 changed=20 unreachable=0 failed=0 skipped=100 rescued=0 ignored=0 host3.example.com : ok=34 changed=17 unreachable=0 failed=0 skipped=70 rescued=0 ignored=0 So it seems that for some strange reason it is not getting the correct he_host_ip and he_local_vm_dir. has anybody else encountered this error? -Adrian

2 5

Bridge Issue
by Constantinos L 11 Oct '19

11 Oct '19

Hello all, I am trying to setup a bridge. I have 3 virtual machines. The 1st acting as router, the second acting as bridge and the 3rd is the client, like the following diagram ROUTER BRIDGE CLIENT internet(ovirtmng network) ----- [eth0 router --eth1 ] -------[eth0 --[br0]-- eth1] -------[--eth Windows 10] The problem is that windows 10 is not able to connect the internet although is able to allocate ip. I use ovirt external network to connect the VMs to each other. It seems that Windows 10 sends ARP to BRIDGE, the BRIDGE sends the ARP to ROUTER but the replies are not passing the br0 when it comes a reply from ROUTER to Windows 10. I use oVirt 4.3.6. Do you know what should be the issue ? Is there a chance that ovirt blocking the bridge?

1 0

Re: owner of vm paused/unpaused operation
by Strahil 11 Oct '19

11 Oct '19

You can always check the queue_if_no_path multipath.conf option and give it a try. But if your system that is queue-ing get's rebooted - that will be data loss -> use on your own risk. Don't forget that the higher in I/O chain you go - the higher the timeout is needed, so your VM should also use multipath with that option, in addition to the host. Still, we can't help you if you use that feature and you loose data. Best Regards, Strahil NikolovOn Oct 10, 2019 10:55, Francesco Romani <fromani(a)redhat.com> wrote: > > On 10/10/19 9:07 AM, Gianluca Cecchi wrote: > > >> > How is determined the timeout to use to put the VM in pause mode? >> >> >> The VM is paused immediately as soon as libvirt, through QEMU, reports >> IOError, to avoid data corruption. Now, when libvirt reports this error >> >> depends laregly on the timeout set for the storage configuration, which >> is done at host level, using system tools (e.g. it is not a Vdsm tunable) >> > > For test I have set this in multipath.conf of host: > > devices { > device { > all_devs yes > # Set timeout of queuing of 5*28 = 140 seconds > # similar to vSphere APD timeout > # no_path_retry fail > no_path_retry 28 > polling_interval 5 > } > > So it should wait at least 140 seconds before passing error to upper layer correct? > > > AFAICT yes > > >> >>> >>> > Sometimes I see after clearing the problems that the VM is >>> > automatically un-paused, sometimes no: how is this managed? >>> >> >> I noticed that if I set disk as virtio-scsi (it seems virtio has no timeout definable and passes suddenly the error to upper layer) and disk timeout of vm disk (through udev rule) to 180 seconds, I can block access to the storage for example for 100 seconds and the host is able to reinstate paths and then vm is always unpaused. >> But I would like to prevent VM from pausing at all >> What else to tweak? > > > The only way Vdsm will not pause the VM is if libvirt+qemu never reports any ioerror, which is something I'm not sure is possible and that I'd never recommend anyway. > > Vdsm always tries hard to be super-careful with respect possible data corruption. > > > Bests, > > > -- > > Francesco Romani > > Senior SW Eng., Virtualization R&D > > Red Hat > > IRC: fromani github: @fromanirh

2 1

iSCSI storage domain with no switch
by MIMMIK _ 10 Oct '19

10 Oct '19

Is it possibile to have a storage domain on an iSCSI LUN if the storage is connected to the cluster physical nodes with no switch and only direct connections? This is the scenario: - we have two physical nodes with 2 iSCSI ports each. - the storage has 2 controllers, 2 iSCSI ports on each controller - each server is directly connected with a port to a storage port on a controller and to another storage port on the other controller. Do you think is this possibile, maybe using any trick or workaround? Thanks. Regards

1 0

Re: Trying to add a LUN to a host and use that, instead of Gluster, as the datastore for my VM's.
by Strahil 10 Oct '19

10 Oct '19

You can create an NFS on that host and export to itself that local storage... Best Regards, Strahil NikolovOn Oct 9, 2019 07:43, TomK <tomkcpr(a)mdevsys.com> wrote: > > Manged to reverse it through the options. Had to deactivate the custom > local cluster. > > Now I have my GlusterFS back and want to try and add some local storage > to each host under the same DC, Cluster and Storage Domain. So > effectively I will have two storage locations. One on my GlusterFS and > the other on the local storage I've just defined. > > Appears I can't make them coexist with oVirt. Either the host is placed > in a separate cluster with local storage, or I move it back to the > GlusterFS. > > Appears I can't have two usable storage types under one host. Is this > correct? > > What I'm looking for is in the image and shared earlier. > > Cheers, > TK > > On 10/8/2019 11:03 PM, TomK wrote: > > I'm working to reverse this scenario. > > > > My storage domain still exists. However I can't activate it. The > > Manage button is greyed out without clear indication as to why. > > Difficult to tell what my next move should be. > > > > Anyway to find out? > > > > Cheers, > > TK > > > > On 10/7/2019 8:25 AM, TomK wrote: > >> Allright. > >> > >> So I followed this and configured local storage. It had some default > >> names that I changed. It would have been either the default name, or > >> one I choose. In either case it would be a different name then the > >> Gluster storage domain the host was a part of anyway. > >> > >> Once the process completed for both hosts, I noticed the Gluster > >> volume storage domain was offline and the gluster volume was gone. I > >> can't seem to add that back in anymore. I get: > >> > >> "Error while executing action DisconnectStorageServerConnection: Error > >> storage server disconnection" > >> > >> Guessing I can't have local storage in addition to Gluster on the same > >> hosts and available for VM's? > >> > >> Reason why I need that is that Gluster is slow but provides redundancy > >> and live migration. But I also wanted direct storage for VM's > >> requiring faster IO. So here's my scenario: > >> > >> host01 > >> /dev/sda OS > >> /dev/sdb 4TB (For Gluster) > >> /dev/sdc 4TB (For Local Storage) > >> > >> host02 > >> /dev/sda OS > >> /dev/sdb 4TB (For Gluster) > >> /dev/sdc 4TB (For Local Storage) > >> > >> I would like to have: > >> > >> 1) GlusterFS volume available to oVirt via the two /dev/sdb drives in > >> both physical hosts. > >> 2) Two locally attached LUN's, each separate and local to that > >> physical host. > >> > >> Is this possible? > >> > >> Cheers, > >> TK > >> > >> On 10/6/2019 12:38 AM, Strahil wrote: > >>> Hi Tom, > >>> > >>> Have you checked > >>> https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/ht… > >>> ? > >>> > >>> Best Regards, > >>> Strahil NikolovOn Oct 6, 2019 06:26, TomK <tomkcpr(a)mdevsys.com> wrote: > >>>> > >>>> Hey All, > >>>> > >>>> I've added a 4TB LUN to my physical storage and now I want to use that, > >>>> instead of GlusterFS as my datastore for VM's on that physical. How do > >>>> I do this? > >>>> > >>>> I've tried a number of options, including adding LUN's but appears I > >>>> can > >>>> only add them to VM's directly but not use them as my datastores > >>>> instead > >>>> of GlusterFS. > >>>> > >>>> -- > >>>> Thx, > >>>> TK. > >>>> _______________________________________________ > >>>> Users mailing list -- users(a)ovirt.org > >>>> To unsubscribe send an email to users-leave(a)ovirt.org > >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >>>> oVirt Code of Conduct: > >>>> https://www.ovirt.org/community/about/community-guidelines/ > >>>> List Archives: > >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZES6FCONNYBXY… > >>>> > >>> _______________________________________________ > >>> Users mailing list -- users(a)ovirt.org > >>> To unsubscribe send an email to users-leave(a)ovirt.org > >>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > >>> oVirt Code of Conduct: > >>> https://www.ovirt.org/community/about/community-guidelines/ > >>> List Archives: > >>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/WE2MPSRXTJCXD… > >>> > >>> > >> > >> > > > > > > > -- > Thx, > TK. > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/QJDCKXESPUMVL…

3 2

owner of vm paused/unpaused operation
by Gianluca Cecchi 10 Oct '19

10 Oct '19

Hello, I'm doing some tests related to storage latency or problems manually created to debug and manage reactions of hosts and VMs. What is the subsystem/process/daemon responsible to pause a VM when problems arise on storage for the host where the VM is running? How is determined the timeout to use to put the VM in pause mode? Sometimes I see after clearing the problems that the VM is automatically un-paused, sometimes no: how is this managed? Are there any counters so that if VM has been paused and and problems are not solved in a certain timeframe the unpause can be done only manually by the sysadmin? Thanks in advance, Gianluca

2 7

Hyperconverged setup questions
by Vrgotic, Marko 10 Oct '19

10 Oct '19

Dear oVirt, Is it possible to add oVirt 3Hosts/Gluster hyperconverged cluster to existing oVirt setup? I need this to achieve Local storage performance, but still have pool of Hypevisors available. Is it possible to have more than 3Hosts in Hyperconverged setup? I have currently 1Shared Cluster (NFS based storage, where also SHE is hosted) and 2Local Storage clusters. oVirt current version running is 4.3.4. Kindly awaiting your reply. — — — Met vriendelijke groet / Kind regards, Marko Vrgotic ActiveVideo

1 0

Re: Delete snapshots task hung
by Strahil 10 Oct '19

10 Oct '19

Maybe you can change the status of the VM in order the engine to know that it has to blockcommit the snapshots. Best Regards, Strahil NikolovOn Oct 9, 2019 09:02, Leo David <leoalex(a)gmail.com> wrote: > > Hi Everyone, > Please let me know if any thoughts or recommandations that could help me solve this issue.. > The real bad luck in this outage is that these 5 vms are part on an Openshift deployment, and now we are not able to start it up... > Before trying to sort this at ocp platform level by replacing the failed nodes with new vms, I would rather prefer to do it at the oVirt level and have the vms starting since the disks are still present on gluster. > Thank you so much ! > > > Leo

2 1

Upgrading from oVirt-Node 4.2.8 to latest
by Sven Achtelik 10 Oct '19

10 Oct '19

Hi All, is there a way to go from 4.2.8 on ovirt node to go directly to the latest version, without reinstalling the node from the iso file ? I wasn't able to find anything in documentation on how to get this done. Thanks, Sven

5 8

Bridge issue
by Constantinos L 10 Oct '19

10 Oct '19

Hello all, I am trying to setup a bridge. I have 3 virtual machines. The 1st acting as router, the second acting as bridge and the 3rd is the client, like the following diagram ROUTER BRIDGE CLIENT internet(ovirtmng network) ----- [eth0 router --eth1 ] -------[eth0 --[br0]-- eth1] -------[--eth Windows 10] The problem is that windows 10 is not able to connect the internet although is able to allocate ip. I use ovirt external network to connect the VMs to each other. It seems that Windows 10 sends ARP to BRIDGE, the BRIDGE sends the ARP to ROUTER but the replies are not passing the br0 when it comes a reply from ROUTER to Windows 10. I use oVirt 4.3.6. Do you know what should be the issue ? Is there a chance that ovirt blocking the bridge? Thanks

1 0

Fail to attach/create disk from template when running via Ansible
by Vrgotic, Marko 10 Oct '19

10 Oct '19

Dear oVirt, Just recently, I have upgraded oVirt staging environment from 4.3.4 to 4.3.6 release. Since then I noticed the issue when creating VMs from template, using Ansible. When creating VM from template using UI, all works well. Tests were executed with same user, with SuperUser priviledges. pip freeze ansible==2.7.13 asn1crypto==1.0.0 bcrypt==3.1.7 cffi==1.12.3 cryptography==2.7 dnspython==1.16.0 ipaddress==1.0.22 Jinja2==2.10.1 lxml==4.4.1 MarkupSafe==1.1.1 netaddr==0.7.19 ovirt-engine-sdk-python==4.3.3 paramiko==2.6.0 pycparser==2.19 pycurl==7.43.0.3 PyNaCl==1.3.0 PyYAML==5.1.2 six==1.12.0 Logs from engine: 2019-10-08 09:57:42,294Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Lock Acquired to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:57:42,299Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43687) [390b4c75] FINISH, HSMClearTaskVDSCommand, return: , log id: be932a0 2019-10-08 09:57:42,299Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43687) [390b4c75] FINISH, SPMClearTaskVDSCommand, return: , log id: 3e25883 2019-10-08 09:57:42,303Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43687) [390b4c75] BaseAsyncTask::removeTaskFromDB: Removed task '7e1d68a2-bb49-4456-b228-db851fe6603c' from DataBase 2019-10-08 09:57:42,303Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43687) [390b4c75] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity '4f8a6897-66e8-453b-b706-246cb8505b3e' 2019-10-08 09:57:42,375Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Running command: AddVmCommand internal: false. Entities affected : ID: 2e428504-c339-11e9-87ef-00163e3ec101 Type: ClusterAction group CREATE_VM with role type USER, ID: 9ac6f4ad-58d0-4a7e-b424-91f2d76abcac Type: VmTemplateAction group CREATE_VM with role type USER, ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: StorageAction group CREATE_DISK with role type USER 2019-10-08 09:57:42,488Z INFO [org.ovirt.engine.core.bll.AddRngDeviceCommand] (default task-23) [332c57d4] Running command: AddRngDeviceCommand internal: true. Entities affected : ID: d5719cf0-2a64-4a28-bbf2-edc1bb05b777 Type: VMAction group EDIT_VM_PROPERTIES with role type USER 2019-10-08 09:57:42,498Z INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-23) [332c57d4] START, SetVmStatusVDSCommand( SetVmStatusVDSCommandParameters:{vmId='d5719cf0-2a64-4a28-bbf2-edc1bb05b777', status='ImageLocked', exitStatus='Normal'}), log id: 2fe004e2 2019-10-08 09:57:42,507Z INFO [org.ovirt.engine.core.vdsbroker.SetVmStatusVDSCommand] (default task-23) [332c57d4] FINISH, SetVmStatusVDSCommand, return: , log id: 2fe004e2 2019-10-08 09:57:42,528Z INFO [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Running command: CreateSnapshotFromTemplateCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:57:42,554Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateVolumeVDSCommand] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] START, CreateVolumeVDSCommand( CreateVolumeVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', imageSizeInBytes='8589934592', volumeFormat='COW', newImageId='9a96018c-ce7d-4c77-a91a-30fc16c7b2ba', imageType='Sparse', newImageDescription='', imageInitialSizeInBytes='0', imageId='f9e0c4d9-81dd-4e44-8a71-33e55d4399c6', sourceImageGroupId='a7e42574-be60-4c94-94a5-cc4b30fdb16f'}), log id: 37b29c7d 2019-10-08 09:57:42,639Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.CreateVolumeVDSCommand] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] FINISH, CreateVolumeVDSCommand, return: 9a96018c-ce7d-4c77-a91a-30fc16c7b2ba, log id: 37b29c7d 2019-10-08 09:57:42,645Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'f1665b66-2032-4a60-944e-c04beace088e' 2019-10-08 09:57:42,645Z INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] CommandMultiAsyncTasks::attachTask: Attaching task '99abc310-b44b-435c-8d43-922b782a69dc' to command 'f1665b66-2032-4a60-944e-c04beace088e'. 2019-10-08 09:57:42,660Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Adding task '99abc310-b44b-435c-8d43-922b782a69dc' (Parent Command 'CreateSnapshotFromTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2019-10-08 09:57:42,934Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] EVENT_ID: USER_ADD_VM_STARTED(37), VM centos-testvm-024 creation was initiated by mteam@internal-authz. 2019-10-08 09:57:42,935Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (default task-23) [067ec489-ae6c-4871-8ab6-8296016ca1ce] BaseAsyncTask::startPollingTask: Starting to poll task '99abc310-b44b-435c-8d43-922b782a69dc'. 2019-10-08 09:57:44,384Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-74) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Command 'AddVm' (id: '1b65f576-365d-4e8b-a920-621516cbdaed') waiting on child command id: 'f1665b66-2032-4a60-944e-c04beace088e' type:'CreateSnapshotFromTemplate' to complete 2019-10-08 09:57:48,393Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Command 'AddVm' (id: '1b65f576-365d-4e8b-a920-621516cbdaed') waiting on child command id: 'f1665b66-2032-4a60-944e-c04beace088e' type:'CreateSnapshotFromTemplate' to complete 2019-10-08 09:57:48,394Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'AddDisk' id: '937776b8-b94d-4178-b9c2-e39fe42cb06c' child commands '[63409b0b-749a-45d5-85a5-baf55d5fa2a4]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:48,394Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'AddDisk' id: '937776b8-b94d-4178-b9c2-e39fe42cb06c' Updating status to 'SUCCEEDED', The command end method logic will be executed by one of its parent commands. 2019-10-08 09:57:48,398Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'AddDisk' id: 'fdc5571e-a925-4400-a54a-a1ef1dcd23ae' child commands '[54d68cae-6585-4351-aaf7-fe687905acf4]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:48,398Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'AddDisk' id: 'fdc5571e-a925-4400-a54a-a1ef1dcd23ae' Updating status to 'SUCCEEDED', The command end method logic will be executed by one of its parent commands. 2019-10-08 09:57:48,401Z INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'ProcessOvfUpdateForStorageDomain' (id: 'bdcdae3a-00f6-4176-9d27-0bb0f04912a3') waiting on child command id: 'b4050edd-62a7-4b83-a4ab-a21c83bf3ade' type:'CreateOvfVolumeForStorageDomain' to complete 2019-10-08 09:57:48,402Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'CreateOvfVolumeForStorageDomain' id: 'b4050edd-62a7-4b83-a4ab-a21c83bf3ade' child commands '[937776b8-b94d-4178-b9c2-e39fe42cb06c]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:48,404Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-37) [5830ba58] Command 'CreateOvfVolumeForStorageDomain' id: '5fd47910-95b2-4387-8e26-ea2c80b078a0' child commands '[fdc5571e-a925-4400-a54a-a1ef1dcd23ae]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:49,416Z INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [390b4c75] Command 'ProcessOvfUpdateForStorageDomain' id: '849bd16c-00d2-48a2-8fde-631af0dfe1b5' child commands '[a80a3657-c813-42c1-a43d-27442b5dc238, 4f8a6897-66e8-453b-b706-246cb8505b3e]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:49,418Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStorageDomainCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStorageDomainCommand' successfully. 2019-10-08 09:57:49,420Z INFO [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.disk.AddDiskCommand' successfully. 2019-10-08 09:57:49,426Z INFO [org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand' successfully. 2019-10-08 09:57:49,427Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='66984468-bbfa-4941-8c6b-df72a6d920f2', imageId='7db55211-041a-40c8-85a6-a2ff1bd182ce'}), log id: 6c02a73c 2019-10-08 09:57:49,429Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] START, GetVolumeInfoVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, GetVolumeInfoVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c', storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='66984468-bbfa-4941-8c6b-df72a6d920f2', imageId='7db55211-041a-40c8-85a6-a2ff1bd182ce'}), log id: 2e131d9d 2019-10-08 09:57:49,456Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@26992dd0, log id: 2e131d9d 2019-10-08 09:57:49,456Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@26992dd0, log id: 6c02a73c 2019-10-08 09:57:49,464Z WARN [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [] VM is null - no unlocking 2019-10-08 09:57:49,481Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [] EVENT_ID: USER_ADD_DISK_FINISHED_SUCCESS(2,021), The disk 'OVF_STORE' was successfully added. 2019-10-08 09:57:49,490Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStorageDomainCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.CreateOvfVolumeForStorageDomainCommand' successfully. 2019-10-08 09:57:49,491Z INFO [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.disk.AddDiskCommand' successfully. 2019-10-08 09:57:49,495Z INFO [org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.disk.image.AddImageFromScratchCommand' successfully. 2019-10-08 09:57:49,497Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] START, GetImageInfoVDSCommand( GetImageInfoVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='ee208fd3-e05c-4ba8-aba9-fa521d69a870', imageId='484c1eed-b03a-4fa0-a12d-0087294dec86'}), log id: 7a3dad31 2019-10-08 09:57:49,498Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] START, GetVolumeInfoVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, GetVolumeInfoVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c', storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='ee208fd3-e05c-4ba8-aba9-fa521d69a870', imageId='484c1eed-b03a-4fa0-a12d-0087294dec86'}), log id: 5b85c4a5 2019-10-08 09:57:49,522Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.GetVolumeInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] FINISH, GetVolumeInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@13555e4, log id: 5b85c4a5 2019-10-08 09:57:49,522Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.GetImageInfoVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [5830ba58] FINISH, GetImageInfoVDSCommand, return: org.ovirt.engine.core.common.businessentities.storage.DiskImage@13555e4, log id: 7a3dad31 2019-10-08 09:57:49,529Z WARN [org.ovirt.engine.core.bll.storage.disk.AddDiskCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [] VM is null - no unlocking 2019-10-08 09:57:49,540Z INFO [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-3) [] EVENT_ID: USER_ADD_DISK_FINISHED_SUCCESS(2,021), The disk 'OVF_STORE' was successfully added. 2019-10-08 09:57:50,552Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [390b4c75] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand' successfully. 2019-10-08 09:57:50,575Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='ee208fd3-e05c-4ba8-aba9-fa521d69a870', imageId='484c1eed-b03a-4fa0-a12d-0087294dec86'}), log id: 7e77a9a8 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGroupGUID=ee208fd3-e05c-4ba8-aba9-fa521d69a870 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=484c1eed-b03a-4fa0-a12d-0087294dec86 2019-10-08 09:57:50,576Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ description={"Updated":false,"Last Updated":null,"Storage Domains":[{"uuid":"935de3ac-a735-4fc8-9161-26bfd751ffc7"}],"Disk Description":"OVF_STORE"} 2019-10-08 09:57:50,606Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 7e77a9a8 2019-10-08 09:57:50,614Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[bfc4c379-bd5f-45be-98a4-f0469dc3817c=VDS_EXECUTION]'}' 2019-10-08 09:57:50,623Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Running command: UploadStreamCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:57:50,624Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, UploadStreamVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, UploadStreamVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c'}), log id: 531a4a4a 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeVdsBrokerCommand, parameters: 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGUID=ee208fd3-e05c-4ba8-aba9-fa521d69a870 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=484c1eed-b03a-4fa0-a12d-0087294dec86 2019-10-08 09:57:50,625Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ size=30720 2019-10-08 09:57:51,331Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, UploadStreamVDSCommand, return: , log id: 531a4a4a 2019-10-08 09:57:51,342Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'c11f4d3c-f85b-43c5-986f-a517aed66d7e' 2019-10-08 09:57:51,342Z INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] CommandMultiAsyncTasks::attachTask: Attaching task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' to command 'c11f4d3c-f85b-43c5-986f-a517aed66d7e'. 2019-10-08 09:57:51,351Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Adding task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2019-10-08 09:57:51,355Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] BaseAsyncTask::startPollingTask: Starting to poll task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d'. 2019-10-08 09:57:51,358Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[bfc4c379-bd5f-45be-98a4-f0469dc3817c=VDS_EXECUTION]'}' 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='ee208fd3-e05c-4ba8-aba9-fa521d69a870', imageId='484c1eed-b03a-4fa0-a12d-0087294dec86'}), log id: 238169f6 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGroupGUID=ee208fd3-e05c-4ba8-aba9-fa521d69a870 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=484c1eed-b03a-4fa0-a12d-0087294dec86 2019-10-08 09:57:51,360Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ description={"Updated":true,"Size":30720,"Last Updated":"Tue Oct 08 09:57:50 UTC 2019","Storage Domains":[{"uuid":"935de3ac-a735-4fc8-9161-26bfd751ffc7"}],"Disk Description":"OVF_STORE"} 2019-10-08 09:57:51,389Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 238169f6 2019-10-08 09:57:51,596Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='66984468-bbfa-4941-8c6b-df72a6d920f2', imageId='7db55211-041a-40c8-85a6-a2ff1bd182ce'}), log id: 59bde70f 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGroupGUID=66984468-bbfa-4941-8c6b-df72a6d920f2 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=7db55211-041a-40c8-85a6-a2ff1bd182ce 2019-10-08 09:57:51,627Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ description={"Updated":false,"Last Updated":null,"Storage Domains":[{"uuid":"935de3ac-a735-4fc8-9161-26bfd751ffc7"}],"Disk Description":"OVF_STORE"} 2019-10-08 09:57:51,664Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 59bde70f 2019-10-08 09:57:51,672Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Lock Acquired to object 'EngineLock:{exclusiveLocks='', sharedLocks='[bfc4c379-bd5f-45be-98a4-f0469dc3817c=VDS_EXECUTION]'}' 2019-10-08 09:57:51,683Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Running command: UploadStreamCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, UploadStreamVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, UploadStreamVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c'}), log id: 1cb812b6 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeVdsBrokerCommand, parameters: 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGUID=66984468-bbfa-4941-8c6b-df72a6d920f2 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=7db55211-041a-40c8-85a6-a2ff1bd182ce 2019-10-08 09:57:51,684Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ size=30720 2019-10-08 09:57:51,933Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.UploadStreamVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, UploadStreamVDSCommand, return: , log id: 1cb812b6 2019-10-08 09:57:51,943Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] CommandAsyncTask::Adding CommandMultiAsyncTasks object for command 'd0c07988-0e43-4e6d-b86c-e736b28d4537' 2019-10-08 09:57:51,943Z INFO [org.ovirt.engine.core.bll.CommandMultiAsyncTasks] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] CommandMultiAsyncTasks::attachTask: Attaching task 'ba88edd0-8188-472e-8f59-aa3f90e63181' to command 'd0c07988-0e43-4e6d-b86c-e736b28d4537'. 2019-10-08 09:57:51,952Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Adding task 'ba88edd0-8188-472e-8f59-aa3f90e63181' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters'), polling hasn't started yet.. 2019-10-08 09:57:51,958Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] BaseAsyncTask::startPollingTask: Starting to poll task 'ba88edd0-8188-472e-8f59-aa3f90e63181'. 2019-10-08 09:57:51,961Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] Lock freed to object 'EngineLock:{exclusiveLocks='', sharedLocks='[bfc4c379-bd5f-45be-98a4-f0469dc3817c=VDS_EXECUTION]'}' 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] START, SetVolumeDescriptionVDSCommand( SetVolumeDescriptionVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='66984468-bbfa-4941-8c6b-df72a6d920f2', imageId='7db55211-041a-40c8-85a6-a2ff1bd182ce'}), log id: 213b3075 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] -- executeIrsBrokerCommand: calling 'setVolumeDescription', parameters: 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ spUUID=2e3b3484-c339-11e9-8d02-00163e3ec101 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ sdUUID=935de3ac-a735-4fc8-9161-26bfd751ffc7 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ imageGroupGUID=66984468-bbfa-4941-8c6b-df72a6d920f2 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ volUUID=7db55211-041a-40c8-85a6-a2ff1bd182ce 2019-10-08 09:57:51,962Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] ++ description={"Updated":true,"Size":30720,"Last Updated":"Tue Oct 08 09:57:50 UTC 2019","Storage Domains":[{"uuid":"935de3ac-a735-4fc8-9161-26bfd751ffc7"}],"Disk Description":"OVF_STORE"} 2019-10-08 09:57:52,028Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SetVolumeDescriptionVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-1) [5830ba58] FINISH, SetVolumeDescriptionVDSCommand, return: , log id: 213b3075 2019-10-08 09:57:52,224Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] Polling and updating Async Tasks: 7 tasks, 3 tasks to poll now 2019-10-08 09:57:52,228Z ERROR [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMGetAllTasksStatusesVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] Failed in 'HSMGetAllTasksStatusesVDS' method 2019-10-08 09:57:52,235Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] EVENT_ID: VDS_BROKER_COMMAND_FAILURE(10,802), VDSM ovirt-staging-hv-01.avinity.tv command HSMGetAllTasksStatusesVDS failed: Invalid parameter: 'size=16777216' 2019-10-08 09:57:52,235Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] SPMAsyncTask::PollTask: Polling task '99abc310-b44b-435c-8d43-922b782a69dc' (Parent Command 'CreateSnapshotFromTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'cleanSuccess'. 2019-10-08 09:57:52,240Z ERROR [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] BaseAsyncTask::logEndTaskFailure: Task '99abc310-b44b-435c-8d43-922b782a69dc' (Parent Command 'CreateSnapshotFromTemplate', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended with failure: -- Result: 'cleanSuccess' -- Message: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Invalid parameter: 'size=16777216', code = 100', -- Exception: 'VDSGenericException: VDSErrorException: Failed to HSMGetAllTasksStatusesVDS, error = Invalid parameter: 'size=16777216', code = 100' 2019-10-08 09:57:52,241Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endActionIfNecessary: All tasks of command 'f1665b66-2032-4a60-944e-c04beace088e' has ended -> executing 'endAction' 2019-10-08 09:57:52,241Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: 'f1665b66-2032-4a60-944e-c04beace088e'): calling endAction '. 2019-10-08 09:57:52,241Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] SPMAsyncTask::PollTask: Polling task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2019-10-08 09:57:52,241Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] BaseAsyncTask::onTaskEndSuccess: Task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-10-08 09:57:52,241Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'CreateSnapshotFromTemplate', 2019-10-08 09:57:52,242Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endActionIfNecessary: All tasks of command 'c11f4d3c-f85b-43c5-986f-a517aed66d7e' has ended -> executing 'endAction' 2019-10-08 09:57:52,243Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: 'c11f4d3c-f85b-43c5-986f-a517aed66d7e'): calling endAction '. 2019-10-08 09:57:52,243Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] SPMAsyncTask::PollTask: Polling task 'ba88edd0-8188-472e-8f59-aa3f90e63181' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') returned status 'finished', result 'success'. 2019-10-08 09:57:52,243Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] BaseAsyncTask::onTaskEndSuccess: Task 'ba88edd0-8188-472e-8f59-aa3f90e63181' (Parent Command 'UploadStream', Parameters Type 'org.ovirt.engine.core.common.asynctasks.AsyncTaskParameters') ended successfully. 2019-10-08 09:57:52,243Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'UploadStream', 2019-10-08 09:57:52,244Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endActionIfNecessary: All tasks of command 'd0c07988-0e43-4e6d-b86c-e736b28d4537' has ended -> executing 'endAction' 2019-10-08 09:57:52,244Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engineScheduled-Thread-91) [] CommandAsyncTask::endAction: Ending action for '1' tasks (command ID: 'd0c07988-0e43-4e6d-b86c-e736b28d4537'): calling endAction '. 2019-10-08 09:57:52,245Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [] CommandAsyncTask::endCommandAction [within thread] context: Attempting to endAction 'UploadStream', 2019-10-08 09:57:52,245Z INFO [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Command [id=f1665b66-2032-4a60-944e-c04beace088e]: Updating status to 'FAILED', The command end method logic will be executed by one of its parent commands. 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateSnapshotFromTemplate' completed, handling the result. 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'CreateSnapshotFromTemplate' succeeded, clearing tasks. 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] SPMAsyncTask::ClearAsyncTask: Attempting to clear task '99abc310-b44b-435c-8d43-922b782a69dc' 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' successfully. 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.UploadStreamCommand' successfully. 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', taskId='99abc310-b44b-435c-8d43-922b782a69dc'}), log id: 66a4e67 2019-10-08 09:57:52,247Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] START, HSMClearTaskVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, HSMTaskGuidBaseVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c', taskId='99abc310-b44b-435c-8d43-922b782a69dc'}), log id: 7ec6d8b8 2019-10-08 09:57:52,249Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'UploadStream' completed, handling the result. 2019-10-08 09:57:52,249Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'UploadStream' succeeded, clearing tasks. 2019-10-08 09:57:52,249Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' 2019-10-08 09:57:52,249Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', taskId='e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d'}), log id: 356927d 2019-10-08 09:57:52,250Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'UploadStream' completed, handling the result. 2019-10-08 09:57:52,250Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: endAction for action type 'UploadStream' succeeded, clearing tasks. 2019-10-08 09:57:52,250Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] SPMAsyncTask::ClearAsyncTask: Attempting to clear task 'ba88edd0-8188-472e-8f59-aa3f90e63181' 2019-10-08 09:57:52,251Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] START, SPMClearTaskVDSCommand( SPMTaskGuidBaseVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', taskId='ba88edd0-8188-472e-8f59-aa3f90e63181'}), log id: 75844e34 2019-10-08 09:57:52,259Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] FINISH, HSMClearTaskVDSCommand, return: , log id: 7ec6d8b8 2019-10-08 09:57:52,259Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] FINISH, SPMClearTaskVDSCommand, return: , log id: 66a4e67 2019-10-08 09:57:52,260Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] START, HSMClearTaskVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, HSMTaskGuidBaseVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c', taskId='e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d'}), log id: 3b239769 2019-10-08 09:57:52,263Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] BaseAsyncTask::removeTaskFromDB: Removed task '99abc310-b44b-435c-8d43-922b782a69dc' from DataBase 2019-10-08 09:57:52,263Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43693) [067ec489-ae6c-4871-8ab6-8296016ca1ce] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'f1665b66-2032-4a60-944e-c04beace088e' 2019-10-08 09:57:52,297Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] FINISH, HSMClearTaskVDSCommand, return: , log id: 3b239769 2019-10-08 09:57:52,297Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] FINISH, SPMClearTaskVDSCommand, return: , log id: 356927d 2019-10-08 09:57:52,298Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] START, HSMClearTaskVDSCommand(HostName = ovirt-staging-hv-01.avinity.tv, HSMTaskGuidBaseVDSCommandParameters:{hostId='bfc4c379-bd5f-45be-98a4-f0469dc3817c', taskId='ba88edd0-8188-472e-8f59-aa3f90e63181'}), log id: 32d44e1 2019-10-08 09:57:52,301Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] BaseAsyncTask::removeTaskFromDB: Removed task 'e5d00ff5-eaa2-48e9-ab10-d6d06a01e89d' from DataBase 2019-10-08 09:57:52,301Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43694) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'c11f4d3c-f85b-43c5-986f-a517aed66d7e' 2019-10-08 09:57:52,314Z INFO [org.ovirt.engine.core.vdsbroker.vdsbroker.HSMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] FINISH, HSMClearTaskVDSCommand, return: , log id: 32d44e1 2019-10-08 09:57:52,314Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.SPMClearTaskVDSCommand] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] FINISH, SPMClearTaskVDSCommand, return: , log id: 75844e34 2019-10-08 09:57:52,317Z INFO [org.ovirt.engine.core.bll.tasks.SPMAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] BaseAsyncTask::removeTaskFromDB: Removed task 'ba88edd0-8188-472e-8f59-aa3f90e63181' from DataBase 2019-10-08 09:57:52,317Z INFO [org.ovirt.engine.core.bll.tasks.CommandAsyncTask] (EE-ManagedThreadFactory-engine-Thread-43695) [5830ba58] CommandAsyncTask::HandleEndActionResult [within thread]: Removing CommandMultiAsyncTasks object for entity 'd0c07988-0e43-4e6d-b86c-e736b28d4537' 2019-10-08 09:57:54,039Z INFO [org.ovirt.engine.core.bll.SerialChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-44) [5830ba58] Command 'ProcessOvfUpdateForStorageDomain' id: 'bdcdae3a-00f6-4176-9d27-0bb0f04912a3' child commands '[b4050edd-62a7-4b83-a4ab-a21c83bf3ade, 5fd47910-95b2-4387-8e26-ea2c80b078a0, c11f4d3c-f85b-43c5-986f-a517aed66d7e, d0c07988-0e43-4e6d-b86c-e736b28d4537]' executions were completed, status 'SUCCEEDED' 2019-10-08 09:57:55,046Z INFO [org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-40) [5830ba58] Ending command 'org.ovirt.engine.core.bll.storage.ovfstore.ProcessOvfUpdateForStorageDomainCommand' successfully. 2019-10-08 09:57:58,077Z INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-76) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Command 'AddVm' id: '1b65f576-365d-4e8b-a920-621516cbdaed' child commands '[f1665b66-2032-4a60-944e-c04beace088e]' executions were completed, status 'FAILED' 2019-10-08 09:57:59,094Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:57:59,099Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:57:59,104Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:57:59,126Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 69861a78 2019-10-08 09:57:59,139Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:57:59,139Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:57:59,139Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] FINISH, DeleteImageGroupVDSCommand, return: , log id: 69861a78 2019-10-08 09:57:59,139Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:57:59,149Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [5794ed92] Removed task 'b4c4c723-4ba1-4d5a-86e6-24658a293a04' from DataBase 2019-10-08 09:57:59,220Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] transaction rolled back 2019-10-08 09:57:59,220Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:57:59,233Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:57:59,233Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-27) [] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries 2019-10-08 09:58:01,252Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:58:01,257Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:58:01,270Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:58:01,308Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 35fec55 2019-10-08 09:58:01,324Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:01,324Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:01,324Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] FINISH, DeleteImageGroupVDSCommand, return: , log id: 35fec55 2019-10-08 09:58:01,324Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:58:01,335Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [19251787] Removed task '6ed0a844-5363-4afd-bffc-a80b6ace69ea' from DataBase 2019-10-08 09:58:01,395Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] transaction rolled back 2019-10-08 09:58:01,395Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] Trying to release a shared lock for key: '9ac6f4ad-58d0-4a7e-b424-91f2d76abcacTEMPLATE' , but lock does not exist 2019-10-08 09:58:01,395Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] Trying to release a shared lock for key: 'a7e42574-be60-4c94-94a5-cc4b30fdb16fDISK' , but lock does not exist 2019-10-08 09:58:01,395Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] Trying to release exclusive lock which does not exist, lock key: 'centos-testvm-024VM_NAME' 2019-10-08 09:58:01,395Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:58:01,407Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:58:01,407Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-85) [] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries 2019-10-08 09:58:05,427Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:58:05,430Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:58:05,437Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:58:05,452Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 6ac726b6 2019-10-08 09:58:05,466Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:05,466Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:05,466Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] FINISH, DeleteImageGroupVDSCommand, return: , log id: 6ac726b6 2019-10-08 09:58:05,467Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:58:05,477Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [cf91364] Removed task '31bd7a11-203d-4459-ac54-3c398c85771d' from DataBase 2019-10-08 09:58:05,490Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] transaction rolled back 2019-10-08 09:58:05,490Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Trying to release a shared lock for key: '9ac6f4ad-58d0-4a7e-b424-91f2d76abcacTEMPLATE' , but lock does not exist 2019-10-08 09:58:05,490Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Trying to release a shared lock for key: 'a7e42574-be60-4c94-94a5-cc4b30fdb16fDISK' , but lock does not exist 2019-10-08 09:58:05,490Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Trying to release exclusive lock which does not exist, lock key: 'centos-testvm-024VM_NAME' 2019-10-08 09:58:05,490Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:58:05,503Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:58:05,503Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-17) [] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries 2019-10-08 09:58:13,523Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:58:13,527Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:58:13,533Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:58:13,548Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 33164673 2019-10-08 09:58:13,560Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:13,560Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:13,560Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] FINISH, DeleteImageGroupVDSCommand, return: , log id: 33164673 2019-10-08 09:58:13,560Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:58:13,570Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [56103b5a] Removed task 'ce6882de-2d89-4298-bdcf-86d621a54084' from DataBase 2019-10-08 09:58:13,583Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] transaction rolled back 2019-10-08 09:58:13,583Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Trying to release a shared lock for key: '9ac6f4ad-58d0-4a7e-b424-91f2d76abcacTEMPLATE' , but lock does not exist 2019-10-08 09:58:13,583Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Trying to release a shared lock for key: 'a7e42574-be60-4c94-94a5-cc4b30fdb16fDISK' , but lock does not exist 2019-10-08 09:58:13,583Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Trying to release exclusive lock which does not exist, lock key: 'centos-testvm-024VM_NAME' 2019-10-08 09:58:13,583Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:58:13,595Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:58:13,595Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-35) [] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries 2019-10-08 09:58:23,616Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:58:23,620Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:58:23,627Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:58:23,642Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 7346a5bd 2019-10-08 09:58:23,663Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:23,663Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:23,663Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] FINISH, DeleteImageGroupVDSCommand, return: , log id: 7346a5bd 2019-10-08 09:58:23,663Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:58:23,682Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [48f2f9bd] Removed task '8483357d-6d4f-418e-9751-70fcb69685cb' from DataBase 2019-10-08 09:58:23,695Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] transaction rolled back 2019-10-08 09:58:23,695Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Trying to release a shared lock for key: '9ac6f4ad-58d0-4a7e-b424-91f2d76abcacTEMPLATE' , but lock does not exist 2019-10-08 09:58:23,695Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Trying to release a shared lock for key: 'a7e42574-be60-4c94-94a5-cc4b30fdb16fDISK' , but lock does not exist 2019-10-08 09:58:23,695Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Trying to release exclusive lock which does not exist, lock key: 'centos-testvm-024VM_NAME' 2019-10-08 09:58:23,695Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:58:23,706Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:58:23,706Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-71) [] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries 2019-10-08 09:58:33,725Z ERROR [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.AddVmCommand' with failure. 2019-10-08 09:58:33,729Z ERROR [org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Ending command 'org.ovirt.engine.core.bll.snapshots.CreateSnapshotFromTemplateCommand' with failure. 2019-10-08 09:58:33,736Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] Running command: RemoveImageCommand internal: true. Entities affected : ID: 935de3ac-a735-4fc8-9161-26bfd751ffc7 Type: Storage 2019-10-08 09:58:33,752Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] START, DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'}), log id: 6d120001 2019-10-08 09:58:33,767Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] EVENT_ID: IRS_BROKER_COMMAND_FAILURE(10,803), VDSM command DeleteImageGroupVDS failed: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:33,767Z ERROR [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] Command 'DeleteImageGroupVDSCommand( DeleteImageGroupVDSCommandParameters:{storagePoolId='2e3b3484-c339-11e9-8d02-00163e3ec101', ignoreFailoverLimit='false', storageDomainId='935de3ac-a735-4fc8-9161-26bfd751ffc7', imageGroupId='d680a46a-7188-45f6-b6a7-3333f830c0b4', postZeros='false', discard='false', forceDelete='false'})' execution failed: IRSGenericException: IRSErrorException: Image does not exist in domain: u'image=d680a46a-7188-45f6-b6a7-3333f830c0b4, domain=935de3ac-a735-4fc8-9161-26bfd751ffc7' 2019-10-08 09:58:33,767Z INFO [org.ovirt.engine.core.vdsbroker.irsbroker.DeleteImageGroupVDSCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] FINISH, DeleteImageGroupVDSCommand, return: , log id: 6d120001 2019-10-08 09:58:33,767Z INFO [org.ovirt.engine.core.bll.storage.disk.image.RemoveImageCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] Disk 'd680a46a-7188-45f6-b6a7-3333f830c0b4' doesn't exist on storage domain '935de3ac-a735-4fc8-9161-26bfd751ffc7', rolling forward 2019-10-08 09:58:33,778Z INFO [org.ovirt.engine.core.bll.tasks.AsyncTaskManager] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [3eea0406] Removed task 'bfb5e88a-0e62-4e03-b02b-a5a0b36f84db' from DataBase 2019-10-08 09:58:33,791Z INFO [org.ovirt.engine.core.utils.transaction.TransactionSupport] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] transaction rolled back 2019-10-08 09:58:33,791Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Trying to release a shared lock for key: '9ac6f4ad-58d0-4a7e-b424-91f2d76abcacTEMPLATE' , but lock does not exist 2019-10-08 09:58:33,791Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Trying to release a shared lock for key: 'a7e42574-be60-4c94-94a5-cc4b30fdb16fDISK' , but lock does not exist 2019-10-08 09:58:33,791Z WARN [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Trying to release exclusive lock which does not exist, lock key: 'centos-testvm-024VM_NAME' 2019-10-08 09:58:33,791Z INFO [org.ovirt.engine.core.bll.AddVmCommand] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] Lock freed to object 'EngineLock:{exclusiveLocks='[centos-testvm-024=VM_NAME]', sharedLocks='[9ac6f4ad-58d0-4a7e-b424-91f2d76abcac=TEMPLATE, a7e42574-be60-4c94-94a5-cc4b30fdb16f=DISK]'}' 2019-10-08 09:58:33,804Z ERROR [org.ovirt.engine.core.dal.dbbroker.auditloghandling.AuditLogDirector] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [] EVENT_ID: USER_ADD_VM_FINISHED_FAILURE(60), Failed to complete VM centos-testvm-024 creation. 2019-10-08 09:58:33,824Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Failed invoking callback end method 'onFailed' for command '1b65f576-365d-4e8b-a920-621516cbdaed' with exception 'null', the callback is marked for end method retries but max number of retries have been attempted. The command will be marked as Failed. 2019-10-08 09:58:33,824Z INFO [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Exception in invoking callback of command AddVm (1b65f576-365d-4e8b-a920-621516cbdaed): NullPointerException: 2019-10-08 09:58:33,824Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Error invoking callback method 'onFailed' for 'FAILED' command '1b65f576-365d-4e8b-a920-621516cbdaed' 2019-10-08 09:58:33,824Z ERROR [org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller] (EE-ManagedThreadFactory-engineScheduled-Thread-63) [067ec489-ae6c-4871-8ab6-8296016ca1ce] Exception: javax.ejb.EJBTransactionRolledbackException at org.jboss.as.ejb3.tx.CMTTxInterceptor.invokeInCallerTx(CMTTxInterceptor.java:203) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.supports(CMTTxInterceptor.java:420) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.as.ejb3.tx.CMTTxInterceptor.processInvocation(CMTTxInterceptor.java:148) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext$Invocation.proceed(InterceptorContext.java:509) at org.jboss.weld.module.ejb.AbstractEJBRequestScopeActivationInterceptor.aroundInvoke(AbstractEJBRequestScopeActivationInterceptor.java:81) [weld-ejb-3.1.1.Final.jar:3.1.1.Final] at org.jboss.as.weld.ejb.EjbRequestScopeActivationInterceptor.processInvocation(EjbRequestScopeActivationInterceptor.java:89) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.CurrentInvocationContextInterceptor.processInvocation(CurrentInvocationContextInterceptor.java:41) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.invocationmetrics.WaitTimeInterceptor.processInvocation(WaitTimeInterceptor.java:47) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.security.SecurityContextInterceptor.processInvocation(SecurityContextInterceptor.java:100) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.deployment.processors.StartupAwaitInterceptor.processInvocation(StartupAwaitInterceptor.java:22) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.ShutDownInterceptorFactory$1.processInvocation(ShutDownInterceptorFactory.java:64) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ejb3.component.interceptors.LoggingInterceptor.processInvocation(LoggingInterceptor.java:67) [wildfly-ejb3-17.0.1.Final.jar:17.0.1.Final] at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.as.ee.component.NamespaceContextInterceptor.processInvocation(NamespaceContextInterceptor.java:50) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ContextClassLoaderInterceptor.processInvocation(ContextClassLoaderInterceptor.java:60) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.InterceptorContext.run(InterceptorContext.java:438) at org.wildfly.security.manager.WildFlySecurityManager.doChecked(WildFlySecurityManager.java:618) at org.jboss.invocation.AccessCheckingInterceptor.processInvocation(AccessCheckingInterceptor.java:57) at org.jboss.invocation.InterceptorContext.proceed(InterceptorContext.java:422) at org.jboss.invocation.ChainedInterceptor.processInvocation(ChainedInterceptor.java:53) at org.jboss.as.ee.component.ViewService$View.invoke(ViewService.java:198) at org.jboss.as.ee.component.ViewDescription$1.processInvocation(ViewDescription.java:185) at org.jboss.as.ee.component.ProxyInvocationHandler.invoke(ProxyInvocationHandler.java:81) at org.ovirt.engine.core.bll.interfaces.BackendInternal$$$view3.endAction(Unknown Source) [bll.jar:] at sun.reflect.GeneratedMethodAccessor1032.invoke(Unknown Source) [:1.8.0_222] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) [rt.jar:1.8.0_222] at java.lang.reflect.Method.invoke(Method.java:498) [rt.jar:1.8.0_222] at org.jboss.weld.util.reflection.Reflections.invokeAndUnwrap(Reflections.java:410) [weld-core-impl-3.1.1.Final.jar:3.1.1.Final] at org.jboss.weld.module.ejb.EnterpriseBeanProxyMethodHandler.invoke(EnterpriseBeanProxyMethodHandler.java:134) [weld-ejb-3.1.1.Final.jar:3.1.1.Final] at org.jboss.weld.bean.proxy.EnterpriseTargetBeanInstance.invoke(EnterpriseTargetBeanInstance.java:56) [weld-core-impl-3.1.1.Final.jar:3.1.1.Final] at org.jboss.weld.module.ejb.InjectionPointPropagatingEnterpriseTargetBeanInstance.invoke(InjectionPointPropagatingEnterpriseTargetBeanInstance.java:68) [weld-ejb-3.1.1.Final.jar:3.1.1.Final] at org.jboss.weld.bean.proxy.ProxyMethodHandler.invoke(ProxyMethodHandler.java:106) [weld-core-impl-3.1.1.Final.jar:3.1.1.Final] at org.ovirt.engine.core.bll.BackendCommandObjectsHandler$BackendInternal$BackendLocal$2049259618$Proxy$_$$_Weld$EnterpriseProxy$.endAction(Unknown Source) [bll.jar:] at org.ovirt.engine.core.bll.VmCommand.endActionOnDisks(VmCommand.java:266) [bll.jar:] at org.ovirt.engine.core.bll.AddVmCommand.endWithFailure(AddVmCommand.java:1452) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.internalEndWithFailure(CommandBase.java:736) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.endActionInTransactionScope(CommandBase.java:658) [bll.jar:] at org.ovirt.engine.core.bll.CommandBase.runInTransaction(CommandBase.java:2003) [bll.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInNewTransaction(TransactionSupport.java:202) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInRequired(TransactionSupport.java:137) [utils.jar:] at org.ovirt.engine.core.utils.transaction.TransactionSupport.executeInScope(TransactionSupport.java:105) [utils.jar:] at org.ovirt.engine.core.bll.CommandBase.endAction(CommandBase.java:520) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.endAction(ChildCommandsCallbackBase.java:134) [bll.jar:] at org.ovirt.engine.core.bll.ChildCommandsCallbackBase.onFailed(ChildCommandsCallbackBase.java:174) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.endCallback(CommandCallbacksPoller.java:67) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethodsImpl(CommandCallbacksPoller.java:166) [bll.jar:] at org.ovirt.engine.core.bll.tasks.CommandCallbacksPoller.invokeCallbackMethods(CommandCallbacksPoller.java:109) [bll.jar:] at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [rt.jar:1.8.0_222] at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308) [rt.jar:1.8.0_222] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.access$201(ManagedScheduledThreadPoolExecutor.java:383) [javax.enterprise.concurrent-1.0.jar:] at org.glassfish.enterprise.concurrent.internal.ManagedScheduledThreadPoolExecutor$ManagedScheduledFutureTask.run(ManagedScheduledThreadPoolExecutor.java:534) [javax.enterprise.concurrent-1.0.jar:] at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) [rt.jar:1.8.0_222] at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) [rt.jar:1.8.0_222] at java.lang.Thread.run(Thread.java:748) [rt.jar:1.8.0_222] at org.glassfish.enterprise.concurrent.ManagedThreadFactoryImpl$ManagedThread.run(ManagedThreadFactoryImpl.java:250) [javax.enterprise.concurrent-1.0.jar:] Caused by: java.lang.NullPointerException Ansible output: The full traceback is: Traceback (most recent call last): File "/var/folders/40/w2c8fp151854mddz_4n3czwm0000gn/T/ansible_ovirt_vm_payload_uy4lqway/__main__.py", line 2024, in main clone_permissions=module.params['clone_permissions'], File "/var/folders/40/w2c8fp151854mddz_4n3czwm0000gn/T/ansible_ovirt_vm_payload_uy4lqway/ansible_ovirt_vm_payload.zip/ansible/module_utils/ovirt.py", line 610, in create poll_interval=self._module.params['poll_interval'], File "/var/folders/40/w2c8fp151854mddz_4n3czwm0000gn/T/ansible_ovirt_vm_payload_uy4lqway/ansible_ovirt_vm_payload.zip/ansible/module_utils/ovirt.py", line 341, in wait raise Exception("Timeout exceed while waiting on result state of the entity.") Exception: Timeout exceed while waiting on result state of the entity. fatal: [centos-testvm-024.avinity.tv -> localhost]: FAILED! => { "changed": false, "invocation": { "module_args": { "affinity_group_mappings": [], "affinity_label_mappings": [], "allow_partial_import": null, "ballooning_enabled": null, "boot_devices": null, "boot_menu": null, "cd_iso": null, "clone": false, "clone_permissions": false, "cloud_init": { "authorized_ssh_keys": "ssh-rsa removed pubkey", "host_name": "centos-testvm-024.avinity.tv", "user_name": "centos" }, "cloud_init_nics": [], "cloud_init_persist": false, "cluster": "avshared1", "cluster_mappings": [], "comment": "oVirt Template Bakery Validation", "cpu_cores": null, "cpu_mode": null, "cpu_pinning": null, "cpu_shares": null, "cpu_sockets": null, "cpu_threads": null, "custom_compatibility_version": null, "custom_properties": null, "delete_protected": null, "description": null, "disk_format": "cow", "disks": [], "domain_mappings": [], "fetch_nested": false, "force": false, "graphical_console": null, "high_availability": null, "high_availability_priority": null, "host": null, "host_devices": null, "id": null, "initrd_path": null, "instance_type": "Medium", "io_threads": null, "kernel_params": null, "kernel_path": null, "kvm": null, "lease": null, "lun_mappings": [], "memory": null, "memory_guaranteed": null, "memory_max": null, "name": "centos-testvm-024", "nested_attributes": [], "nics": [ { "interface": "virtio", "name": "nic1", "profile_name": "tenant1" } ], "numa_nodes": [], "numa_tune_mode": null, "operating_system": null, "placement_policy": null, "poll_interval": 3, "quota_id": null, "reassign_bad_macs": null, "rng_device": null, "role_mappings": [], "serial_console": null, "serial_policy": null, "serial_policy_value": null, "smartcard_enabled": null, "soundcard_enabled": null, "sso": null, "state": "running", "stateless": null, "storage_domain": null, "sysprep": null, "template": "av-centos-75-baseimage_nfs9000", "template_version": null, "ticket": null, "timeout": 180, "timezone": null, "type": null, "usb_support": null, "use_latest_template_version": null, "vmware": null, "vnic_profile_mappings": [], "wait": true, "watchdog": null, "xen": null } }, "msg": "Timeout exceed while waiting on result state of the entity." }

3 4

ovirt UI update and webproxy server
by kim.kargaard＠noroff.no 09 Oct '19

09 Oct '19

Hi, I want our users to be able to use novnc via the VM portal, so I installed a separate version of the UI, following the information I could find on that process. It is now possible to get the novnc console option in the VM portal, but it seems something with this broke the internal CA for the webproxy, as I get the wonderful: Disconnected from ConsoleCannot connect to websocket proxy server. Please check your websocket proxy certificate or ask your administrator for help. For further information please refer to the console manual. Press the 'Connect' button to reconnect the console. When using the Admin Portal, I of course get a bit more detail and a suggestion to check if websocket proxy server is running, if the firewall is blocking it or if the CA is installed. But, it was working perfectly before the UI update. When I check the status of the websocket proxy on the engine, I see the following: Active: active (running) since Fri 2019-09-27 10:58:28 CEST; 1 weeks 5 days ago Main PID: 58139 (ovirt-websocket) CGroup: /system.slice/ovirt-websocket-proxy.service └─58139 /usr/bin/python /usr/share/ovirt-engine/services/ovirt-websocket-proxy/ovirt-websocket-proxy.py --systemd=notify start Oct 09 14:14:00 enginedomain ovirt-websocket[3569]: 2019-10-09 14:14:00,746+0200 ovirt-websocket-proxy: INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:14:00 enginedomain ovirt-websocket-proxy.py[58139]: ovirt-websocket-proxy[3569] INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:17:44 enginedomain ovirt-websocket[3855]: 2019-10-09 14:17:44,797+0200 ovirt-websocket-proxy: INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:17:44 enginedomain ovirt-websocket-proxy.py[58139]: ovirt-websocket-proxy[3855] INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:21:43 enginedomain ovirt-websocket[4146]: 2019-10-09 14:21:43,126+0200 ovirt-websocket-proxy: INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:21:43 enginedomain ovirt-websocket-proxy.py[58139]: ovirt-websocket-proxy[4146] INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:22:25 enginedomain ovirt-websocket[4180]: 2019-10-09 14:22:25,004+0200 ovirt-websocket-proxy: INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:22:25 enginedomain ovirt-websocket-proxy.py[58139]: ovirt-websocket-proxy[4180] INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:22:40 enginedomain ovirt-websocket[4206]: 2019-10-09 14:22:40,506+0200 ovirt-websocket-proxy: INFO msg:887 handler exception: [Errno 13] Permission denied Oct 09 14:22:40 enginedomain ovirt-websocket-proxy.py[58139]: ovirt-websocket-proxy[4206] INFO msg:887 handler exception: [Errno 13] Permission denied The CA files have not changed permissions, so I am at a bit of a loss. Any suggestions? I checked the ovirt logs, but I struggled to find a log for the websocket proxy, so I can't find a more detailed error log. I installed ovirt-web-ui-1.6.1-0.20190923.git6ca35b5.el7.noarch.rpm to get the novnc console on the VM portal.

1 0

Cloud Init static route
by Tommaso - Shellrent 09 Oct '19

09 Oct '19

Hi, on our system we need to setup a static route to all the vm's. How can we do it via cloud-init? it seems to be not possible via the ovirt interface regards, -- Shellrent Logo *Tommaso De Marchi* COO Chief Operating Officer - Shellrent S.r.l. Tel. 0444321155 <tel:+390444321155> | Fax 04441492177 Via dell'Edilizia, 19 - 36100 Vicenza

2 1

Re: VDI
by Alex McWhirter 09 Oct '19

09 Oct '19

To achieve that all you need to do is create a template of the desktop base vm, make sure the vm type is set to desktop. Afterwards just create new vms from that template. As long as the VM type is set to desktop each new VM will use a qcow overlay on top of the base image. Taking this a step further you can then create VM pools from said template, allowing users to dynamically be assigned a new VM on login. Granted pools are usually stateless, so you need to have network file storage. We use pools for windows 10 VDI instances, where we use sysprep to autojoin the new pool vm to the domain where redirected folders are already setup. For VDI only use spice protocol. By default we found spice to be semi lackluster, so we do apply custom settings and we have recompiled spice on both servers and clients with h264 support. This is not 100% necessary, but makes things like youtube much more usable. We have also backported some audio patches to KVM. CentOS 8 should resolve a lot of these customizations that we've had to do. As far as updating, pretty much. We create a VM from the template, update it, then push it back as a new version of the template. The pools are set to always use the latest template version. Users have to log out, then back in to the VDI system in order to get the new image as logging out will destroy the users current instance and create a new one on log in. On 2019-09-23 15:16, Fabio Marzocca wrote: > Hi Alex, thanks for answering. > > I am approaching and studying oVirt in order to propose the solution to a customer as a replacement for a commercial solution they have now. > They only need Desktop virtualization. > Sorry for the silly question, but I can't find a way to deploy a VM (template) to users as a "linked-clone", meaning that the users' image still refers to the original image but modification are written (and afterwards read) from a new location. This technique is called Copy-on-write. > Can this be achieved with oVirt? > > Then, what is the Best Practice to update WIndows OS for the all the users? Currently they simply "check-out" the Gold Image, update it and check-in, while all users are running... > > Fabio > > On Mon, Sep 23, 2019 at 8:04 PM Alex McWhirter <Alex(a)triadic.us> wrote: > > yes, we do. All spice, with some customizations done at source level for spice / kvm packages. > > On 2019-09-23 13:44, Fabio Marzocca wrote: > Is there anyone who uses oVirt as a full VDI environment? I would have a bunch of questions... > > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/D44YB5VOKNBNC…

5 4

is it possible to add host on which ovirt is installed ?
by levisik＠gmail.com 09 Oct '19

09 Oct '19

Hi. I am trying to do some lab. I have one Unix server, on which I installed oVirt package. I have already access to console. I would like to create some virtual servers, but to do so, I need to have datacenter up. I checked in docummentation, that to have it running, I need to have: - cluster - which I already made - host - here I have problem, because I am trying to add the same host on which ovirt is running, but I get error without any notification - storage domain - which I guess is created automatically during ovirt installation. Any ideas :) ?

3 4

Re: Delete snapshots task hung
by Strahil 09 Oct '19

09 Oct '19

Try to migrate a VM from one host to another. I had a similar issue (1000 warnings in UI) that have stopped immediately I have migrated that VM. Best Regards, Strahil NikolovOn Oct 8, 2019 09:59, Leo David <leoalex(a)gmail.com> wrote: > > Hi Everyone, > I'm waiting since 3 days for 5 x delete snapshot tasks to finish, and for some reason it seems to be stucked.For other vms snapshot removal took at most 20 mins, with havin the disks pretty much same size, and snapshots numbers. > Any thoughts on how should I get this fixed ? > Below, some lines from the engine.log, and it seems to show some complains regarding locks ( Failed to acquire lock and wait lock) , although I am not sure if thats the root cause: > Thank you very much ! > > Leo > > 2019-10-08 09:52:48,692+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8') waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5' type:'DestroyImage' to complete > 2019-10-08 09:52:48,702+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '580fa033-35fd-44f0-9979-e60e9bbf8a29') waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052' type:'DestroyImage' to complete > 2019-10-08 09:52:49,713+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshotSingleDiskLive' (id: 'de747f91-ec59-4e70-9345-77e16234bfe0') waiting on child command id: '10812160-cf4c-4239-bb92-1d5a847687ee' type:'DestroyImage' to complete > 2019-10-08 09:52:50,725+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshotSingleDiskLive' (id: '4919b287-e980-4d34-a219-c08a169cd8f7') waiting on child command id: '5eceb6a8-f08e-42aa-8258-c907f5927e6c' type:'DestroyImage' to complete > 2019-10-08 09:52:51,563+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' > 2019-10-08 09:52:51,583+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' > 2019-10-08 09:52:51,604+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' > 2019-10-08 09:52:51,606+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' > 2019-10-08 09:52:51,735+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshot' (id: 'c9ab1344-ae27-4934-9358-d6a7b10a4f0a') waiting on child command id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8' type:'RemoveSnapshotSingleDiskLive' to complete > 2019-10-08 09:52:52,706+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@21830b5f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@676adc3e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@385a3510, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@af24d00, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@331266f2], log id: 1a3515fe > 2019-10-08 09:52:52,708+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName =

2 2

Re: Trying to add a LUN to a host and use that, instead of Gluster, as the datastore for my VM's.
by Strahil 09 Oct '19

09 Oct '19

Hi Tom, Have you checked https://access.redhat.com/documentation/en-us/red_hat_virtualization/4.3/ht… ? Best Regards, Strahil NikolovOn Oct 6, 2019 06:26, TomK <tomkcpr(a)mdevsys.com> wrote: > > Hey All, > > I've added a 4TB LUN to my physical storage and now I want to use that, > instead of GlusterFS as my datastore for VM's on that physical. How do > I do this? > > I've tried a number of options, including adding LUN's but appears I can > only add them to VM's directly but not use them as my datastores instead > of GlusterFS. > > -- > Thx, > TK. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZES6FCONNYBXY…

2 3

Inconsisten datacenter [urgent]
by jplorier＠gmail.com 08 Oct '19

08 Oct '19

I have a gluster engine domain on 4.2 replica 3 1 arbiter and the arbiter got down. I've been trying to replace the arbiter with no success. In the meantime, the two remaining hosts are non- responsive, both with one vm (one actually with the engine). The one without the engine can't be set to maintenance because shows one vm and in non-responsive state, but in the vm list, it shows none. I manually tried to shut the vm down with virsh but no domain is listed. I can't power off the host as it's part of the gluster. Is there a way I can manually remove the ghost vm so I can set the host to maintenance and try to fix it? Regards,

3 2

update node 4.3.5 to 4.3.6
by Vincent Royer 08 Oct '19

08 Oct '19

Ovirt node on 4.3.5. Was installed from 4.3.5 iso and has never been updated GUI said update available for hosts. Here's what I did: - Global HA maintenance - yum update and engine-upgrade-check engine-setup of hosted engine - engine now at Version 4.3.6.7-1.el7 - put node 1/3 into maintenance (vms migrate off) - update host in gui until it says its done - reboot node The node reboots but is still at 4.3.5 So I try manually: yum update > Loaded plugins: enabled_repos_upload, fastestmirror, imgbased-persist, > package_upload, product-id, search-disabled-repos, subscription-manager, > vdsmupgrade, versionlock > This system is not registered with an entitlement server. You can use > subscription-manager to register. > Loading mirror speeds from cached hostfile > * ovirt-4.3: mirrors.rit.edu > * ovirt-4.3-epel: d2lzkl7pfhq30w.cloudfront.net > Excluding 70 updates due to versionlock (use "yum versionlock status" to > show them) > No packages marked for update > Uploading Enabled Repositories Report > Cannot upload enabled repos report, is this client registered? > [root@letty ~]# in the etc/yum.repos.d folder I have: CentOS-Base.repo CentOS-CR.repo CentOS-Debuginfo.repo CentOS-fasttrack.repo CentOS-Media.repo CentOS-Sources.repo CentOS-Vault.repo ovirt-4.3-dependencies.repo ovirt-4.3.repo So how do I update to 4.3.6? isn't it part of ovirt-4.3.repo?

1 0

Delete snapshots task hung
by Leo David 08 Oct '19

08 Oct '19

Hi Everyone, I'm waiting since 3 days for 5 x delete snapshot tasks to finish, and for some reason it seems to be stucked.For other vms snapshot removal took at most 20 mins, with havin the disks pretty much same size, and snapshots numbers. Any thoughts on how should I get this fixed ? Below, some lines from the engine.log, and it seems to show some complains regarding locks ( Failed to acquire lock and wait lock) , although I am not sure if thats the root cause: Thank you very much ! Leo 2019-10-08 09:52:48,692+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8') waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5' type:'DestroyImage' to complete 2019-10-08 09:52:48,702+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-47) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '580fa033-35fd-44f0-9979-e60e9bbf8a29') waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052' type:'DestroyImage' to complete 2019-10-08 09:52:49,713+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-50) [539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshotSingleDiskLive' (id: 'de747f91-ec59-4e70-9345-77e16234bfe0') waiting on child command id: '10812160-cf4c-4239-bb92-1d5a847687ee' type:'DestroyImage' to complete 2019-10-08 09:52:50,725+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-100) [baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshotSingleDiskLive' (id: '4919b287-e980-4d34-a219-c08a169cd8f7') waiting on child command id: '5eceb6a8-f08e-42aa-8258-c907f5927e6c' type:'DestroyImage' to complete 2019-10-08 09:52:51,563+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,583+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,604+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,606+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:51,735+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-94) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshot' (id: 'c9ab1344-ae27-4934-9358-d6a7b10a4f0a') waiting on child command id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:52,706+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@21830b5f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@676adc3e, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@385a3510, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@af24d00, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@331266f2], log id: 1a3515fe 2019-10-08 09:52:52,708+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 2a8c782 2019-10-08 09:52:52,745+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-22) [539ba19e-0cb5-42cf-9a23-7916ee2de4a9] Command 'RemoveSnapshot' (id: '28ac73f3-8b07-43d4-8b4b-1df90831ed50') waiting on child command id: 'de747f91-ec59-4e70-9345-77e16234bfe0' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:53,115+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 2a8c782 2019-10-08 09:52:53,116+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVolumeAdvancedDetailsVDSCommand(HostName = host-dell-1.domain.int, GlusterVolumeAdvancedDetailsVDSParameters:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b', volumeName='samsung_sata'}), log id: 6bb38072 2019-10-08 09:52:54,659+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,662+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,663+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,679+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:54,755+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command 'RemoveSnapshot' (id: 'ceeb99c3-68be-4e0f-8dd5-3ef6509d4013') waiting on child command id: '66094952-88e3-41d9-9191-7ef8873de511' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:54,762+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-21) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshotSingleDiskLive' (id: '5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c') waiting on child command id: 'b3b0a892-e652-45dd-a71f-b2f77a942491' type:'DestroyImage' to complete 2019-10-08 09:52:55,276+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVolumeAdvancedDetailsVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVolumeAdvancedDetailsVDSCommand, return: org.ovirt.engine.core.common.businessentities.gluster.GlusterVolumeAdvancedDetails@db5a3b94, log id: 6bb38072 2019-10-08 09:52:55,288+03 INFO [org.ovirt.engine.core.bll.gluster.GlusterThinDeviceService] (DefaultQuartzScheduler10) [8921c9c] Volume samsung_sata have non-thin bricks, skipping confirmed free size calculation 2019-10-08 09:52:55,295+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:55,308+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: cc40e52 2019-10-08 09:52:55,376+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@723faf2a, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@faa8525, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@17581efa, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3bc224dd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@340b8a70, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6fd3a279, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@382bceb7, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1265e1a3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@7a55a195, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@86714cc, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6e5f3d01, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@538aa190, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@484eb7d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@34754e55, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2355daef], log id: cc40e52 2019-10-08 09:52:55,378+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: 25557934 2019-10-08 09:52:55,436+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@629b1e0, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@18305096, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@153f73dd], log id: 25557934 2019-10-08 09:52:55,438+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-3.domain.int, VdsIdVDSCommandParametersBase:{hostId='a129fd8c-33b4-4425-92f8-1fb3bd22541e'}), log id: 1bad2497 2019-10-08 09:52:55,740+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1bad2497 2019-10-08 09:52:55,742+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 504020b3 2019-10-08 09:52:55,890+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a6e6ecd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@24a15dc6, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@56da4c59, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74eab89, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3e6fb5ab, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@100e90cd, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14b2ae11, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@66d25ded, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@25537e71, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@41bf0aee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@40fce6ee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3435f629, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@a2ba628, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@337a25f9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@14967eee, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@f8946eb, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5844abd4], log id: 504020b3 2019-10-08 09:52:55,891+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 3af318c8 2019-10-08 09:52:56,620+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler2) [5f2fb2c6] START, GlusterTasksListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 4dd7bc07 2019-10-08 09:52:56,681+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@1d356076, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@584ec325, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@2529c9f3, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@73eb0c6, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@31be4687], log id: 3af318c8 2019-10-08 09:52:56,682+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-1.domain.int, VdsIdVDSCommandParametersBase:{hostId='109ffe86-6498-415d-8b00-1e1303a0085b'}), log id: 1f47fc7e 2019-10-08 09:52:56,774+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-32) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshotSingleDiskLive' (id: '1593ccd5-4e2a-4f8c-9c10-7944b95836b3') waiting on child command id: '8b017634-d965-4273-a4a4-6281a337ed94' type:'DestroyImage' to complete 2019-10-08 09:52:57,095+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 1f47fc7e 2019-10-08 09:52:57,096+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: 74cad834 2019-10-08 09:52:57,144+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@600eeeb, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@45491ad2, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@654daa09, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4764d5ec, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@46121c5d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4b0001c7, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4315e74c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@22bf8177, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@74999189, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@3f5ca8e4, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@531f17c4], log id: 74cad834 2019-10-08 09:52:57,145+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: df580e9 2019-10-08 09:52:57,191+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalPhysicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalPhysicalVolume@35585d07], log id: df580e9 2019-10-08 09:52:57,192+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterVDOVolumeListVDSCommand(HostName = host-dell-4.domain.int, VdsIdVDSCommandParametersBase:{hostId='5dde7a16-8fdb-4ae3-a061-75c451397d81'}), log id: 6f19c8c0 2019-10-08 09:52:57,198+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GlusterTasksListVDSCommand] (DefaultQuartzScheduler2) [5f2fb2c6] FINISH, GlusterTasksListVDSCommand, return: [], log id: 4dd7bc07 2019-10-08 09:52:57,432+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterVDOVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterVDOVolumeListVDSCommand, return: [], log id: 6f19c8c0 2019-10-08 09:52:57,433+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalLogicalVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 449353d0 2019-10-08 09:52:57,576+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,594+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,616+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,619+03 INFO [org.ovirt.engine.core.bll.lock.InMemoryLockManager] (DefaultQuartzScheduler7) [306a2296] Failed to acquire lock and wait lock 'EngineLock:{exclusiveLocks='[c6087b9e-2214-11e9-9288-00163e168814=GLUSTER]', sharedLocks=''}' 2019-10-08 09:52:57,785+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [4f2bedf7-b9b3-481e-936f-e180803ac1b8] Command 'RemoveSnapshotSingleDiskLive' (id: '66094952-88e3-41d9-9191-7ef8873de511') waiting on child command id: '122be070-d45f-4e6b-bb61-4943a096b88b' type:'DestroyImage' to complete 2019-10-08 09:52:57,792+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [baed2fa3-bcad-43b2-8164-480598bc72f3] Command 'RemoveSnapshot' (id: '7aafe2fe-e05b-44bc-a716-c48734a8c2de') waiting on child command id: '4919b287-e980-4d34-a219-c08a169cd8f7' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:57,799+03 INFO [org.ovirt.engine.core.bll.ConcurrentChildCommandsExecutionCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-90) [825c211d-b40d-45d7-9dc7-e5c6af28a269] Command 'RemoveSnapshot' (id: 'cb6e7a1d-1902-4e61-9c3d-e541cf4c6348') waiting on child command id: '5e9f9b03-4df9-4e31-b2e1-5efaa4fcf66c' type:'RemoveSnapshotSingleDiskLive' to complete 2019-10-08 09:52:58,345+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalLogicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] FINISH, GetGlusterLocalLogicalVolumeListVDSCommand, return: [org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@65406163, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@1207b8a9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@33f93de1, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6d2ae59d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@b07ca22, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@246ae91b, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@5e86ce42, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@643ade02, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@51d95e7d, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@2288576c, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@4a9469a1, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@78ec643f, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@63dde7e9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@30effc9, org.ovirt.engine.core.common.businessentities.gluster.GlusterLocalLogicalVolume@6a3e4768], log id: 449353d0 2019-10-08 09:52:58,346+03 INFO [org.ovirt.engine.core.vdsbroker.gluster.GetGlusterLocalPhysicalVolumeListVDSCommand] (DefaultQuartzScheduler10) [8921c9c] START, GetGlusterLocalPhysicalVolumeListVDSCommand(HostName = host-dell-2.domain.int, VdsIdVDSCommandParametersBase:{hostId='f598de28-296a-46ce-8b8d-6f19a2c892e2'}), log id: 71680b8c 2019-10-08 09:52:58,807+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '341d9c1b-2915-48d6-a8a9-9146ab19d5f8') waiting on child command id: '329da0fd-801b-4e0d-b7c0-fbb5c2a98bb5' type:'DestroyImage' to complete 2019-10-08 09:52:58,817+03 INFO [org.ovirt.engine.core.bll.snapshots.RemoveSnapshotSingleDiskLiveCommandCallback] (EE-ManagedThreadFactory-engineScheduled-Thread-81) [73016a4a-bb2f-487f-91c5-cd027b278930] Command 'RemoveSnapshotSingleDiskLive' (id: '580fa033-35fd-44f0-9979-e60e9bbf8a29') waiting on child command id: 'c00bdeb6-2e8b-4ef8-a3dc-1aaa088ae052' type:'DestroyImage' to complete -- Best regards, Leo David

2 3

Problem with ovirt 4.3 and systemd-journal-gateway
by Fabrice Bacchella 08 Oct '19

08 Oct '19

I'm forwarding logs using systemd-journal-gateway (https://www.freedesktop.org/software/systemd/man/systemd-journal-gatewayd.s…) But it fails with ovirt 4.3 on CentOS 7.7: lsb_release -a LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.7.1908 (Core) Release: 7.7.1908 Codename: Core yum update ... Error: Package: systemd-journal-gateway-219-67.el7_7.1.x86_64 (@updates) Requires: libmicrohttpd.so.10()(64bit) Removing: libmicrohttpd-0.9.33-2.el7.x86_64 (@base) libmicrohttpd.so.10()(64bit) Updated By: 1:libmicrohttpd-0.9.59-2.el7.x86_64 (ovirt-4.3-centos-opstools) ~libmicrohttpd.so.12()(64bit) Is that a known bug ?

1 0

Re: Problem with logical network
by Strahil 08 Oct '19

08 Oct '19

Can you check your hosts and especially their networks. If any network is not OK (out of sync), you can fix it from the UI. Best Regards, Strahil NikolovOn Oct 4, 2019 23:47, Alex Irmel Oviedo Solis <alexove(a)alexove.me> wrote: > > Hello, I'm trying to run a vm with two ethernet interfaces, one of them attached to ovirmnt and the other attached to a logical network named internal_servers and got this error: > > "VM is down with error. Exit message: can not get MTU interface in 'br-int': No such device2 > > Then I created the bridge manually but I get the following error: > > "VM is down with error. Exit message: Hook Error: ('',)." > > Please help me. > > Best Regards >

4 3

[ANN] oVirt Engine 4.3.6 async update is now available
by Sandro Bonazzola 08 Oct '19

08 Oct '19

The oVirt Team has just released a new version of ovirt-engine package that fixes a few high severity bugs: - [BZ 1755869 <https://bugzilla.redhat.com/1755869>] ovirt-engine throws an exception when trying to access VMs which have snapshots from unsupported compatibility levels - [BZ 1757782 <https://bugzilla.redhat.com/1757782>] Deleting a Single snapshot disk live merge 'REDUCE_IMAGE' step fails with NPE and snapshot disk remains in illegal state - [BZ 1758052 <https://bugzilla.redhat.com/1758052>] Create VM rollback fails with NPE leaving VM locked - Exception in invoking callback of command AddVmFromTemplate- Exception: javax.ejb.EJBTransactionRolledbackException We recommend to update ovirt-engine in order to avoid these issues. Thanks, -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours.*

1 0

Cannot start a VM after making it headless
by Thorsten Glaser 07 Oct '19

07 Oct '19

Hi everyone, I’ve installed a Windows Server 2016 VM, enabled RDP in it, and now want to make it headless. Upon starting I get a failure; the detail message is: VM 〈name〉 is down with error. Exit message: Nicht unterstützte Konfiguration: graphics device is needed for attribute value 'display=on' in <hostdev>. “Nicht unterstützte Konfiguration” means “unsupported configuration”. Software Version:4.3.5.5-1.el7 This works on another cluster which runs: 4.1.7.6-1.el7.centos Thanks in advance, //mirabilos -- tarent solutions GmbH Rochusstraße 2-4, D-53123 Bonn • http://www.tarent.de/ Tel: +49 228 54881-393 • Fax: +49 228 54881-235 HRB 5168 (AG Bonn) • USt-ID (VAT): DE122264941 Geschäftsführer: Dr. Stefan Barth, Kai Ebenrett, Boris Esser, Alexander Steeg

1 0

Thin pool best proctices
by jeremy_tourville＠hotmail.com 07 Oct '19

07 Oct '19

After a week of downtime from a thin pool that had 100% full metadata I decided to rebuild my single node. My post was formerly titled: Ovirt 4.2.7 won't start and drops to emergency console. I have backups of nearly all my VMs and I am working on importing them. I went from version 4.2.7.1 to 4.3.6. I am comfortable with LVM but I am new to administering thin pool LVM. I am asking myself what can I do to avoid this headache in the future? What practical steps should I take? -Regular backups -Monitoring -Proper configuration It's the last one that is the most difficult for me. You don't know what you don't know.... Can anyone share tips and advice? I learned that /etc/LVM has backups. It should be part of the backup strategy. I now know that thin pools have a special metadata volume that should be monitored for disk space. At the time I built the system I used the gedploy gluster config file. I had assumed the values in there were "safe". As I found out the hard way, reality proved otherwise. I was reading elsewhere about allowing the thin volume to grow 20% as needed but no real mention of how to do that or a good example. As I understand it, the config to make that happen exists in lvm.conf. Is that right? Have there been any changes in Ansible GlusterFS setup since 4.2.7 to choose different "safe" values with newer versions? I hope my comments won't be taken as critical by any Dev Team members regarding Ansible. My sole intent for this post is so I learn what to do better/right next time. I do have most of my VMs and the few I lost are not critical. It was lucky that I learned a hard lesson the easy way. Thanks for your input!

2 1

oVirt Software after upgrade
by Rick A 07 Oct '19

07 Oct '19

I'm wondering why after upgrading our hosts, when viewing the host details, it's still showing the OS Description as 4.3.3.1. We are using the oVirt Node. Shouldn't the OS Description show the version we just upgraded to? I do the upgrade using the web UI. Thanks OS Version: RHEL - 7 - 6.1810.2.el7.centos OS Description: oVirt Node 4.3.3.1 <-------------------------------------- Kernel Version: 3.10.0 - 957.10.1.el7.x86_64 KVM Version: 2.12.0 - 18.el7_6.3.1 LIBVIRT Version: libvirt-4.5.0-10.el7_6.6 VDSM Version: vdsm-4.30.13-1.el7 SPICE Version: 0.14.0 - 6.el7_6.1 GlusterFS Version: glusterfs-5.5-1.el7 CEPH Version: librbd1-10.2.5-4.el7 Open vSwitch Version: openvswitch-2.10.1-3.el7 Kernel Features: PTI: 1, IBRS: 1, RETP: 0, SSBD: 3 VNC Encryption: Disabled

2 1

Re: Managed Block Storage/Ceph: Experiences from Catastrophic Hardware failure
by Strahil 07 Oct '19

07 Oct '19

Hi Dan, As CEPH support is quite new, we need DEV clarification. Hi Sandro, Who can help to clarify if Ovirt supports direct RBD LUNs presented on the VMs? Are there any limitations in the current solution ? Best Regards, Strahil NikolovOn Oct 7, 2019 13:54, Dan Poltawski <dan.poltawski(a)tnp.net.uk> wrote: > > On Mon, 2019-10-07 at 01:56 +0300, Strahil Nikolov wrote: > > I'm not very sure that you are supposed to use the CEPH by giving > > each VM direct access. > > > > Have you considered using an iSCSI gateway as an entry point for your > > storage domain ? This way oVirt will have no issues dealing with the > > rbd locks. > > > > Of course, oVirt might be able to deal with RBD locks , but that can > > be confirmed/denied by the devs. > > Thanks for your response - regarding the locks point, I realised later > that this was my own incorrect permissions given to the client. The > ceph client was detecting the broken locks when mounting the rbd device > and unable to blacklist it. I addressed this by swithcign to the > permissions 'profile rbd'. > > Regarding iSCSI, we are using this for the hosted engine. However, I am > attracted to the idea of managing block devices with individual rbd > devices to facilate individual block device level snapshotting and I > assume performance will be better. > > thanks, > > Dan > > ________________________________ > > The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 > > This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

2 1

Re: VDI
by Leo David 07 Oct '19

07 Oct '19

Thank you for sharing the informations Alex, they are very helpfull. I am now able to get sound from the vms, although performance is pretty poor even with "adjust for performance" setting in Win10. Cannot even talk about youtube video playing - freezing and crackling. Could you please be so kind to share the following infos: 1. Have you upgraded the "spice-server" installed on the hosts with a newer version than 1.4.0 ? If so, could you provide me how could I get these packages ? 2. What graphic card have you used for getting better graphic performance with the vms ? Im trying to understand what "accepted" card could I use with my 1U chassis servers... 3. Is it only needed to install the card and the platform will alocate physical video memory to "desktop" vms ? ( Will the card RAM be automatically shared across the desktop tyoe vms running on top of the host ? ) 4. Is it necesarilly to activate sr-iov in the hosts bios or any other platform configurations ? I am really sorry for asking too many things, but im just trying to get these vdi vms working at least decent... Thank you so much ! Leo ---------- Forwarded message --------- From: <alex(a)triadic.us> Date: Tue, Sep 24, 2019 at 7:50 PM Subject: Re: [ovirt-users] Re: VDI To: Leo David <leoalex(a)gmail.com> Audio should just work as long as the VM is of the desktop type. On Sep 24, 2019 6:50 AM, Leo David <leoalex(a)gmail.com> wrote: Thank you Alex, When you say "gpu backed" are you referring to sr-iov to share same gpu to multiple vms ? Any thoughts regarding passing audio form the vm to the client ? Did you do any update of the spice-server on the hosts ? Thanks, Leo On Tue, Sep 24, 2019 at 12:01 PM <alex(a)triadic.us> wrote: I believe a lot of the package updates in CentOS 8 will solve some of the issues. But for now we get around them by disabling all visual effects on our VMS. If you are gpu backing the VMS with something like Nvidia grid the issues are non existent, but for non gpu backed VMS currently disabling all the effects is a must. We deploy the changes via gpo directly to the registry, so they take effect on first VM boot. On Sep 24, 2019 2:03 AM, Leo David <leoalex(a)gmail.com> wrote: Thank you Alex from my side as well, very usefull information. I am the middle of vdi implementation as well, and i'm having issues with the spice console since 4.2, and it seems that latest 4.3 is still having the problem. What am i confrunting is: - spice console is very slaggy and slow for Win10 vms ( not even talking about running videos..) - i can't find a way to get audio from the vm At the moment i am running 4.3, latest virt-viewer installed on the client, and latest qxl-dod driver installed on the vm. Any thoughts on solving video performance and audio redirection ? Thank you again, Leo On Mon, Sep 23, 2019, 22:53 Alex McWhirter <Alex(a)triadic.us> wrote: To achieve that all you need to do is create a template of the desktop base vm, make sure the vm type is set to desktop. Afterwards just create new vms from that template. As long as the VM type is set to desktop each new VM will use a qcow overlay on top of the base image. Taking this a step further you can then create VM pools from said template, allowing users to dynamically be assigned a new VM on login. Granted pools are usually stateless, so you need to have network file storage. We use pools for windows 10 VDI instances, where we use sysprep to autojoin the new pool vm to the domain where redirected folders are already setup. For VDI only use spice protocol. By default we found spice to be semi lackluster, so we do apply custom settings and we have recompiled spice on both servers and clients with h264 support. This is not 100% necessary, but makes things like youtube much more usable. We have also backported some audio patches to KVM. CentOS 8 should resolve a lot of these customizations that we've had to do. As far as updating, pretty much. We create a VM from the template, update it, then push it back as a new version of the template. The pools are set to always use the latest template version. Users have to log out, then back in to the VDI system in order to get the new image as logging out will destroy the users current instance and create a new one on log in. On 2019-09-23 15:16, Fabio Marzocca wrote: Hi Alex, thanks for answering. I am approaching and studying oVirt in order to propose the solution to a customer as a replacement for a commercial solution they have now. They only need Desktop virtualization. Sorry for the silly question, but I can't find a way to deploy a VM (template) to users as a "linked-clone", meaning that the users' image still refers to the original image but modification are written (and afterwards read) from a new location. This technique is called Copy-on-write. Can this be achieved with oVirt? Then, what is the Best Practice to update WIndows OS for the all the users? Currently they simply "check-out" the Gold Image, update it and check-in, while all users are running... Fabio On Mon, Sep 23, 2019 at 8:04 PM Alex McWhirter <Alex(a)triadic.us> wrote: yes, we do. All spice, with some customizations done at source level for spice / kvm packages. On 2019-09-23 13:44, Fabio Marzocca wrote: Is there anyone who uses oVirt as a full VDI environment? I would have a bunch of questions... _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/D44YB5VOKNBNC… _______________________________________________ Users mailing list -- users(a)ovirt.org To unsubscribe send an email to users-leave(a)ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/DWY46KJGSYJGR… -- Best regards, Leo David -- Best regards, Leo David

2 3

Managed Block Storage/Ceph: Experiences from Catastrophic Hardware failure
by Dan Poltawski 07 Oct '19

07 Oct '19

Yesterday we had a catastrophic hardware failure with one of our nodes using ceph and the experimental cinderlib integration. Unfortunately the ovirt cluster recover the situation well and took some manual intervention to resolve. I thought I'd share what happened and how we resolved it in case there is any best practice to share/bugs which are worth creating to help others in similar situaiton. We are early in our use of ovirt, so its quite possible we have things incorreclty configured. Our setup: We have two nodes, hosted engine on iSCSI, about 40vms all using managed block storage mounting the rbd volumes directly. I hadn't configured power management (perhaps this is the fundamental problem). Yesterday a hardware fault caused one of the nodes to crash and stay down awaiting user input in POST screens, taking 20 vms with it. The hosted engine was fortunately on the 'good' node and detected that the node had become unresponsive, but noted 'Host cannot be fenced automatically because power management for the host is disabled.'. At this point, knowing that one node was dead, I wanted to bring up the failed vms on the good node. However, the vms were appearing in an unknown state and I couldn't do any operations on them. It wasn't clear to me what the best course of action to do there would be. I am not sure if there is a way to mark the node as failed? In my urgency to try and resolve the situation I managed to get the failed node startred back up, shortly after it came up the engine detected that all the vms were down, I put the failed host into maintaince mode and tried to start the failed vms. Unfortunately the failed vms did not start up cleanly - it turned out that they still had rbd locks preventing writing from the failed node. To finally gets the vms to start I then manually went through every vm's managed block, found the id and found the lock and removed it: rbd lock list rbd/volume-{id} rbd lock remove rbd/voleume-{id} 'auto {lockid}' {lockername} Some overall thoughts I had: * I'm not sure what the best course of action is to notify the engine about a catastrophic hardware failure? If power management was configured, I suppose it would've removed the power and marked them all down? * Would ovirt have been able to deal with clearing the rbd locks, or did I miss a trick somewhere to resolve this situation with manually going through each device and clering the lock? * Might it be possible for ovirt to detect when the rbd images are locked for writing and prevent launching? regards, Dan ________________________________ The Networking People (TNP) Limited. Registered office: Network House, Caton Rd, Lancaster, LA1 3PE. Registered in England & Wales with company number: 07667393 This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. This message contains confidential information and is intended only for the individual named. If you are not the named addressee you should not disseminate, distribute or copy this e-mail. Please notify the sender immediately by e-mail if you have received this e-mail by mistake and delete this e-mail from your system. If you are not the intended recipient you are notified that disclosing, copying, distributing or taking any action in reliance on the contents of this information is strictly prohibited.

3 2

[ANN] oVirt 4.3.6 is now generally available
by Sandro Bonazzola 06 Oct '19

06 Oct '19

The oVirt Project is pleased to announce the general availability of oVirt 4.3.6 as of September 26th, 2019. This update is the sixth in a series of stabilization updates to the 4.3 series. This release is available now on x86_64 architecture for: * Red Hat Enterprise Linux 7.7 or later (but < 8) * CentOS Linux (or similar) 7.7 or later (but < 8) This release supports Hypervisor Hosts on x86_64 and ppc64le architectures for: * Red Hat Enterprise Linux 7.7 or later (but < 8) * CentOS Linux (or similar) 7.7 or later (but < 8) * oVirt Node 4.3 (available for x86_64 only) Due to Fedora 28 being now at end of life this release is missing experimental tech preview for x86_64 and s390x architectures for Fedora 28. We are working on Fedora 29 and 30 support and we may re-introduce experimental support for Fedora in next release. See the release notes [1] for installation / upgrade instructions and a list of new features and bugs fixed. Notes: - oVirt Appliance is already available - oVirt Node is already available[2] oVirt Node and Appliance have been updated including: - oVirt 4.3.6: http://www.ovirt.org/release/4.3.6/ - Wildfly 17.0.1: https://wildfly.org/news/2019/07/07/WildFly-1701-Released/ - Latest CentOS 7.7 updates including: - Release for CentOS Linux 7 (1908) on the x86_64 Architecture <https://lists.centos.org/pipermail/centos-announce/2019-September/023405.ht…> - CEBA-2019:2601 CentOS 7 NetworkManager BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023423.ht…> - CEBA-2019:2023 CentOS 7 efivar BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023445.ht…> - CEBA-2019:2614 CentOS 7 firewalld BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023412.ht…> - CEBA-2019:2227 CentOS 7 grubby BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023441.ht…> - CESA-2019:2258 Moderate CentOS 7 http-parser Security Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023439.ht…> - CESA-2019:2600 Important CentOS 7 kernel Security Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023444.ht…> - CEBA-2019:2599 CentOS 7 krb5 BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023420.ht…> - CEBA-2019:2358 CentOS 7 libguestfs BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023421.ht…> - CEBA-2019:2679 CentOS 7 libvirt BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023422.ht…> - CEBA-2019:2501 CentOS 7 rsyslog BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023431.ht…> - CEBA-2019:2355 CentOS 7 selinux-policy BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023432.ht…> - CEBA-2019:2612 CentOS 7 sg3_utils BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023433.ht…> - CEBA-2019:2602 CentOS 7 sos BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023434.ht…> - CEBA-2019:2564 CentOS 7 subscription-manager BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023435.ht…> - CEBA-2019:2356 CentOS 7 systemd BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023436.ht…> - CEBA-2019:2605 CentOS 7 tuned BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023437.ht…> - CEBA-2019:2871 CentOS 7 tzdata BugFix Update <https://lists.centos.org/pipermail/centos-announce/2019-September/023450.ht…> - latest CentOS Virt and Storage SIG updates: - Ansible 2.8.5: https://github.com/ansible/ansible/blob/stable-2.8/changelogs/CHANGELOG-v2.… - Glusterfs 6.5: https://docs.gluster.org/en/latest/release-notes/6.5/ - QEMU KVM EV 2.12.0-33.1 : https://cbs.centos.org/koji/buildinfo?buildID=26484 Given the amount of security fixes provided by this release, upgrade is recommended as soon as practical. Additional Resources: * Read more about the oVirt 4.3.6 release highlights: http://www.ovirt.org/release/4.3.6/ * Get more oVirt Project updates on Twitter: https://twitter.com/ovirt * Check out the latest project news on the oVirt blog: http://www.ovirt.org/blog/ [1] http://www.ovirt.org/release/4.3.6/ [2] http://resources.ovirt.org/pub/ovirt-4.3/iso/ -- Sandro Bonazzola MANAGER, SOFTWARE ENGINEERING, EMEA R&D RHV Red Hat EMEA <https://www.redhat.com/> sbonazzo(a)redhat.com <https://www.redhat.com/>*Red Hat respects your work life balance. Therefore there is no need to answer this email out of your office hours. <https://mojo.redhat.com/docs/DOC-1199578>*

13 33

Trying to add a LUN to a host and use that, instead of Gluster, as the datastore for my VM's.
by TomK 06 Oct '19

06 Oct '19

Hey All, I've added a 4TB LUN to my physical storage and now I want to use that, instead of GlusterFS as my datastore for VM's on that physical. How do I do this? I've tried a number of options, including adding LUN's but appears I can only add them to VM's directly but not use them as my datastores instead of GlusterFS. -- Thx, TK.

1 0

Problem with logical network
by Alex Irmel Oviedo Solis 04 Oct '19

04 Oct '19

Hello, I'm trying to run a vm with two ethernet interfaces, one of them attached to ovirmnt and the other attached to a logical network named internal_servers and got this error: "VM is down with error. Exit message: can not get MTU interface in 'br-int': No such device2 Then I created the bridge manually but I get the following error: "VM is down with error. Exit message: Hook Error: ('',)." Please help me. Best Regards

1 0

oVirt 4.3.5 glusterfs 6.3 performance tunning
by adrianquintero＠gmail.com 04 Oct '19

04 Oct '19

Hello, I have a hyperconverged setup using ovirt 4.3.5 and the "optimize for ovirt store" seems to fail on gluster volumes. I am seeing poor performance and trying to see how should I tune gluster to give better performance. Can you provide any suggestions on the following volume settings(parameters)? option Value ------ ----- cluster.lookup-unhashed on cluster.lookup-optimize on cluster.min-free-disk 10% cluster.min-free-inodes 5% cluster.rebalance-stats off cluster.subvols-per-directory (null) cluster.readdir-optimize off cluster.rsync-hash-regex (null) cluster.extra-hash-regex (null) cluster.dht-xattr-name trusted.glusterfs.dht cluster.randomize-hash-range-by-gfid off cluster.rebal-throttle normal cluster.lock-migration off cluster.force-migration off cluster.local-volume-name (null) cluster.weighted-rebalance on cluster.switch-pattern (null) cluster.entry-change-log on cluster.read-subvolume (null) cluster.read-subvolume-index -1 cluster.read-hash-mode 1 cluster.background-self-heal-count 8 cluster.metadata-self-heal off cluster.data-self-heal off cluster.entry-self-heal off cluster.self-heal-daemon on cluster.heal-timeout 600 cluster.self-heal-window-size 1 cluster.data-change-log on cluster.metadata-change-log on cluster.data-self-heal-algorithm full cluster.eager-lock enable disperse.eager-lock on disperse.other-eager-lock on disperse.eager-lock-timeout 1 disperse.other-eager-lock-timeout 1 cluster.quorum-type auto cluster.quorum-count (null) cluster.choose-local off cluster.self-heal-readdir-size 1KB cluster.post-op-delay-secs 1 cluster.ensure-durability on cluster.consistent-metadata no cluster.heal-wait-queue-length 128 cluster.favorite-child-policy none cluster.full-lock yes diagnostics.latency-measurement off diagnostics.dump-fd-stats off diagnostics.count-fop-hits off diagnostics.brick-log-level INFO diagnostics.client-log-level INFO diagnostics.brick-sys-log-level CRITICAL diagnostics.client-sys-log-level CRITICAL diagnostics.brick-logger (null) diagnostics.client-logger (null) diagnostics.brick-log-format (null) diagnostics.client-log-format (null) diagnostics.brick-log-buf-size 5 diagnostics.client-log-buf-size 5 diagnostics.brick-log-flush-timeout 120 diagnostics.client-log-flush-timeout 120 diagnostics.stats-dump-interval 0 diagnostics.fop-sample-interval 0 diagnostics.stats-dump-format json diagnostics.fop-sample-buf-size 65535 diagnostics.stats-dnscache-ttl-sec 86400 performance.cache-max-file-size 0 performance.cache-min-file-size 0 performance.cache-refresh-timeout 1 performance.cache-priority performance.cache-size 32MB performance.io-thread-count 16 performance.high-prio-threads 16 performance.normal-prio-threads 16 performance.low-prio-threads 32 performance.least-prio-threads 1 performance.enable-least-priority on performance.iot-watchdog-secs (null) performance.iot-cleanup-disconnected- reqsoff performance.iot-pass-through false performance.io-cache-pass-through false performance.cache-size 128MB performance.qr-cache-timeout 1 performance.cache-invalidation false performance.ctime-invalidation false performance.flush-behind on performance.nfs.flush-behind on performance.write-behind-window-size 1MB performance.resync-failed-syncs-after -fsyncoff performance.nfs.write-behind-window-s ize1MB performance.strict-o-direct on performance.nfs.strict-o-direct off performance.strict-write-ordering off performance.nfs.strict-write-ordering off performance.write-behind-trickling-wr iteson performance.aggregate-size 128KB performance.nfs.write-behind-tricklin g-writeson performance.lazy-open yes performance.read-after-open yes performance.open-behind-pass-through false performance.read-ahead-page-count 4 performance.read-ahead-pass-through false performance.readdir-ahead-pass-throug h false performance.md-cache-pass-through false performance.md-cache-timeout 1 performance.cache-swift-metadata true performance.cache-samba-metadata false performance.cache-capability-xattrs true performance.cache-ima-xattrs true performance.md-cache-statfs off performance.xattr-cache-list performance.nl-cache-pass-through false features.encryption off network.frame-timeout 1800 network.ping-timeout 30 network.tcp-window-size (null) client.ssl off network.remote-dio off client.event-threads 4 client.tcp-user-timeout 0 client.keepalive-time 20 client.keepalive-interval 2 client.keepalive-count 9 network.tcp-window-size (null) network.inode-lru-limit 16384 auth.allow * auth.reject (null) transport.keepalive 1 server.allow-insecure on server.root-squash off server.all-squash off server.anonuid 65534 server.anongid 65534 server.statedump-path /var/run/gluster server.outstanding-rpc-limit 64 server.ssl off auth.ssl-allow * server.manage-gids off server.dynamic-auth on client.send-gids on server.gid-timeout 300 server.own-thread (null) server.event-threads 4 server.tcp-user-timeout 42 server.keepalive-time 20 server.keepalive-interval 2 server.keepalive-count 9 transport.listen-backlog 1024 transport.address-family inet performance.write-behind on performance.read-ahead off performance.readdir-ahead on performance.io-cache off performance.open-behind on performance.quick-read off performance.nl-cache off performance.stat-prefetch on performance.client-io-threads on performance.nfs.write-behind on performance.nfs.read-ahead off performance.nfs.io-cache off performance.nfs.quick-read off performance.nfs.stat-prefetch off performance.nfs.io-threads off performance.force-readdirp true performance.cache-invalidation false performance.global-cache-invalidation true features.uss off features.snapshot-directory .snaps features.show-snapshot-directory off features.tag-namespaces off network.compression off network.compression.window-size -15 network.compression.mem-level 8 network.compression.min-size 0 network.compression.compression-level -1 network.compression.debug false features.default-soft-limit 80% features.soft-timeout 60 features.hard-timeout 5 features.alert-time 86400 features.quota-deem-statfs off geo-replication.indexing off geo-replication.indexing off geo-replication.ignore-pid-check off geo-replication.ignore-pid-check off features.quota off features.inode-quota off features.bitrot disable debug.trace off debug.log-history no debug.log-file no debug.exclude-ops (null) debug.include-ops (null) debug.error-gen off debug.error-failure (null) debug.error-number (null) debug.random-failure off debug.error-fops (null) nfs.disable on features.read-only off features.worm off features.worm-file-level off features.worm-files-deletable on features.default-retention-period 120 features.retention-mode relax features.auto-commit-period 180 storage.linux-aio off storage.batch-fsync-mode reverse-fsync storage.batch-fsync-delay-usec 0 storage.owner-uid 36 storage.owner-gid 36 storage.node-uuid-pathinfo off storage.health-check-interval 30 storage.build-pgfid off storage.gfid2path on storage.gfid2path-separator : storage.reserve 1 storage.health-check-timeout 10 storage.fips-mode-rchecksum off storage.force-create-mode 0 storage.force-directory-mode 0 storage.create-mask 777 storage.create-directory-mask 777 storage.max-hardlinks 100 features.ctime on config.gfproxyd off cluster.server-quorum-type server cluster.server-quorum-ratio 0 changelog.changelog off changelog.changelog-dir {{ brick.path }}/.glusterfs/changelogs changelog.encoding ascii changelog.rollover-time 15 changelog.fsync-interval 5 changelog.changelog-barrier-timeout 120 changelog.capture-del-path off features.barrier disable features.barrier-timeout 120 features.trash off features.trash-dir .trashcan features.trash-eliminate-path (null) features.trash-max-filesize 5MB features.trash-internal-op off cluster.enable-shared-storage disable locks.trace off locks.mandatory-locking off cluster.disperse-self-heal-daemon enable cluster.quorum-reads no client.bind-insecure (null) features.shard on features.shard-block-size 64MB features.shard-lru-limit 16384 features.shard-deletion-rate 100 features.scrub-throttle lazy features.scrub-freq biweekly features.scrub false features.expiry-time 120 features.cache-invalidation off features.cache-invalidation-timeout 60 features.leases off features.lease-lock-recall-timeout 60 disperse.background-heals 8 disperse.heal-wait-qlength 128 cluster.heal-timeout 600 dht.force-readdirp on disperse.read-policy gfid-hash cluster.shd-max-threads 8 cluster.shd-wait-qlength 10000 cluster.locking-scheme granular cluster.granular-entry-heal enable features.locks-revocation-secs 0 features.locks-revocation-clear-all false features.locks-revocation-max-blocked 0 features.locks-monkey-unlocking false features.locks-notify-contention no features.locks-notify-contention-dela y 5 disperse.shd-max-threads 1 disperse.shd-wait-qlength 1024 disperse.cpu-extensions auto disperse.self-heal-window-size 1 cluster.use-compound-fops off performance.parallel-readdir off performance.rda-request-size 131072 performance.rda-low-wmark 4096 performance.rda-high-wmark 128KB performance.rda-cache-limit 10MB performance.nl-cache-positive-entry false performance.nl-cache-limit 10MB performance.nl-cache-timeout 60 cluster.brick-multiplex off cluster.max-bricks-per-process 250 disperse.optimistic-change-log on disperse.stripe-cache 4 cluster.halo-enabled False cluster.halo-shd-max-latency 99999 cluster.halo-nfsd-max-latency 5 cluster.halo-max-latency 5 cluster.halo-max-replicas 99999 cluster.halo-min-replicas 2 features.selinux on cluster.daemon-log-level INFO debug.delay-gen off delay-gen.delay-percentage 10% delay-gen.delay-duration 100000 delay-gen.enable disperse.parallel-writes on features.sdfs off features.cloudsync off features.ctime on ctime.noatime on feature.cloudsync-storetype (null) features.enforce-mandatory-lock off thank you!

2 2

Re: Recover node from partial upgrade
by Strahil 04 Oct '19

04 Oct '19

You can boot from the older image (grub menu) and then a cleanup procedure should be performed. I'm not sure if the thin LVs will be removed when you rerun the upgrade process , or they will remain the same. Beat Regards, Strahil NikolovOn Oct 4, 2019 14:04, Stefano Danzi <s.danzi(a)hawai.it> wrote: > > Hello!!! > > I upgraded oVirtNode 4.3.5.2 to 4.3.6 using Engine UI. > In the middle of the upgrade Engine fenced it, I can't understand why. > > Now Engine UI, and yum, report that there are no updates but node still > to version 4.3.5.2. > If I run "yum reinstall ovirt-node-ng-image-update" The result is as > follow, but the system remain not upgraded. > > ovirt-node-ng-image-update-4.3.6-1.el7.noarch.rpm | 719 MB 00:06:31 > Running transaction check > Running transaction test > Transaction test succeeded > Running transaction > Installazione : ovirt-node-ng-image-update-4.3.6-1.el7.noarch 1/1 > warning: %post(ovirt-node-ng-image-update-4.3.6-1.el7.noarch) scriptlet > failed, exit status 1 > Non-fatal POSTIN scriptlet failure in rpm package > ovirt-node-ng-image-update-4.3.6-1.el7.noarch > Uploading Package Profile > Cannot upload package profile. Is this client registered? > Verifica in corso : ovirt-node-ng-image-update-4.3.6-1.el7.noarch 1/1 > > Installato: > ovirt-node-ng-image-update.noarch 0:4.3.6-1.el7 > > Completo! > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Y52X663MAR347…

2 1

Recover node from partial upgrade
by Stefano Danzi 04 Oct '19

04 Oct '19

Hello!!! I upgraded oVirtNode 4.3.5.2 to 4.3.6 using Engine UI. In the middle of the upgrade Engine fenced it, I can't understand why. Now Engine UI, and yum, report that there are no updates but node still to version 4.3.5.2. If I run "yum reinstall ovirt-node-ng-image-update" The result is as follow, but the system remain not upgraded. ovirt-node-ng-image-update-4.3.6-1.el7.noarch.rpm | 719 MB 00:06:31 Running transaction check Running transaction test Transaction test succeeded Running transaction Installazione : ovirt-node-ng-image-update-4.3.6-1.el7.noarch 1/1 warning: %post(ovirt-node-ng-image-update-4.3.6-1.el7.noarch) scriptlet failed, exit status 1 Non-fatal POSTIN scriptlet failure in rpm package ovirt-node-ng-image-update-4.3.6-1.el7.noarch Uploading Package Profile Cannot upload package profile. Is this client registered? Verifica in corso : ovirt-node-ng-image-update-4.3.6-1.el7.noarch 1/1 Installato: ovirt-node-ng-image-update.noarch 0:4.3.6-1.el7 Completo!

1 0

Re: Ovirt 4.2.7 won't start and drops to emergency console
by Strahil 04 Oct '19

04 Oct '19

Trust Red Hat :) At least their approach should be safer. Of course, you can raise a docu bug but RHEL7 is in such phase that it might not be fixed unless this is found in v8. Best Regards, Strahil NikolovOn Oct 2, 2019 05:43, jeremy_tourville(a)hotmail.com wrote: > > http://man7.org/linux/man-pages/man7/lvmthin.7.html > Command to repair a thin pool: > lvconvert --repair VG/ThinPoolLV > > Repair performs the following steps: > > 1. Creates a new, repaired copy of the metadata. > lvconvert runs the thin_repair command to read damaged metadata from > the existing pool metadata LV, and writes a new repaired copy to the > VG's pmspare LV. > > 2. Replaces the thin pool metadata LV. > If step 1 is successful, the thin pool metadata LV is replaced with > the pmspare LV containing the corrected metadata. The previous thin > pool metadata LV, containing the damaged metadata, becomes visible > with the new name ThinPoolLV_tmetaN (where N is 0,1,...). > > If the repair works, the thin pool LV and its thin LVs can be > activated, and the LV containing the damaged thin pool metadata can > be removed. It may be useful to move the new metadata LV (previously > pmspare) to a better PV. > > If the repair does not work, the thin pool LV and its thin LVs are > lost. > > This info seems to conflict with Red Hat advice. Red Hat says if metadat volume is full don't run a lvconvert --repair operation. Now I am confused. I am familiar with LVM and comfortable with it but this is my first time trying to repair thin LVM. The concepts of a metadata volume are new to me. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/FFN3HWUQVRPFS…

2 2