- Users - oVirt List Archives

Hosted-engine.metadata header is corrupted
by du_hongyu＠yeah.net 11 Jan '20

11 Jan '20

Hi my Hosted-engine.metadata header is corrupted. I should how to fix it? thanks Regards Hongyu Du

1 0

Re: HCI Disaster Recovery
by Strahil 10 Jan '20

10 Jan '20

It's actually not so easy. The fastest way to recover is just restore from backup. Otherwise the flow should be: 1. Install the new node (new hostname will be easier). 2. Use gluster's replace-brick to change the dead brick with new one. 3. Once ovirt 's integration with gluster detects the change, you will be able to forcefully remove the dead node. 4. Add the newly installed node to the relevant cluster (with or without hosted-engine deployment) 5. Test to move a low-priority VM to the new host. 6. Power up a test VM on the new host to test the functionality. 7. You can make the new node SPM and test snapahots and new vm creation. As you can see , in order to remove a missing node - it should not be part of the Gluster Cluster. Best Regards, Strahil NikolovOn Jan 10, 2020 20:10, Christian Reiss wrote: > > Hey, > > is there really no ovirt native way to restore a single host and bring > it back into the cluster? > > -Chris. > > On 07.01.2020 09:54, Christian Reiss wrote: > > Hey folks, > > > > - theoretical question, no live data in jeopardy - > > > > Let's say a 3-way HCI cluster is up and running, with engine running, > > all is well. The setup was done via gui, including gluster. > > > > Now I would kill a host, poweroff & disk wipe. Simulating a full node > > failure. > > > > The remaining nodes should keep running on (3 copy sync, no arbiter), > > vms keep running or will be restarted. I would reinstall using the ovirt > > node installer on the "failed" node. > > > > This would net me with a completely empty, no-gluster setup. What is the > > oVirt way to recover from this point onward? > > > > Thanks for your continued support! <3 > > -Christian. > > > > -- > Christian Reiss - email(a)christian-reiss.de /"\ ASCII Ribbon > christian(a)reiss.nrw \ / Campaign > X against HTML > XMPP chris(a)alpha-labs.net / \ in eMails > WEB christian-reiss.de, reiss.nrw > > GPG Retrieval http://gpg.christian-reiss.de > GPG ID ABCD43C5, 0x44E29126ABCD43C5 > GPG fingerprint = 9549 F537 2596 86BA 733C A4ED 44E2 9126 ABCD 43C5 > > "It's better to reign in hell than to serve in heaven.", > John Milton, Paradise lost. > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/GQECSH72L4SMO…

1 0

Hyperconverged setup - storage architecture - scaling
by Leo David 10 Jan '20

10 Jan '20

Hello Everyone, Reading through the document: "Red Hat Hyperconverged Infrastructure for Virtualization 1.5 Automating RHHI for Virtualization deployment" Regarding storage scaling, i see the following statements: *2.7. SCALINGRed Hat Hyperconverged Infrastructure for Virtualization is supported for one node, and for clusters of 3, 6, 9, and 12 nodes.The initial deployment is either 1 or 3 nodes.There are two supported methods of horizontally scaling Red Hat Hyperconverged Infrastructure for Virtualization:* *1 Add new hyperconverged nodes to the cluster, in sets of three, up to the maximum of 12 hyperconverged nodes.* *2 Create new Gluster volumes using new disks on existing hyperconverged nodes.You cannot create a volume that spans more than 3 nodes, or expand an existing volume so that it spans across more than 3 nodes at a time* *2.9.1. Prerequisites for geo-replicationBe aware of the following requirements and limitations when configuring geo-replication:One geo-replicated volume onlyRed Hat Hyperconverged Infrastructure for Virtualization (RHHI for Virtualization) supports only one geo-replicated volume. Red Hat recommends backing up the volume that stores the data of your virtual machines, as this is usually contains the most valuable data.* ------ Also in oVirtEngine UI, when I add a brick to an existing volume i get the following warning: *"Expanding gluster volume in a hyper-converged setup is not recommended as it could lead to degraded performance. To expand storage for cluster, it is advised to add additional gluster volumes." * Those things are raising a couple of questions that maybe for some for you guys are easy to answer, but for me it creates a bit of confusion... I am also referring to RedHat product documentation, because I treat oVirt as production-ready as RHHI is. *1*. Is there any reason for not going to distributed-replicated volumes ( ie: spread one volume across 6,9, or 12 nodes ) ? - ie: is recomanded that in a 9 nodes scenario I should have 3 separated volumes, but how should I deal with the folowing question *2.* If only one geo-replicated volume can be configured, how should I deal with 2nd and 3rd volume replication for disaster recovery *3.* If the limit of hosts per datacenter is 250, then (in theory ) the recomended way in reaching this treshold would be to create 20 separated oVirt logical clusters with 12 nodes per each ( and datacenter managed from one ha-engine ) ? *4.* In present, I have the folowing one 9 nodes cluster , all hosts contributing with 2 disks each to a single replica 3 distributed replicated volume. They where added to the volume in the following order: node1 - disk1 node2 - disk1 ...... node9 - disk1 node1 - disk2 node2 - disk2 ...... node9 - disk2 At the moment, the volume is arbitrated, but I intend to go for full distributed replica 3. Is this a bad setup ? Why ? It oviously brakes the redhat recommended rules... Is there anyone so kind to discuss on these things ? Thank you very much ! Leo -- Best regards, Leo David -- Best regards, Leo David

3 5

Re: Ovirt OVN help needed
by Strahil 10 Jan '20

10 Jan '20

Hi Dominik, Thanks for your reply. On ovirt1 I got the following: [root@ovirt1 openvswitch]# less ovn-controller.log-20191216.gz 2019-12-15T01:49:02.988Z|00032|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log 2019-12-16T01:18:02.114Z|00033|vlog|INFO|closing log file ovn-controller.log-20191216.gz (END) Same is on the other node: [root@ovirt2 openvswitch]# less ovn-controller.log-20191216.gz 2019-12-15T01:26:03.477Z|00028|vlog|INFO|opened log file /var/log/openvswitch/ovn-controller.log 2019-12-16T01:30:01.718Z|00029|vlog|INFO|closing log file ovn-controller.log-20191216.gz (END) The strange thing is that the geneve tunnels are there: [root@ovirt1 ~]# ovs-vsctl show c0e938f1-b5b5-4d5a-9cda-29dae2986f29 Bridge br-int fail_mode: secure Port "ovn-25cc77-0" Interface "ovn-25cc77-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.64"} Port "ovn-566849-0" Interface "ovn-566849-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.41"} Port br-int Interface br-int type: internal Port "vnet2" Interface "vnet2" ovs_version: "2.11.0" [root@ovirt1 ~]# ovs-vsctl list ports ovs-vsctl: unknown table "ports" [root@ovirt1 ~]# ovs-vsctl list port _uuid : fbf40569-925e-4430-a7c5-c78d58979bbc bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [3207c0cb-3000-40f2-a850-83548f76f090]lacp : [] mac : [] name : "vnet2" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 8947f82d-a089-429b-8843-71371314cb52 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [ec6a6688-e5d6-4346-ac47-ece1b8379440]lacp : [] mac : [] name : br-int other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 72d612be-853e-43e9-8f5c-ce66cef0bebe bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3(a)192.168.1.41"} fake_bridge : false interfaces : [a31574fe-515b-420b-859d-7f2ac729638f]lacp : [] mac : [] name : "ovn-566849-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 2043a15f-ec39-4cc3-b875-7be00423dd7a bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="25cc77b3-046f-45c5-af0c-ffb2f77d73f1(a)192.168.1.64"} fake_bridge : false interfaces : [f9a9e3ff-070e-4044-b601-7f7394dc295f]lacp : [] mac : [] name : "ovn-25cc77-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] [root@ovirt1 ~]# [root@ovirt2 ~]# ovs-vsctl show 3dbab138-6b90-44c5-af05-b8a944c9bf20 Bridge br-int fail_mode: secure Port "ovn-baa019-0" Interface "ovn-baa019-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.90"} Port br-int Interface br-int type: internal Port "vnet5" Interface "vnet5" Port "ovn-566849-0" Interface "ovn-566849-0" type: geneve options: {csum="true", key=flow, remote_ip="192.168.1.41"} ovs_version: "2.11.0" [root@ovirt2 ~]# ovs-vsctl list port _uuid : 151e1188-f07a-4750-a620-392a08e7e7fe bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="baa0199e-d1a4-484c-af13-a41bcad19dbc(a)192.168.1.90"} fake_bridge : false interfaces : [4d4bc12a-609a-4917-b839-d4f652acdc33]lacp : [] mac : [] name : "ovn-baa019-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : 3a862f96-b3ec-46a9-bcf6-f385e5def410 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [777f2819-ca27-4890-8d2f-11349ca0d398]lacp : [] mac : [] name : br-int other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : a65109fa-f8b4-4670-8ae8-a2bd0bf6aba3 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {ovn-chassis-id="5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3(a)192.168.1.41"} fake_bridge : false interfaces : [ed442077-f897-4e0b-97a1-a8051e9c3d56]lacp : [] mac : [] name : "ovn-566849-0" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] _uuid : a1622e6f-fcd0-4a8a-b259-ca4d0ccf1cd2 bond_active_slave : [] bond_downdelay : 0 bond_fake_iface : false bond_mode : [] bond_updelay : 0 cvlans : [] external_ids : {} fake_bridge : false interfaces : [ca368654-54f3-49d0-a71c-8894426df6bf]lacp : [] mac : [] name : "vnet5" other_config : {} protected : false qos : [] rstp_statistics : {} rstp_status : {} statistics : {} status : {} tag : [] trunks : [] vlan_mode : [] [root@ovirt2 ~]# Best Regards, Strahil Nikolov On Dec 16, 2019 23:28, Dominik Holler <dholler(a)redhat.com> wrote: > > > > On Sat, Dec 14, 2019 at 11:36 AM Strahil Nikolov <hunter86_bg(a)yahoo.com> wrote: >> >> Hi Dominik, >> >> yes I was looking for those settings. >> >> I have added again the external provider , but I guess the mess is even bigger as I made some stupid decisions (like removing 2 port groups :) without knowing what I'm doing) . >> Sadly I can't remove all packages on the engine and hosts and reinstall them from scratch. >> >> Pip fails to install the openstacksdk (centOS7 is not great for such tasks) on the engine and my lack of knowledge in OVN makes it even more difficult. >> >> So the symptoms are that 2 machines can communicate with each other only if they are on the same host ,while on separate - no communications is happening. >> > > This indicates that the tunnels between the hosts are not created. > Can you please check the /var/log/openvswitch/ovn-controller.log on both hosts for errors and warnings, or share parts of the files here? > If this does not point us to a problem, ovn has to be reconfigured. If possible, most easy way to do this would be to ensure that > ovirt-provider-ovn is the default network provider of the cluster of the hosts, put one host after another in maintance mode and reinstall. > > >> >> How I created the network via UI: >> >> 1. Networks - new >> 2. Fill in the name >> 3. Create on external provider >> 4. Network Port security -> disabled (even undefined does not work) >> 5.Connect to physical network -> ovirtmgmt >> >> >> I would be happy to learn more about OVN and thus I would like to make it work. >> >> Here is some info from the engine: >> >> [root@engine ~]# ovn-nbctl show >> switch 1288ed26-471c-4bc2-8a7d-4531f306f44c (ovirt-pxelan-2a88b2e0-d04b-4196-ad50-074501e4ed08) >> port c1eba112-5eed-4c04-b25c-d3dcfb934546 >> addresses: ["56:6f:5a:65:00:06"] >> port 8b52ab60-f474-4d51-b258-cb2e0a53c34a >> type: localnet >> addresses: ["unknown"] >> port b2753040-881b-487a-92a1-9721da749be4 >> addresses: ["56:6f:5a:65:00:09"] >> [root@engine ~]# ovn-sbctl show >> Chassis "5668499c-7dd0-41ee-bc5d-2e6ee9cd61c3" >> hostname: "ovirt3.localdomain" >> Encap geneve >> ip: "192.168.1.41" >> options: {csum="true"} >> Chassis "baa0199e-d1a4-484c-af13-a41bcad19dbc" >> hostname: "ovirt1.localdomain" >> Encap geneve >> ip: "192.168.1.90" >> options: {csum="true"} >> Chassis "25cc77b3-046f-45c5-af0c-ffb2f77d73f1" >> hostname: "ovirt2.localdomain" >> Encap geneve >> ip: "192.168.1.64" >> options: {csum="true"} >> Port_Binding "b2753040-881b-487a-92a1-9721da749be4" >> Port_Binding &quo

4 8

is there any feature of load balancing for engine??
by yam yam 10 Jan '20

10 Jan '20

Hello, given massive oVirt environment, I think single engine looks too small to deal with all workloads. so, I want to make active-active engine cluster for distributing workloads. is it possible for an oVirt environment to be made up of multiple engines & DBs for load balancing?

3 4

Re: is there any feature of load balancing for engine??
by Strahil 10 Jan '20

10 Jan '20

It is not a good idea. If you think that the engine can't cope with the load, add a separate engine for the specific cluster(s). What is the count of Datacenters, Clusters, Hosts, VMs & Storage domains ? Best Regards, Strahil NikolovOn Jan 8, 2020 03:51, yam yam <hyunooudy(a)gmail.com> wrote: > > Hello, > > given massive oVirt environment, I think single engine looks too small to deal with all workloads. > so, I want to make active-active engine cluster for distributing workloads. > is it possible for an oVirt environment to be made up of multiple engines & DBs for load balancing? > _______________________________________________ > Users mailing list -- users(a)ovirt.org > To unsubscribe send an email to users-leave(a)ovirt.org > Privacy Statement: https://www.ovirt.org/site/privacy-policy/ > oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ > List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XCO5DJS63YKC6…

2 1

Re: Setting up cockpit?
by Strahil 10 Jan '20

10 Jan '20

On Jan 9, 2020 18:58, m.skrzetuski(a)gmail.com wrote: > > Hello everyone, > > I'd like to get cockpit to work because currently when I click "Host Console" on a host I just get "connection refused". I checked and after the engine installation the cockpit service was not running. When I start it, it runs and answers on port 9090, however the SSL certificate is broken. > > - How do I auto enable cockpit on installation? systemctl enable --now cockpit.socket -H < destination server> > - How do I supply my own SSL certification to cockpit? Put them in /etc/cockpit/ws-certs.d/ Source:https://www.redhat.com/en/blog/linux-system-administration-managemen… > > Kind regards > Skrzetuski Best Regards, Strahil Nikolov

1 0

Re: After NAS crash, one VM will not start up, Cannot prepare illegal volume
by David Johnson 09 Jan '20

09 Jan '20

Thank you again. After updating legality to LEGAL, [root@mx-ovirt-host2 ~]# vdsm-client Volume getInfo storagepoolID=25cd9bfc-bab6-11e8-90f3-78acc0b47b4d storagedomainID=6e627364-5e0c-4250-ac95-7cd914d0175f imageID=4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6 volumeID=f8066c56-6db1-4605-8d7c-0739335d30b8 { "status": "OK", "lease": { "path": "/rhev/data-center/mnt/192.168.2.220: _mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease", "owners": [], "version": null, "offset": 0 }, "domain": "6e627364-5e0c-4250-ac95-7cd914d0175f", "capacity": "1503238553600", "voltype": "LEAF", "description": "", "parent": "a912e388-d80d-4f56-805b-ea5e2f35d741", "format": "COW", "generation": 0, "image": "4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6", "uuid": "f8066c56-6db1-4605-8d7c-0739335d30b8", "disktype": "DATA", "legality": "LEGAL", "mtime": "0", "apparentsize": "36440899584", "truesize": "16916186624", "type": "SPARSE", "children": [], "pool": "", "ctime": "1571669201" } Attempt to start the VM result are: Log excerpt: 2020-01-09 06:47:46,575-0600 INFO (vm/c5d0a42f) [storage.StorageDomain] Creating symlink from /rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6 to /var/run/vdsm/storage/6e627364-5e0c-4250-ac95-7cd914d0175f/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6 (fileSD:580) 2020-01-09 06:47:46,581-0600 INFO (vm/c5d0a42f) [vdsm.api] FINISH prepareImage return={'info': {'path': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', 'type': 'file'}, 'path': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', 'imgVolumesInfo': [{'domainID': '6e627364-5e0c-4250-ac95-7cd914d0175f', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/a912e388-d80d-4f56-805b-ea5e2f35d741', 'volumeID': u'a912e388-d80d-4f56-805b-ea5e2f35d741', 'leasePath': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/a912e388-d80d-4f56-805b-ea5e2f35d741.lease', 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6'}, {'domainID': '6e627364-5e0c-4250-ac95-7cd914d0175f', 'leaseOffset': 0, 'path': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', 'volumeID': u'f8066c56-6db1-4605-8d7c-0739335d30b8', 'leasePath': u'/rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6'}]} from=internal, task_id=865d2ff4-4e63-44dc-b8f8-9d93cad9892f (api:52) 2020-01-09 06:47:46,582-0600 INFO (vm/c5d0a42f) [vds] prepared volume path: /rhev/data-center/mnt/192.168.2.220:_mnt_ovirt-freenas/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8 (clientIF:497) 2020-01-09 06:47:46,583-0600 INFO (vm/c5d0a42f) [vdsm.api] START prepareImage(sdUUID='ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', spUUID='25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', imgUUID='60077050-6f99-41db-b280-446f018b104b', leafUUID='a67eb40c-e0a1-49cc-9179-bebb263d6e9c', allowIllegal=False) from=internal, task_id=08830292-0f75-4c5b-a411-695894c66475 (api:46) 2020-01-09 06:47:46,632-0600 INFO (vm/c5d0a42f) [vdsm.api] FINISH prepareImage error=Cannot prepare illegal volume: (u'a67eb40c-e0a1-49cc-9179-bebb263d6e9c',) from=internal, task_id=08830292-0f75-4c5b-a411-695894c66475 (api:50) 2020-01-09 06:47:46,632-0600 ERROR (vm/c5d0a42f) [storage.TaskManager.Task] (Task='08830292-0f75-4c5b-a411-695894c66475') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in prepareImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3187, in prepareImage raise se.prepareIllegalVolumeError(volUUID) prepareIllegalVolumeError: Cannot prepare illegal volume: (u'a67eb40c-e0a1-49cc-9179-bebb263d6e9c',) 2020-01-09 06:47:46,633-0600 INFO (vm/c5d0a42f) [storage.TaskManager.Task] (Task='08830292-0f75-4c5b-a411-695894c66475') aborting: Task is aborted: "Cannot prepare illegal volume: (u'a67eb40c-e0a1-49cc-9179-bebb263d6e9c',)" - code 227 (task:1181) 2020-01-09 06:47:46,634-0600 ERROR (vm/c5d0a42f) [storage.Dispatcher] FINISH prepareImage error=Cannot prepare illegal volume: (u'a67eb40c-e0a1-49cc-9179-bebb263d6e9c',) (dispatcher:82) 2020-01-09 06:47:46,634-0600 ERROR (vm/c5d0a42f) [virt.vm] (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') The vm start process failed (vm:949) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 878, in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2798, in _run self._devices = self._make_devices() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2639, in _make_devices disk_objs = self._perform_host_local_adjustment() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2712, in _perform_host_local_adjustment self._preparePathsForDrives(disk_params) File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1023, in _preparePathsForDrives drive['path'] = self.cif.prepareVolumePath(drive, self.id) File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 417, in prepareVolumePath raise vm.VolumeError(drive) VolumeError: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '3'}, 'serial': '60077050-6f99-41db-b280-446f018b104b', 'index': 1, 'iface': 'scsi', 'apparentsize': '16842752', 'specParams': {}, 'cache': 'writeback', 'imageID': '60077050-6f99-41db-b280-446f018b104b', 'truesize': '2840576', 'type': 'disk', 'domainID': 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', 'reqsize': '0', 'format': 'cow', 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/60077050-6f99-41db-b280-446f018b104b/a67eb40c-e0a1-49cc-9179-bebb263d6e9c', 'propagateErrors': 'off', 'name': 'sdb', 'volumeID': 'a67eb40c-e0a1-49cc-9179-bebb263d6e9c', 'diskType': 'file', 'alias': 'ua-60077050-6f99-41db-b280-446f018b104b', 'discard': False} 2020-01-09 06:47:46,634-0600 INFO (vm/c5d0a42f) [virt.vm] (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') Changed state to Down: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '3'}, 'serial': '60077050-6f99-41db-b280-446f018b104b', 'index': 1, 'iface': 'scsi', 'apparentsize': '16842752', 'specParams': {}, 'cache': 'writeback', 'imageID': '60077050-6f99-41db-b280-446f018b104b', 'truesize': '2840576', 'type': 'disk', 'domainID': 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', 'reqsize': '0', 'format': 'cow', 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/60077050-6f99-41db-b280-446f018b104b/a67eb40c-e0a1-49cc-9179-bebb263d6e9c', 'propagateErrors': 'off', 'name': 'sdb', 'volumeID': 'a67eb40c-e0a1-49cc-9179-bebb263d6e9c', 'diskType': 'file', 'alias': 'ua-60077050-6f99-41db-b280-446f018b104b', 'discard': False} (code=1) (vm:1689) 2 Regards, David Johnson Director of Development, Maxis Technology 844.696.2947 ext 702 (o) | 479.531.3590 (c) djohnson(a)maxistechnology.com [image: Maxis Techncology] <http://www.maxistechnology.com> www.maxistechnology.com *stay connected <http://www.linkedin.com/in/pojoguy>* On Thu, Jan 9, 2020 at 6:39 AM Benny Zlotnik <bzlotnik(a)redhat.com> wrote: > this `/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.meta` > > > You can see here[1] an example for updating volume metadata using > vdsm-client > > [1] https://bugzilla.redhat.com/show_bug.cgi?id=1409380#c3 > > On Thu, Jan 9, 2020 at 2:26 PM David Johnson <djohnson(a)maxistechnology.com> > wrote: > >> Thank you for the quick response. >> >> Where do I find that? >> >> Regards, >> David Johnson >> Director of Development, Maxis Technology >> 844.696.2947 ext 702 (o) | 479.531.3590 (c) >> djohnson(a)maxistechnology.com >> >> >> [image: Maxis Techncology] <http://www.maxistechnology.com> >> www.maxistechnology.com >> >> >> *stay connected <http://www.linkedin.com/in/pojoguy>* >> >> >> On Thu, Jan 9, 2020 at 6:24 AM Benny Zlotnik <bzlotnik(a)redhat.com> wrote: >> >>> Did you change the volume metadata to LEGAL on the storage as well? >>> >>> >>> On Thu, Jan 9, 2020 at 2:19 PM David Johnson < >>> djohnson(a)maxistechnology.com> wrote: >>> >>>> We had a drive in our NAS fail, but afterwards one of our VM's will not >>>> start. >>>> >>>> The boot drive on the VM is (so near as I can tell) the only drive >>>> affected. >>>> >>>> I confirmed that the disk images (active and snapshot) are both valid >>>> with qemu. >>>> >>>> I followed the instructions at >>>> https://www.canarytek.com/2017/07/02/Recover_oVirt_Illegal_Snapshots.html to >>>> identify the snapshot images that were marked "invalid" and marked them as >>>> valid. >>>> >>>> update images set imagestatus=1 where imagestatus=4; >>>> >>>> >>>> >>>> Log excerpt from attempt to start VM: >>>> 2020-01-09 02:18:44,908-0600 INFO (vm/c5d0a42f) [vdsm.api] START >>>> prepareImage(sdUUID='6e627364-5e0c-4250-ac95-7cd914d0175f', >>>> spUUID='25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', >>>> imgUUID='4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', >>>> leafUUID='f8066c56-6db1-4605-8d7c-0739335d30b8', allowIllegal=False) >>>> from=internal, task_id=26053225-6569-4b73-abdd-7d6c7e15d1e9 (api:46) >>>> 2020-01-09 02:18:44,931-0600 INFO (vm/c5d0a42f) [vdsm.api] FINISH >>>> prepareImage error=Cannot prepare illegal volume: >>>> (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) from=internal, >>>> task_id=26053225-6569-4b73-abdd-7d6c7e15d1e9 (api:50) >>>> 2020-01-09 02:18:44,932-0600 ERROR (vm/c5d0a42f) >>>> [storage.TaskManager.Task] (Task='26053225-6569-4b73-abdd-7d6c7e15d1e9') >>>> Unexpected error (task:875) >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line >>>> 882, in _run >>>> return fn(*args, **kargs) >>>> File "<string>", line 2, in prepareImage >>>> File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, >>>> in method >>>> ret = func(*args, **kwargs) >>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line >>>> 3187, in prepareImage >>>> raise se.prepareIllegalVolumeError(volUUID) >>>> prepareIllegalVolumeError: Cannot prepare illegal volume: >>>> (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) >>>> 2020-01-09 02:18:44,932-0600 INFO (vm/c5d0a42f) >>>> [storage.TaskManager.Task] (Task='26053225-6569-4b73-abdd-7d6c7e15d1e9') >>>> aborting: Task is aborted: "Cannot prepare illegal volume: >>>> (u'f8066c56-6db1-4605-8d7c-0739335d30b8',)" - code 227 (task:1181) >>>> 2020-01-09 02:18:44,933-0600 ERROR (vm/c5d0a42f) [storage.Dispatcher] >>>> FINISH prepareImage error=Cannot prepare illegal volume: >>>> (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) (dispatcher:82) >>>> 2020-01-09 02:18:44,933-0600 ERROR (vm/c5d0a42f) [virt.vm] >>>> (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') The vm start process failed >>>> (vm:949) >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 878, in >>>> _startUnderlyingVm >>>> self._run() >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2798, >>>> in _run >>>> self._devices = self._make_devices() >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2639, >>>> in _make_devices >>>> disk_objs = self._perform_host_local_adjustment() >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2712, >>>> in _perform_host_local_adjustment >>>> self._preparePathsForDrives(disk_params) >>>> File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1023, >>>> in _preparePathsForDrives >>>> drive['path'] = self.cif.prepareVolumePath(drive, self.id) >>>> File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 417, >>>> in prepareVolumePath >>>> raise vm.VolumeError(drive) >>>> VolumeError: Bad volume specification {'address': {'bus': '0', >>>> 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'serial': >>>> '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'index': 0, 'iface': 'scsi', >>>> 'apparentsize': '36440899584', 'specParams': {}, 'cache': 'writeback', >>>> 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'truesize': >>>> '16916186624', 'type': 'disk', 'domainID': >>>> '6e627364-5e0c-4250-ac95-7cd914d0175f', 'reqsize': '0', 'format': 'cow', >>>> 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': >>>> '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': >>>> 'f8066c56-6db1-4605-8d7c-0739335d30b8', 'diskType': 'file', 'alias': >>>> 'ua-4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'discard': False} >>>> 2020-01-09 02:18:44,934-0600 INFO (vm/c5d0a42f) [virt.vm] >>>> (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') Changed state to Down: Bad >>>> volume specification {'address': {'bus': '0', 'controller': '0', 'type': >>>> 'drive', 'target': '0', 'unit': '0'}, 'serial': >>>> '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'index': 0, 'iface': 'scsi', >>>> 'apparentsize': '36440899584', 'specParams': {}, 'cache': 'writeback', >>>> 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'truesize': >>>> '16916186624', 'type': 'disk', 'domainID': >>>> '6e627364-5e0c-4250-ac95-7cd914d0175f', 'reqsize': '0', 'format': 'cow', >>>> 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': >>>> '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': >>>> 'f8066c56-6db1-4605-8d7c-0739335d30b8', 'diskType': 'file', 'alias': >>>> 'ua-4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'discard': False} (code=1) >>>> (vm:1689) >>>> 2020-01-09 02:18:44,963-0600 INFO (vm/c5d0a42f) [virt.vm] >>>> (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') Stopping connection >>>> (guestagent:442) >>>> 2020-01-09 02:18:44,978-0600 INFO (jsonrpc/4) [api.virt] START >>>> destroy(gracefulAttempts=1) from=::ffff:192.168.2.15,57858, >>>> flow_id=4b57f1a6, vmId=c5d0a42f-3b1e-43ee-a567-7844654011f5 (api:46) >>>> 2 >>>> >>>> >>>> Log excerpt from attempt to migrate the affected drive to another >>>> storage device: >>>> 2020-01-09 05:27:14,362-0600 INFO (jsonrpc/2) [storage.VolumeManifest] >>>> ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8 >>>> info is {'status': 'OK', 'domain': 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', >>>> 'voltype': 'LEAF', 'description': 'None', 'parent': >>>> 'a912e388-d80d-4f56-805b-ea5e2f35d741', 'format': 'COW', 'generation': 0, >>>> 'image': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'ctime': '1578559013', >>>> 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': >>>> '219008', 'children': [], 'pool': '', 'capacity': '1503238553600', 'uuid': >>>> u'f8066c56-6db1-4605-8d7c-0739335d30b8', 'truesize': '26624', 'type': >>>> 'SPARSE', 'lease': {'path': u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> 'owners': [], 'version': None, 'offset': 0}} (volume:282) >>>> 2020-01-09 05:27:14,362-0600 INFO (jsonrpc/2) [vdsm.api] FINISH >>>> getVolumeInfo return={'info': {'status': 'OK', 'domain': >>>> 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', 'voltype': 'LEAF', 'description': >>>> 'None', 'parent': 'a912e388-d80d-4f56-805b-ea5e2f35d741', 'format': 'COW', >>>> 'generation': 0, 'image': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'ctime': >>>> '1578559013', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', >>>> 'apparentsize': '219008', 'children': [], 'pool': '', 'capacity': >>>> '1503238553600', 'uuid': u'f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> 'truesize': '26624', 'type': 'SPARSE', 'lease': {'path': >>>> u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> 'owners': [], 'version': None, 'offset': 0}}} >>>> from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, >>>> task_id=28e885f7-df18-4ea5-bb6e-eaee7385b519 (api:52) >>>> 2020-01-09 05:27:14,363-0600 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] >>>> RPC call Volume.getInfo succeeded in 0.04 seconds (__init__:573) >>>> 2020-01-09 05:27:14,394-0600 INFO (jsonrpc/6) [vdsm.api] START >>>> sdm_copy_data(job_id=u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', >>>> source={u'img_id': u'4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', u'sd_id': >>>> u'6e627364-5e0c-4250-ac95-7cd914d0175f', u'endpoint_type': u'div', >>>> u'vol_id': u'f8066c56-6db1-4605-8d7c-0739335d30b8'}, >>>> destination={u'generation': 0, u'img_id': >>>> u'4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', u'sd_id': >>>> u'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', u'endpoint_type': u'div', >>>> u'vol_id': u'f8066c56-6db1-4605-8d7c-0739335d30b8'}) >>>> from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, >>>> task_id=f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (api:46) >>>> 2020-01-09 05:27:14,395-0600 INFO (jsonrpc/6) [vdsm.api] FINISH >>>> sdm_copy_data return=None from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, >>>> task_id=f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (api:52) >>>> 2020-01-09 05:27:14,395-0600 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] >>>> RPC call SDM.copy_data succeeded in 0.00 seconds (__init__:573) >>>> 2020-01-09 05:27:14,396-0600 INFO (tasks/4) >>>> [storage.ThreadPool.WorkerThread] START task >>>> f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (cmd=<bound method Task.commit of >>>> <vdsm.storage.task.Task instance at 0x7ff3a42613b0>>, args=None) >>>> (threadPool:208) >>>> 2020-01-09 05:27:14,396-0600 INFO (tasks/4) [root] Running job >>>> u'9a47a14f-d398-4fe8-81fa-59cba7122aa0'... (jobs:183) >>>> 2020-01-09 05:27:14,429-0600 INFO (jsonrpc/1) [api.host] START >>>> getJobs(job_type=u'storage', >>>> job_ids=[u'9a47a14f-d398-4fe8-81fa-59cba7122aa0']) >>>> from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:46) >>>> 2020-01-09 05:27:14,430-0600 INFO (jsonrpc/1) [api.host] FINISH >>>> getJobs return={'status': {'message': 'Done', 'code': 0}, 'jobs': >>>> {u'9a47a14f-d398-4fe8-81fa-59cba7122aa0': {'status': 'running', 'job_type': >>>> 'storage', 'id': u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', 'description': >>>> 'copy_data'}}} from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:52) >>>> 2020-01-09 05:27:14,430-0600 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] >>>> RPC call Host.getJobs succeeded in 0.00 seconds (__init__:573) >>>> 2020-01-09 05:27:14,459-0600 INFO (tasks/4) [storage.SANLock] >>>> Acquiring Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> offset=0) for host id 2 (clusterlock:386) >>>> 2020-01-09 05:27:14,496-0600 INFO (tasks/4) [storage.SANLock] >>>> Successfully acquired Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> offset=0) for host id 2 (clusterlock:424) >>>> 2020-01-09 05:27:14,498-0600 INFO (tasks/4) [storage.VolumeManifest] >>>> Volume: preparing volume >>>> 6e627364-5e0c-4250-ac95-7cd914d0175f/f8066c56-6db1-4605-8d7c-0739335d30b8 >>>> (volume:569) >>>> 2020-01-09 05:27:14,502-0600 INFO (tasks/4) [storage.SANLock] >>>> Releasing Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> offset=0) (clusterlock:487) >>>> 2020-01-09 05:27:14,504-0600 INFO (tasks/4) [storage.SANLock] >>>> Successfully released Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', >>>> path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', >>>> offset=0) (clusterlock:496) >>>> 2020-01-09 05:27:14,504-0600 ERROR (tasks/4) [root] Job >>>> u'9a47a14f-d398-4fe8-81fa-59cba7122aa0' failed (jobs:221) >>>> Traceback (most recent call last): >>>> File "/usr/lib/python2.7/site-packages/vdsm/jobs.py", line 157, in run >>>> self._run() >>>> File >>>> "/usr/lib/python2.7/site-packages/vdsm/storage/sdm/api/copy_data.py", line >>>> 61, in _run >>>> with self._source.prepare(), self._dest.prepare(): >>>> File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ >>>> return self.gen.next() >>>> File >>>> "/usr/lib/python2.7/site-packages/vdsm/storage/sdm/api/copy_data.py", line >>>> 172, in prepare >>>> self.volume.prepare(rw=self._writable, justme=False) >>>> File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line >>>> 574, in prepare >>>> raise se.prepareIllegalVolumeError(self.volUUID) >>>> prepareIllegalVolumeError: Cannot prepare illegal volume: >>>> ('f8066c56-6db1-4605-8d7c-0739335d30b8',) >>>> 2020-01-09 05:27:14,505-0600 INFO (tasks/4) [root] Job >>>> u'9a47a14f-d398-4fe8-81fa-59cba7122aa0' will be deleted in 3600 seconds >>>> (jobs:249) >>>> 2020-01-09 05:27:14,506-0600 INFO (tasks/4) >>>> [storage.ThreadPool.WorkerThread] FINISH task >>>> f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (threadPool:210) >>>> 2020-01-09 05:27:16,459-0600 INFO (jsonrpc/4) [api.host] START >>>> getJobs(job_type=u'storage', >>>> job_ids=[u'9a47a14f-d398-4fe8-81fa-59cba7122aa0']) >>>> from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:46) >>>> 2020-01-09 05:27:16,460-0600 INFO (jsonrpc/4) [api.host] FINISH >>>> getJobs return={'status': {'message': 'Done', 'code': 0}, 'jobs': >>>> {u'9a47a14f-d398-4fe8-81fa-59cba7122aa0': {'status': 'failed', 'error': >>>> {'message': "Cannot prepare illegal volume: >>>> ('f8066c56-6db1-4605-8d7c-0739335d30b8',)", 'code': 227}, 'job_type': >>>> 'storage', 'id': u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', 'description': >>>> 'copy_data'}}} from=::ffff:192.168.2.15,57858, >>>> flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:52) >>>> 2 >>>> >>>> Regards, >>>> David Johnson >>>> Director of Development, Maxis Technology >>>> 844.696.2947 ext 702 (o) | 479.531.3590 (c) >>>> djohnson(a)maxistechnology.com >>>> >>>> >>>> [image: Maxis Techncology] <http://www.maxistechnology.com> >>>> www.maxistechnology.com >>>> >>>> >>>> *stay connected <http://www.linkedin.com/in/pojoguy>* >>>> _______________________________________________ >>>> Users mailing list -- users(a)ovirt.org >>>> To unsubscribe send an email to users-leave(a)ovirt.org >>>> Privacy Statement: https://www.ovirt.org/site/privacy-policy/ >>>> oVirt Code of Conduct: >>>> https://www.ovirt.org/community/about/community-guidelines/ >>>> List Archives: >>>> https://lists.ovirt.org/archives/list/users@ovirt.org/message/KKLZKSPCESYVJ… >>>> >>>

1 2

AWX and error using ovirt as an inventory source
by Gianluca Cecchi 09 Jan '20

09 Jan '20

Hello, I have awx 9.0.1 and ansible 2.8.5 in container of a CentOS 7.7 server. I'm trying to use oVirt 4.3.6.7-1.el7 as a source of an inventory in awx but I get error when syncing Find at bottom below the error messages. I see that in recent past (around June this year) there were some problems, but they should be solved now, correct? There was also a problem in syncing when some powered off VMs were present in oVirt env, but I think this solved too, correct? Any way to replicate / test from command line of awx container? I try some things but in command line I always get error regarding oVirt inventory script requires ovirt-engine-sdk-python >= 4.0.0 that I think depends on not using correct command line and/or not setting needed env. Thanks in advance, Gianluca 2.536 INFO Updating inventory 4: MYDC_OVIRT 3.011 INFO Reading Ansible inventory source: /var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/plugins/inventory/ovirt4.py 3.013 INFO Using VIRTUAL_ENV: /var/lib/awx/venv/ansible 3.013 INFO Using PATH: /var/lib/awx/venv/ansible/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/var/lib/awx/venv/awx/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin 3.013 INFO Using PYTHONPATH: /var/lib/awx/venv/ansible/lib/python3.6/site-packages: Traceback (most recent call last): File "/var/lib/awx/venv/awx/bin/awx-manage", line 11, in <module> load_entry_point('awx==9.0.1.0', 'console_scripts', 'awx-manage')() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/__init__.py", line 158, in manage execute_from_command_line(sys.argv) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line utility.execute() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/__init__.py", line 375, in execute self.fetch_command(subcommand).run_from_argv(self.argv) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 323, in run_from_argv self.execute(*args, **cmd_options) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/django/core/management/base.py", line 364, in execute output = self.handle(*args, **options) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1153, in handle raise exc File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 1043, in handle venv_path=venv_path, verbosity=self.verbosity).load() File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 214, in load return self.command_to_json(base_args + ['--list']) File "/var/lib/awx/venv/awx/lib64/python3.6/site-packages/awx/main/management/commands/inventory_import.py", line 197, in command_to_json self.method, proc.returncode, stdout, stderr)) RuntimeError: ansible-inventory failed (rc=1) with stdout: stderr: ansible-inventory 2.8.5 config file = /etc/ansible/ansible.cfg configured module search path = ['/var/lib/awx/.ansible/plugins/modules', '/usr/share/ansible/plugins/modules'] ansible python module location = /usr/lib/python3.6/site-packages/ansible executable location = /usr/bin/ansible-inventory python version = 3.6.8 (default, Oct 7 2019, 17:58:22) [GCC 8.2.1 20180905 (Red Hat 8.2.1-3)] Using /etc/ansible/ansible.cfg as config file [WARNING]: * Failed to parse /var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py with script plugin: Inventory script (/var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py) had an execution error: File "/usr/lib/python3.6/site-packages/ansible/inventory/manager.py", line 268, in parse_source plugin.parse(self._inventory, self._loader, source, cache=cache) File "/usr/lib/python3.6/site-packages/ansible/plugins/inventory/script.py", line 161, in parse raise AnsibleParserError(to_native(e)) [WARNING]: Unable to parse /var/lib/awx/venv/awx/lib64/python3.6/site- packages/awx/plugins/inventory/ovirt4.py as an inventory source ERROR! No inventory was parsed, please check your configuration and options.

3 23

After NAS crash, one VM will not start up, Cannot prepare illegal volume
by David Johnson 09 Jan '20

09 Jan '20

We had a drive in our NAS fail, but afterwards one of our VM's will not start. The boot drive on the VM is (so near as I can tell) the only drive affected. I confirmed that the disk images (active and snapshot) are both valid with qemu. I followed the instructions at https://www.canarytek.com/2017/07/02/Recover_oVirt_Illegal_Snapshots.html to identify the snapshot images that were marked "invalid" and marked them as valid. update images set imagestatus=1 where imagestatus=4; Log excerpt from attempt to start VM: 2020-01-09 02:18:44,908-0600 INFO (vm/c5d0a42f) [vdsm.api] START prepareImage(sdUUID='6e627364-5e0c-4250-ac95-7cd914d0175f', spUUID='25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', imgUUID='4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', leafUUID='f8066c56-6db1-4605-8d7c-0739335d30b8', allowIllegal=False) from=internal, task_id=26053225-6569-4b73-abdd-7d6c7e15d1e9 (api:46) 2020-01-09 02:18:44,931-0600 INFO (vm/c5d0a42f) [vdsm.api] FINISH prepareImage error=Cannot prepare illegal volume: (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) from=internal, task_id=26053225-6569-4b73-abdd-7d6c7e15d1e9 (api:50) 2020-01-09 02:18:44,932-0600 ERROR (vm/c5d0a42f) [storage.TaskManager.Task] (Task='26053225-6569-4b73-abdd-7d6c7e15d1e9') Unexpected error (task:875) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/storage/task.py", line 882, in _run return fn(*args, **kargs) File "<string>", line 2, in prepareImage File "/usr/lib/python2.7/site-packages/vdsm/common/api.py", line 48, in method ret = func(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/storage/hsm.py", line 3187, in prepareImage raise se.prepareIllegalVolumeError(volUUID) prepareIllegalVolumeError: Cannot prepare illegal volume: (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) 2020-01-09 02:18:44,932-0600 INFO (vm/c5d0a42f) [storage.TaskManager.Task] (Task='26053225-6569-4b73-abdd-7d6c7e15d1e9') aborting: Task is aborted: "Cannot prepare illegal volume: (u'f8066c56-6db1-4605-8d7c-0739335d30b8',)" - code 227 (task:1181) 2020-01-09 02:18:44,933-0600 ERROR (vm/c5d0a42f) [storage.Dispatcher] FINISH prepareImage error=Cannot prepare illegal volume: (u'f8066c56-6db1-4605-8d7c-0739335d30b8',) (dispatcher:82) 2020-01-09 02:18:44,933-0600 ERROR (vm/c5d0a42f) [virt.vm] (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') The vm start process failed (vm:949) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 878, in _startUnderlyingVm self._run() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2798, in _run self._devices = self._make_devices() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2639, in _make_devices disk_objs = self._perform_host_local_adjustment() File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 2712, in _perform_host_local_adjustment self._preparePathsForDrives(disk_params) File "/usr/lib/python2.7/site-packages/vdsm/virt/vm.py", line 1023, in _preparePathsForDrives drive['path'] = self.cif.prepareVolumePath(drive, self.id) File "/usr/lib/python2.7/site-packages/vdsm/clientIF.py", line 417, in prepareVolumePath raise vm.VolumeError(drive) VolumeError: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'serial': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'index': 0, 'iface': 'scsi', 'apparentsize': '36440899584', 'specParams': {}, 'cache': 'writeback', 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'truesize': '16916186624', 'type': 'disk', 'domainID': '6e627364-5e0c-4250-ac95-7cd914d0175f', 'reqsize': '0', 'format': 'cow', 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': 'f8066c56-6db1-4605-8d7c-0739335d30b8', 'diskType': 'file', 'alias': 'ua-4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'discard': False} 2020-01-09 02:18:44,934-0600 INFO (vm/c5d0a42f) [virt.vm] (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') Changed state to Down: Bad volume specification {'address': {'bus': '0', 'controller': '0', 'type': 'drive', 'target': '0', 'unit': '0'}, 'serial': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'index': 0, 'iface': 'scsi', 'apparentsize': '36440899584', 'specParams': {}, 'cache': 'writeback', 'imageID': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'truesize': '16916186624', 'type': 'disk', 'domainID': '6e627364-5e0c-4250-ac95-7cd914d0175f', 'reqsize': '0', 'format': 'cow', 'poolID': '25cd9bfc-bab6-11e8-90f3-78acc0b47b4d', 'device': 'disk', 'path': '/rhev/data-center/25cd9bfc-bab6-11e8-90f3-78acc0b47b4d/6e627364-5e0c-4250-ac95-7cd914d0175f/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8', 'propagateErrors': 'off', 'name': 'sda', 'bootOrder': '1', 'volumeID': 'f8066c56-6db1-4605-8d7c-0739335d30b8', 'diskType': 'file', 'alias': 'ua-4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'discard': False} (code=1) (vm:1689) 2020-01-09 02:18:44,963-0600 INFO (vm/c5d0a42f) [virt.vm] (vmId='c5d0a42f-3b1e-43ee-a567-7844654011f5') Stopping connection (guestagent:442) 2020-01-09 02:18:44,978-0600 INFO (jsonrpc/4) [api.virt] START destroy(gracefulAttempts=1) from=::ffff:192.168.2.15,57858, flow_id=4b57f1a6, vmId=c5d0a42f-3b1e-43ee-a567-7844654011f5 (api:46) 2 Log excerpt from attempt to migrate the affected drive to another storage device: 2020-01-09 05:27:14,362-0600 INFO (jsonrpc/2) [storage.VolumeManifest] ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8 info is {'status': 'OK', 'domain': 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', 'voltype': 'LEAF', 'description': 'None', 'parent': 'a912e388-d80d-4f56-805b-ea5e2f35d741', 'format': 'COW', 'generation': 0, 'image': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'ctime': '1578559013', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '219008', 'children': [], 'pool': '', 'capacity': '1503238553600', 'uuid': u'f8066c56-6db1-4605-8d7c-0739335d30b8', 'truesize': '26624', 'type': 'SPARSE', 'lease': {'path': u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', 'owners': [], 'version': None, 'offset': 0}} (volume:282) 2020-01-09 05:27:14,362-0600 INFO (jsonrpc/2) [vdsm.api] FINISH getVolumeInfo return={'info': {'status': 'OK', 'domain': 'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', 'voltype': 'LEAF', 'description': 'None', 'parent': 'a912e388-d80d-4f56-805b-ea5e2f35d741', 'format': 'COW', 'generation': 0, 'image': '4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', 'ctime': '1578559013', 'disktype': 'DATA', 'legality': 'LEGAL', 'mtime': '0', 'apparentsize': '219008', 'children': [], 'pool': '', 'capacity': '1503238553600', 'uuid': u'f8066c56-6db1-4605-8d7c-0739335d30b8', 'truesize': '26624', 'type': 'SPARSE', 'lease': {'path': u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', 'owners': [], 'version': None, 'offset': 0}}} from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, task_id=28e885f7-df18-4ea5-bb6e-eaee7385b519 (api:52) 2020-01-09 05:27:14,363-0600 INFO (jsonrpc/2) [jsonrpc.JsonRpcServer] RPC call Volume.getInfo succeeded in 0.04 seconds (__init__:573) 2020-01-09 05:27:14,394-0600 INFO (jsonrpc/6) [vdsm.api] START sdm_copy_data(job_id=u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', source={u'img_id': u'4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', u'sd_id': u'6e627364-5e0c-4250-ac95-7cd914d0175f', u'endpoint_type': u'div', u'vol_id': u'f8066c56-6db1-4605-8d7c-0739335d30b8'}, destination={u'generation': 0, u'img_id': u'4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6', u'sd_id': u'ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7', u'endpoint_type': u'div', u'vol_id': u'f8066c56-6db1-4605-8d7c-0739335d30b8'}) from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, task_id=f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (api:46) 2020-01-09 05:27:14,395-0600 INFO (jsonrpc/6) [vdsm.api] FINISH sdm_copy_data return=None from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731, task_id=f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (api:52) 2020-01-09 05:27:14,395-0600 INFO (jsonrpc/6) [jsonrpc.JsonRpcServer] RPC call SDM.copy_data succeeded in 0.00 seconds (__init__:573) 2020-01-09 05:27:14,396-0600 INFO (tasks/4) [storage.ThreadPool.WorkerThread] START task f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (cmd=<bound method Task.commit of <vdsm.storage.task.Task instance at 0x7ff3a42613b0>>, args=None) (threadPool:208) 2020-01-09 05:27:14,396-0600 INFO (tasks/4) [root] Running job u'9a47a14f-d398-4fe8-81fa-59cba7122aa0'... (jobs:183) 2020-01-09 05:27:14,429-0600 INFO (jsonrpc/1) [api.host] START getJobs(job_type=u'storage', job_ids=[u'9a47a14f-d398-4fe8-81fa-59cba7122aa0']) from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:46) 2020-01-09 05:27:14,430-0600 INFO (jsonrpc/1) [api.host] FINISH getJobs return={'status': {'message': 'Done', 'code': 0}, 'jobs': {u'9a47a14f-d398-4fe8-81fa-59cba7122aa0': {'status': 'running', 'job_type': 'storage', 'id': u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', 'description': 'copy_data'}}} from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:52) 2020-01-09 05:27:14,430-0600 INFO (jsonrpc/1) [jsonrpc.JsonRpcServer] RPC call Host.getJobs succeeded in 0.00 seconds (__init__:573) 2020-01-09 05:27:14,459-0600 INFO (tasks/4) [storage.SANLock] Acquiring Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', offset=0) for host id 2 (clusterlock:386) 2020-01-09 05:27:14,496-0600 INFO (tasks/4) [storage.SANLock] Successfully acquired Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', offset=0) for host id 2 (clusterlock:424) 2020-01-09 05:27:14,498-0600 INFO (tasks/4) [storage.VolumeManifest] Volume: preparing volume 6e627364-5e0c-4250-ac95-7cd914d0175f/f8066c56-6db1-4605-8d7c-0739335d30b8 (volume:569) 2020-01-09 05:27:14,502-0600 INFO (tasks/4) [storage.SANLock] Releasing Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', offset=0) (clusterlock:487) 2020-01-09 05:27:14,504-0600 INFO (tasks/4) [storage.SANLock] Successfully released Lease(name='f8066c56-6db1-4605-8d7c-0739335d30b8', path=u'/rhev/data-center/mnt/192.168.2.223:_mnt_ovirt-main-pool/ec6ccb14-03c2-49cc-9cc0-b1a87d582ed7/images/4081ce8f-1ce1-4ee1-aa43-69af2dfc5ab6/f8066c56-6db1-4605-8d7c-0739335d30b8.lease', offset=0) (clusterlock:496) 2020-01-09 05:27:14,504-0600 ERROR (tasks/4) [root] Job u'9a47a14f-d398-4fe8-81fa-59cba7122aa0' failed (jobs:221) Traceback (most recent call last): File "/usr/lib/python2.7/site-packages/vdsm/jobs.py", line 157, in run self._run() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdm/api/copy_data.py", line 61, in _run with self._source.prepare(), self._dest.prepare(): File "/usr/lib64/python2.7/contextlib.py", line 17, in __enter__ return self.gen.next() File "/usr/lib/python2.7/site-packages/vdsm/storage/sdm/api/copy_data.py", line 172, in prepare self.volume.prepare(rw=self._writable, justme=False) File "/usr/lib/python2.7/site-packages/vdsm/storage/volume.py", line 574, in prepare raise se.prepareIllegalVolumeError(self.volUUID) prepareIllegalVolumeError: Cannot prepare illegal volume: ('f8066c56-6db1-4605-8d7c-0739335d30b8',) 2020-01-09 05:27:14,505-0600 INFO (tasks/4) [root] Job u'9a47a14f-d398-4fe8-81fa-59cba7122aa0' will be deleted in 3600 seconds (jobs:249) 2020-01-09 05:27:14,506-0600 INFO (tasks/4) [storage.ThreadPool.WorkerThread] FINISH task f1504b2d-c6c2-4866-97ce-e0476b0ba9a2 (threadPool:210) 2020-01-09 05:27:16,459-0600 INFO (jsonrpc/4) [api.host] START getJobs(job_type=u'storage', job_ids=[u'9a47a14f-d398-4fe8-81fa-59cba7122aa0']) from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:46) 2020-01-09 05:27:16,460-0600 INFO (jsonrpc/4) [api.host] FINISH getJobs return={'status': {'message': 'Done', 'code': 0}, 'jobs': {u'9a47a14f-d398-4fe8-81fa-59cba7122aa0': {'status': 'failed', 'error': {'message': "Cannot prepare illegal volume: ('f8066c56-6db1-4605-8d7c-0739335d30b8',)", 'code': 227}, 'job_type': 'storage', 'id': u'9a47a14f-d398-4fe8-81fa-59cba7122aa0', 'description': 'copy_data'}}} from=::ffff:192.168.2.15,57858, flow_id=94ad3bf0-c5f5-4d45-af2a-49aa4eaa6731 (api:52) 2 Regards, David Johnson Director of Development, Maxis Technology 844.696.2947 ext 702 (o) | 479.531.3590 (c) djohnson(a)maxistechnology.com [image: Maxis Techncology] <http://www.maxistechnology.com> www.maxistechnology.com *stay connected <http://www.linkedin.com/in/pojoguy>*

2 3