<div dir="ltr"><br><div class="gmail_extra"><br clear="all"><div><div dir="ltr"><span style="font-family:arial,sans-serif;font-size:16px"><strong>Steve </strong></span></div></div><br><div class="gmail_quote">On Wed, Apr 23, 2014 at 5:14 AM, Dafna Ron <span dir="ltr"><<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>></span> wrote:<br>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">steve,<br>
I did not say that there is a limit. there is no limit and you can take a 1000 snapshots if you like, I simply said that I think that it would not be would a good practice to do so.<br></blockquote><div><br></div><div>I'm not trying to be adversarial here, but this is contradictory; if there's 'no limit' but 'its not good practice' and we assume that we want our virtual infrastructure to run smoothly, then effectively there is a limit we just don't know what it is.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
I also did not say that this is your current problem with the vm so you are jumping to conclusions here.<br></blockquote><div><br></div><div>I wasn't connecting the dots between # of snapshots, and the current issue, I have other VM's with the same amount of snapshots without this problem. No conclusion jumping going on. More interested in what the best practice is for VM's that accumulate snapshots over time.</div>
<div><br></div><div>There is a feature slated for 3.5 <a href="http://www.ovirt.org/Features/Live_Merge">http://www.ovirt.org/Features/Live_Merge</a> which merges snapshots on a running VM, so I suppose in the long run I won't have a high snapshot count.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
i simply explained how snapshots work which is that they are created in a chain, if there is a problem at a single point in time it would effect the rest of the snapshots below it.<br></blockquote><div><br></div><div>Just for clarity, such a problem would affect the snapshots 'below it' means after the problematic snapshot? Example: Snapshot 1,2,3,4,5. #4 has a consistency issue, snaps 1,2,3 should be ok? I can try incrementally rolling back snapshots if this is the case (after vdsm restart suggested).</div>
<div><br></div><div>Is there any way to do a consistency check? I can imagine scheduling a cronjob to run through a nightly check for consistency issues, then roll back to an earlier snapshot to circumvent the issue.</div>
<div> </div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
And that we query all images under the base Image so if you have a lot of them it would take a long time for the results to come back.<br></blockquote><div><br></div><div>That's good to know, is this query done on new snapshot creation only? So over time the more snapshots I have, new snapshots will take longer to complete?</div>
<div><br></div>
<blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex">
<br>
as for your vm, since you fail to create a snapshot on only that vm it means that there is a problem in the current vm and it's chain.<br>
<br>
I can see when comparing the uuid's that the pool, domain, base image and last snapshots all exists in the rhev link.<br>
<br>
2014-04-22 12:13:41,083 INFO [org.ovirt.engine.core.<u></u>vdsbroker.irsbroker.<u></u>CreateSnapshotVDSCommand] (pool-6-thread-49) [7ccaed5] -- createVolume parameters:<br>
sdUUID=95b9d922-4df7-4d3b-<u></u>9bca-467e2fd9d573<br>
spUUID=9497ef2c-8368-4c92-<u></u>8d61-7f318a90748f<br>
imgGUID=466d9ae9-e46a-46f8-<u></u>9f4b-964d8af0675b<br>
size=21,474,836,480 bytes<br>
volFormat=COW<br>
volType=Sparse<br>
volUUID=0b2d15e5-bf4f-4eaf-<u></u>90e2-f1bd51a3a936<br>
descr=<br>
srcImgGUID=466d9ae9-e46a-46f8-<u></u>9f4b-964d8af0675b<br>
srcVolUUID=1a67de4b-aa1c-4436-<u></u>baca-ca55726d54d7<br>
<br>
<br>
<br>
lets see if it's possibly a cache issue - can you please restart vdsm on the hosts?</blockquote><div><br></div><div>I'll update when I have a chance to restart the services.</div><div><br></div><div>Thanks</div><div>
<br></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div><br>
<br>
<br>
<br>
<br>
<br>
<br>
On 04/22/2014 08:22 PM, Steve Dainard wrote:<br>
</div></div><blockquote class="gmail_quote" style="margin:0px 0px 0px 0.8ex;border-left-width:1px;border-left-color:rgb(204,204,204);border-left-style:solid;padding-left:1ex"><div><div>
All snapshots are from before failure.<br>
<br>
That's a bit scary that there may be a 'too many snapshots' issue. I take snapshots for point in time consistency, and without the ability to collapse them while the vm is running I'm not sure what the best option is here. What is the recommended snapshot limit? Or maybe a better question; whats the intended use case for snapshots in ovirt?<br>
<br>
Export domain is currently unavailable, and without it active I can't disable it properly.<br>
<br>
# ls -tl /rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b<br>
total 8517740<br>
-rw-rw----. 1 vdsm kvm 97583104 Apr 22 14:03 1a67de4b-aa1c-4436-baca-<u></u>ca55726d54d7<br>
-rw-r--r--. 1 vdsm kvm 268 Apr 22 12:13 1a67de4b-aa1c-4436-baca-<u></u>ca55726d54d7.meta<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 22 01:06 87390b64-becd-4a6f-a4fc-<u></u>d27655f59b64.meta<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 22 01:04 1a67de4b-aa1c-4436-baca-<u></u>ca55726d54d7.lease<br>
-rw-rw----. 1 vdsm kvm 107413504 Apr 20 22:00 87390b64-becd-4a6f-a4fc-<u></u>d27655f59b64<br>
-rw-rw----. 1 vdsm kvm 104267776 Apr 19 22:00 6f9fd451-6c82-4390-802c-<u></u>9e23a7d89427<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 19 22:00 87390b64-becd-4a6f-a4fc-<u></u>d27655f59b64.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 19 22:00 6f9fd451-6c82-4390-802c-<u></u>9e23a7d89427.meta<br>
-rw-rw----. 1 vdsm kvm 118358016 Apr 18 22:00 c298ce3b-ec6a-4526-9971-<u></u>a769f4d3d69b<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 18 22:00 6f9fd451-6c82-4390-802c-<u></u>9e23a7d89427.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 18 22:00 c298ce3b-ec6a-4526-9971-<u></u>a769f4d3d69b.meta<br>
-rw-rw----. 1 vdsm kvm 120913920 Apr 17 22:00 0ee58208-6be8-4f81-bd51-<u></u>0bd4b6d5d83a<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 17 22:00 c298ce3b-ec6a-4526-9971-<u></u>a769f4d3d69b.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 17 22:00 0ee58208-6be8-4f81-bd51-<u></u>0bd4b6d5d83a.meta<br>
-rw-rw----. 1 vdsm kvm 117374976 Apr 16 22:00 9aeb973d-9a54-441e-9ce9-<u></u>f4f1a233da26<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 16 22:00 0ee58208-6be8-4f81-bd51-<u></u>0bd4b6d5d83a.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 16 22:00 9aeb973d-9a54-441e-9ce9-<u></u>f4f1a233da26.meta<br>
-rw-rw----. 1 vdsm kvm 110886912 Apr 15 22:00 0eae2185-884a-44d3-9099-<u></u>e952b6b7ec37<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 15 22:00 9aeb973d-9a54-441e-9ce9-<u></u>f4f1a233da26.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 15 22:00 0eae2185-884a-44d3-9099-<u></u>e952b6b7ec37.meta<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 14 22:00 0eae2185-884a-44d3-9099-<u></u>e952b6b7ec37.lease<br>
-rw-rw----. 1 vdsm kvm 164560896 Apr 14 22:00 ceffc643-b823-44b3-961e-<u></u>93f3dc971886<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 14 22:00 ceffc643-b823-44b3-961e-<u></u>93f3dc971886.meta<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 13 22:00 ceffc643-b823-44b3-961e-<u></u>93f3dc971886.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 13 22:00 878fc690-ab08-489c-955b-<u></u>9159f62026b1.meta<br>
-rw-rw----. 1 vdsm kvm 109182976 Apr 13 21:59 878fc690-ab08-489c-955b-<u></u>9159f62026b1<br>
-rw-rw----. 1 vdsm kvm 110297088 Apr 12 22:00 5210eec2-a0eb-462e-95d5-<u></u>7cf27db312f5<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 12 22:00 878fc690-ab08-489c-955b-<u></u>9159f62026b1.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 12 22:00 5210eec2-a0eb-462e-95d5-<u></u>7cf27db312f5.meta<br>
-rw-rw----. 1 vdsm kvm 76480512 Apr 11 22:00 dcce0903-0f24-434b-9d1c-<u></u>d70e3969e5ea<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 11 22:00 5210eec2-a0eb-462e-95d5-<u></u>7cf27db312f5.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 11 22:00 dcce0903-0f24-434b-9d1c-<u></u>d70e3969e5ea.meta<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 11 12:34 dcce0903-0f24-434b-9d1c-<u></u>d70e3969e5ea.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 11 12:34 d3a1c505-8f6a-4c2b-97b7-<u></u>764cd5baea47.meta<br>
-rw-rw----. 1 vdsm kvm 208666624 Apr 11 12:33 d3a1c505-8f6a-4c2b-97b7-<u></u>764cd5baea47<br>
-rw-rw----. 1 vdsm kvm 14614528 Apr 10 16:12 638c2164-2edc-4294-ac99-<u></u>c51963140940<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 16:12 d3a1c505-8f6a-4c2b-97b7-<u></u>764cd5baea47.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 16:12 638c2164-2edc-4294-ac99-<u></u>c51963140940.meta<br>
-rw-rw----. 1 vdsm kvm 12779520 Apr 10 16:06 f8f1f164-c0d9-4716-9ab3-<u></u>9131179a79bd<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 16:05 638c2164-2edc-4294-ac99-<u></u>c51963140940.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 16:05 f8f1f164-c0d9-4716-9ab3-<u></u>9131179a79bd.meta<br>
-rw-rw----. 1 vdsm kvm 92995584 Apr 10 16:00 f9b14795-a26c-4edb-ae34-<u></u>22361531a0a1<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 16:00 f8f1f164-c0d9-4716-9ab3-<u></u>9131179a79bd.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 16:00 f9b14795-a26c-4edb-ae34-<u></u>22361531a0a1.meta<br>
-rw-rw----. 1 vdsm kvm 30015488 Apr 10 14:57 39cbf947-f084-4e75-8d6b-<u></u>b3e5c32b82d6<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 14:57 f9b14795-a26c-4edb-ae34-<u></u>22361531a0a1.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 14:57 39cbf947-f084-4e75-8d6b-<u></u>b3e5c32b82d6.meta<br>
-rw-rw----. 1 vdsm kvm 19267584 Apr 10 14:34 3ece1489-9bff-4223-ab97-<u></u>e45135106222<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 14:34 39cbf947-f084-4e75-8d6b-<u></u>b3e5c32b82d6.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 14:34 3ece1489-9bff-4223-ab97-<u></u>e45135106222.meta<br>
-rw-rw----. 1 vdsm kvm 22413312 Apr 10 14:29 dcee2e8a-8803-44e2-80e8-<u></u>82c882af83ef<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 14:28 3ece1489-9bff-4223-ab97-<u></u>e45135106222.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 14:28 dcee2e8a-8803-44e2-80e8-<u></u>82c882af83ef.meta<br>
-rw-rw----. 1 vdsm kvm 54460416 Apr 10 14:26 57066786-613a-46ff-b2f9-<u></u>06d84678975b<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 14:26 dcee2e8a-8803-44e2-80e8-<u></u>82c882af83ef.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 14:26 57066786-613a-46ff-b2f9-<u></u>06d84678975b.meta<br>
-rw-rw----. 1 vdsm kvm 15728640 Apr 10 13:31 121ae509-d2b2-4df2-a56f-<u></u>dfdba4b8d21c<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 13:30 57066786-613a-46ff-b2f9-<u></u>06d84678975b.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 13:30 121ae509-d2b2-4df2-a56f-<u></u>dfdba4b8d21c.meta<br>
-rw-rw----. 1 vdsm kvm 5767168 Apr 10 13:18 1d95a9d2-e4ba-4bcc-ba71-<u></u>5d493a838dcc<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 13:17 121ae509-d2b2-4df2-a56f-<u></u>dfdba4b8d21c.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 13:17 1d95a9d2-e4ba-4bcc-ba71-<u></u>5d493a838dcc.meta<br>
-rw-rw----. 1 vdsm kvm 5373952 Apr 10 13:13 3ce8936a-38f5-43a9-a4e0-<u></u>820094fbeb04<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 13:13 1d95a9d2-e4ba-4bcc-ba71-<u></u>5d493a838dcc.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 13:12 3ce8936a-38f5-43a9-a4e0-<u></u>820094fbeb04.meta<br>
-rw-rw----. 1 vdsm kvm 3815243776 Apr 10 13:11 7211d323-c398-4c1c-8524-<u></u>a1047f9d5ec9<br>
-rw-rw----. 1 vdsm kvm 1048576 Apr 10 13:11 3ce8936a-38f5-43a9-a4e0-<u></u>820094fbeb04.lease<br>
-rw-r--r--. 1 vdsm kvm 272 Apr 10 13:11 7211d323-c398-4c1c-8524-<u></u>a1047f9d5ec9.meta<br>
-rw-r--r--. 1 vdsm kvm 272 Mar 19 10:35 af94adc4-fad4-42f5-a004-<u></u>689670311d66.meta<br>
-rw-rw----. 1 vdsm kvm 21474836480 Mar 19 10:22 af94adc4-fad4-42f5-a004-<u></u>689670311d66<br>
-rw-rw----. 1 vdsm kvm 1048576 Mar 19 09:39 7211d323-c398-4c1c-8524-<u></u>a1047f9d5ec9.lease<br>
-rw-rw----. 1 vdsm kvm 1048576 Mar 19 09:39 af94adc4-fad4-42f5-a004-<u></u>689670311d66.lease<br>
<br>
Its just very odd that I can snapshot any other VM except this one.<br>
<br>
I just cloned a new VM from the last snapshot on this VM and it created without issue. I was also able to snapshot the new VM without a problem.<br>
<br></div></div>
*Steve<br>
*<div><div><br>
<br>
<br>
On Tue, Apr 22, 2014 at 12:51 PM, Dafna Ron <<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>> wrote:<br>
<br>
it's the same error:<br>
<br>
c1d7c4e-392b-4a62-9836-<u></u>3add1360a46d::DEBUG::2014-04-<u></u>22<br>
12:13:44,340::volume::1058::<u></u>Storage.Misc.excCmd::(<u></u>createVolume)<br>
FAILED: <err> =<br>
'/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/4<br>
66d9ae9-e46a-46f8-9f4b-<u></u>964d8af0675b/0b2d15e5-bf4f-<u></u>4eaf-90e2-f1bd51a3a936:<br>
error while creating qcow2: No such file or directory\n'; <rc> = 1<br>
<br>
<br>
were these 23 snapshots created any way each time we fail to<br>
create the snapshot or are these older snapshots which you<br>
actually created before the failure?<br>
<br>
at this point my main theory is that somewhere along the line you<br>
had some sort of failure in your storage and from that time each<br>
snapshot you create will fail.<br>
if the snapshots are created during the failure can you please<br>
delete the snapshots you do not need and try again?<br>
<br>
There should not be a limit on how many snapshots you can have<br>
since it's only a link changing the image the vm should boot from.<br>
Having said that, it's not ideal to have that many snapshots and<br>
can probably lead to unexpected results so I would not recommend<br>
having that many snapshots on a single vm :)<br>
<br>
for example, my second theory would be that because we have so<br>
many snapshots we have some sort of race where part of the<br>
createVolume command expects some result from a query run before<br>
the create itself and because there are so many snapshots there is<br>
"no such file" on the volume because it's too far up the list.<br>
<br>
can you also run: ls -l<br>
/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b<br>
<br>
lets see what images are listed under that vm.<br>
<br>
btw, you know that your export domain is getting<br>
StorageDomainDoesNotExist in the vdsm log? is that domain in up<br>
state? can you try to deactivate the export domain?<br>
<br>
Thanks,<br>
<br>
Dafna<br>
<br>
<br>
<br>
<br>
<br>
On 04/22/2014 05:20 PM, Steve Dainard wrote:<br>
<br>
Ominous..<br>
<br>
23 snapshots. Is there an upper limit?<br>
<br>
Offline snapshot fails as well. Both logs attached again<br>
(snapshot attempted at 12:13 EST).<br>
<br>
*Steve *<br>
<br>
<br>
On Tue, Apr 22, 2014 at 11:20 AM, Dafna Ron <<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
</div></div><div>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>>> wrote:<br>
<br>
are you able to take an offline snapshot? (while the vm is<br>
down)<br>
how many snapshots do you have on this vm?<br>
<br>
<br>
On 04/22/2014 04:19 PM, Steve Dainard wrote:<br>
<br>
No alert in web ui, I restarted the VM yesterday just<br>
in case,<br>
no change. I also restored an earlier snapshot and<br>
tried to<br>
re-snapshot, same result.<br>
<br>
*Steve<br>
*<br>
<br>
<br>
<br>
On Tue, Apr 22, 2014 at 10:57 AM, Dafna Ron<br>
<<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>><br>
</div><div><div>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>>>> wrote:<br>
<br>
This is the actual problem:<br>
<br>
bf025a73-eeeb-4ac5-b8a9-<u></u>32afa4ae482e::DEBUG::2014-04-<u></u>22<br>
10:21:49,374::volume::1058::<u></u>Storage.Misc.excCmd::(<u></u>createVolume)<br>
FAILED: <err> =<br>
'/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/4<br>
66d9ae9-e46a-46f8-9f4b-<u></u>964d8af0675b/87efa937-b31f-<u></u>4bb1-aee1-0ee14a0dc6fb:<br>
error while creating qcow2: No such file or<br>
directory\n';<br>
<rc> = 1<br>
<br>
from that you see the actual failure:<br>
<br>
bf025a73-eeeb-4ac5-b8a9-<u></u>32afa4ae482e::ERROR::2014-04-<u></u>22<br>
10:21:49,392::volume::286::<u></u>Storage.Volume::(clone)<br>
Volume.clone:<br>
can't clone:<br>
/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d<br>
9ae9-e46a-46f8-9f4b-<u></u>964d8af0675b/1a67de4b-aa1c-<u></u>4436-baca-ca55726d54d7<br>
to<br>
/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b/<u></u>87efa937-b31f-4bb1-aee1-0ee1<br>
4a0dc6fb<br>
bf025a73-eeeb-4ac5-b8a9-<u></u>32afa4ae482e::ERROR::2014-04-<u></u>22<br>
10:21:49,392::volume::508::<u></u>Storage.Volume::(create)<br>
Unexpected error<br>
Traceback (most recent call last):<br>
File "/usr/share/vdsm/storage/<u></u>volume.py", line<br>
466, in<br>
create<br>
srcVolUUID, imgPath, volPath)<br>
File "/usr/share/vdsm/storage/<u></u>fileVolume.py",<br>
line 160,<br>
in _create<br>
volParent.clone(imgPath, volUUID, volFormat,<br>
preallocate)<br>
File "/usr/share/vdsm/storage/<u></u>volume.py", line<br>
287, in clone<br>
raise se.CannotCloneVolume(self.<u></u>volumePath,<br>
dst_path,<br>
str(e))<br>
CannotCloneVolume: Cannot clone volume:<br>
'src=/rhev/data-center/<u></u>9497ef2c-8368-4c92-8d61-<u></u>7f318a90748f/95b9d922-4df7-<u></u>4d3b-9bca-467e2fd9d573/images/<u></u>466d9ae9-e46a-46f8-9f4b-<u></u>964d8af0675b/1a67de4b-aa1c-<u></u>4436-baca-ca55726d54d7,<br>
dst=/rhev/data-cen<br>
ter/9497ef2c-8368-4c92-8d61-<u></u>7f318a90748f/95b9d922-4df7-<u></u>4d3b-9bca-467e2fd9d573/images/<u></u>466d9ae9-e46a-46f8-9f4b-<u></u>964d8af0675b/87efa937-b31f-<u></u>4bb1-aee1-0ee14a0dc6fb:<br>
Error creating a new volume: (["Formatting<br>
\'/rhev/data-center/9497ef2c-<u></u>8368-<br>
4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b/<u></u>87efa937-b31f-4bb1-aee1-<u></u>0ee14a0dc6fb\',<br>
fmt=qcow2 size=21474836480<br>
backing_file=\'../466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b/<u></u>1a67de4b-aa<br>
1c-4436-baca-ca55726d54d7\' backing_fmt=\'qcow2\'<br>
encryption=off<br>
cluster_size=65536 "],)'<br>
<br>
<br>
do you have any alert in the webadmin to restart<br>
the vm?<br>
<br>
Dafna<br>
<br>
<br>
On 04/22/2014 03:31 PM, Steve Dainard wrote:<br>
<br>
Sorry for the confusion.<br>
<br>
I attempted to take a live snapshot of a<br>
running VM. After<br>
that failed, I migrated the VM to another<br>
host, and<br>
attempted<br>
the live snapshot again without success,<br>
eliminating a<br>
single<br>
host as the cause of failure.<br>
<br>
Ovirt is 3.3.4, storage domain is gluster<br>
3.4.2.1, OS is<br>
CentOS 6.5.<br>
<br>
Package versions:<br>
libvirt-0.10.2-29.el6_5.5.x86_<u></u>64<br>
libvirt-lock-sanlock-0.10.2-<u></u>29.el6_5.5.x86_64<br>
qemu-img-rhev-0.12.1.2-2.415.<u></u>el6.nux.3.x86_64<br>
qemu-kvm-rhev-0.12.1.2-2.415.<u></u>el6.nux.3.x86_64<br>
qemu-kvm-rhev-tools-0.12.1.2-<u></u>2.415.el6.nux.3.x86_64<br>
vdsm-4.13.3-4.el6.x86_64<br>
vdsm-gluster-4.13.3-4.el6.<u></u>noarch<br>
<br>
<br>
I made another live snapshot attempt at 10:21 EST<br>
today, full<br>
vdsm.log attached, and a truncated engine.log.<br>
<br>
Thanks,<br>
<br>
*Steve<br>
*<br>
<br>
<br>
<br>
On Tue, Apr 22, 2014 at 9:48 AM, Dafna Ron<br>
<<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>><br>
<br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>> <mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a><br>
<mailto:<a href="mailto:dron@redhat.com" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=dron@redhat.com&cc=&bcc=&su=&body=','_blank');return false;">dron@redhat.com</a>>>>>> wrote:<br>
<br>
please explain the flow of what you are<br>
trying to<br>
do, are you<br>
trying to live migrate the disk (from one<br>
storage to<br>
another), are<br>
you trying to migrate the vm and after vm<br>
migration is<br>
finished<br>
you try to take a live snapshot of the vm?<br>
or are you<br>
trying to<br>
take a live snapshot of the vm during a vm<br>
migration from<br>
host1 to<br>
host2?<br>
<br>
Please attach full vdsm logs from any host<br>
you are<br>
using<br>
(if you<br>
are trying to migrate the vm from host1 to<br>
host2)<br>
+ please<br>
attach<br>
engine log.<br>
<br>
Also, what is the vdsm, libvirt and qemu<br>
versions,<br>
what ovirt<br>
version are you using and what is the<br>
storage you<br>
are using?<br>
<br>
Thanks,<br>
<br>
Dafna<br>
<br>
<br>
<br>
<br>
On 04/22/2014 02:12 PM, Steve Dainard wrote:<br>
<br>
I've attempted migrating the vm to another<br>
host and<br>
taking a<br>
snapshot, but I get this error:<br>
<br>
6efd33f4-984c-4513-b5e6-<u></u>fffdca2e983b::ERROR::2014-04-<u></u>22<br>
01:09:37,296::volume::286::<u></u>Storage.Volume::(clone)<br>
Volume.clone: can't clone:<br>
/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b/<u></u>1a67de4b-aa1c-4436-baca-<u></u>ca55726d54d7<br>
to<br>
/rhev/data-center/9497ef2c-<u></u>8368-4c92-8d61-7f318a90748f/<u></u>95b9d922-4df7-4d3b-9bca-<u></u>467e2fd9d573/images/466d9ae9-<u></u>e46a-46f8-9f4b-964d8af0675b/<u></u>b230596f-97bc-4532-ba57-<u></u>5654fa9c6c51<br>
<br>
A bit more of the vdsm log is attached.<br>
<br>
Other vm's are snapshotting without issue.<br>
<br>
<br>
<br>
Any help appreciated,<br>
<br>
*Steve<br>
*<br>
<br>
<br>
______________________________<u></u>_________________<br>
Users mailing list<br>
<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>>>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>><br>
<mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a> <mailto:<a href="mailto:Users@ovirt.org" target="_blank" onclick="window.open('https://mail.google.com/mail/?view=cm&tf=1&to=Users@ovirt.org&cc=&bcc=&su=&body=','_blank');return false;">Users@ovirt.org</a>>>>><br>
<br>
<br>
<a href="http://lists.ovirt.org/mailman/listinfo/users" target="_blank">http://lists.ovirt.org/<u></u>mailman/listinfo/users</a><br>
<br>
<br>
<br>
-- Dafna Ron<br>
<br>
<br>
<br>
<br>
-- Dafna Ron<br>
<br>
<br>
<br>
<br>
-- Dafna Ron<br>
<br>
<br>
<br>
<br>
-- Dafna Ron<br>
<br>
<br>
</div></div></blockquote><span><font color="#888888">
<br>
<br>
-- <br>
Dafna Ron<br>
</font></span></blockquote></div><br></div></div>