Re: Unable to start VM - Image is not a legal chain
(Changing subject, adding Freddy) On Tue, Jul 10, 2018 at 8:06 PM, Alex K <rightkicktech@gmail.com> wrote:
Hi all,
I did a routine maintenance today (updating the hosts) to ovirt cluster (4.2) and I have one VM that was complaining about an invalid snapshot. After shutdown of VM the VM is not able to start again, giving the error:
VM Gitlab is down with error. Exit message: Bad volume specification {'serial': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'index': 0, 'iface': 'virtio', 'apparentsize': '51838976', 'specParams': {}, 'cache': 'none', 'imageID': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'truesize': '52011008', 'type': 'disk', 'domainID': '142bbde6-ef9d-4a52-b9da-2de533c1f1bd', 'reqsize': '0', 'format': 'cow', 'poolID': '00000001-0001-0001-0001-000000000311', 'device': 'disk', 'path': '/rhev/data-center/00000001- 0001-0001-0001-000000000311/142bbde6-ef9d-4a52-b9da- 2de533c1f1bd/images/b6af2856-a164-484a-afe5-9836bbdd14e8/ f3125f62-c909-472f-919c-844e0b8c156d', 'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID': 'f3125f62-c909-472f-919c-844e0b8c156d', 'diskType': 'file', 'alias': 'ua-b6af2856-a164-484a-afe5-9836bbdd14e8', 'discard': False}.
I see also the following error:
VDSM command CopyImageVDS failed: Image is not a legal chain: (u'b6af2856-a164-484a-afe5-9836bbdd14e8',)
This error appears a few more times in the list's archive, all of which seem to be related to rather-old bugs (3.5/3.6 times) or storage problems. I assume you use 4.2. Are you sure the corruption happened only now? Did working with snapshots worked well before the upgrade?
Seems as a corrupt VM disk?
Seems so to me, but I am not a storage expert.
The VM had 3 snapshots. I was able to delete one from GUI then am not able to delete the other two as the task fails. Generally I am not allowed to clone, export or do sth to the VM.
Have you encountered sth similar. Any advice?
The lastest post, from 2016, included a workaround, you might (very carefully!) try that. I suggest to also open a bug and attach all relevant logs (engine, vdsm from all relevant hosts, including SPMs at time of snapshot operations and any other host that ran the VM), and try to give accurate reproduction steps. Best regards, -- Didi
Due to urgency of the case, I fetched the backup copy from weekend and proceeded to push missing data to VM (the VM is a git repo). I lost few notes, though not much damage was done... I'm starting to feel uncomfortable with this solution though and might switch (at least the production VMs) to plain KVM where I had never experienced such issues. Alex On Wed, Jul 11, 2018 at 7:27 AM, Yedidyah Bar David <didi@redhat.com> wrote:
(Changing subject, adding Freddy)
On Tue, Jul 10, 2018 at 8:06 PM, Alex K <rightkicktech@gmail.com> wrote:
Hi all,
I did a routine maintenance today (updating the hosts) to ovirt cluster (4.2) and I have one VM that was complaining about an invalid snapshot. After shutdown of VM the VM is not able to start again, giving the error:
VM Gitlab is down with error. Exit message: Bad volume specification {'serial': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'index': 0, 'iface': 'virtio', 'apparentsize': '51838976', 'specParams': {}, 'cache': 'none', 'imageID': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'truesize': '52011008', 'type': 'disk', 'domainID': '142bbde6-ef9d-4a52-b9da-2de533c1f1bd', 'reqsize': '0', 'format': 'cow', 'poolID': '00000001-0001-0001-0001-000000000311', 'device': 'disk', 'path': '/rhev/data-center/00000001-00 01-0001-0001-000000000311/142bbde6-ef9d-4a52-b9da-2de533c1f1 bd/images/b6af2856-a164-484a-afe5-9836bbdd14e8/f3125f62-c909-472f-919c-844e0b8c156d', 'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID': 'f3125f62-c909-472f-919c-844e0b8c156d', 'diskType': 'file', 'alias': 'ua-b6af2856-a164-484a-afe5-9836bbdd14e8', 'discard': False}.
I see also the following error:
VDSM command CopyImageVDS failed: Image is not a legal chain: (u'b6af2856-a164-484a-afe5-9836bbdd14e8',)
This error appears a few more times in the list's archive, all of which seem to be related to rather-old bugs (3.5/3.6 times) or storage problems. I assume you use 4.2. Are you sure the corruption happened only now? Did working with snapshots worked well before the upgrade?
Seems as a corrupt VM disk?
Seems so to me, but I am not a storage expert.
The VM had 3 snapshots. I was able to delete one from GUI then am not able to delete the other two as the task fails. Generally I am not allowed to clone, export or do sth to the VM.
Have you encountered sth similar. Any advice?
The lastest post, from 2016, included a workaround, you might (very carefully!) try that.
I suggest to also open a bug and attach all relevant logs (engine, vdsm from all relevant hosts, including SPMs at time of snapshot operations and any other host that ran the VM), and try to give accurate reproduction steps.
Best regards, -- Didi
Hi Alex, We had a bigger problem recently which involved the error you mention. I sent it to the mail list and you can find the final solution we chose at [1]. Not the cleanest solution of course, but we managed to recover all VMs... I think in your case the relevant part is the one that mention the "vdsClient setVolumeLegality" command, although I don't know the root reason why you're getting the error (might be a corrupt snapshot, as in our case)... Hope this helps. [1]: https://www.mail-archive.com/users@ovirt.org/msg49300.html El 2018-07-11 09:16, Alex K escribió:
Due to urgency of the case, I fetched the backup copy from weekend and proceeded to push missing data to VM (the VM is a git repo). I lost few notes, though not much damage was done... I'm starting to feel uncomfortable with this solution though and might switch (at least the production VMs) to plain KVM where I had never experienced such issues.
Alex
On Wed, Jul 11, 2018 at 7:27 AM, Yedidyah Bar David <didi@redhat.com> wrote:
(Changing subject, adding Freddy)
On Tue, Jul 10, 2018 at 8:06 PM, Alex K <rightkicktech@gmail.com> wrote:
Hi all,
I did a routine maintenance today (updating the hosts) to ovirt cluster (4.2) and I have one VM that was complaining about an invalid snapshot. After shutdown of VM the VM is not able to start again, giving the error:
VM Gitlab is down with error. Exit message: Bad volume specification {'serial': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'index': 0, 'iface': 'virtio', 'apparentsize': '51838976', 'specParams': {}, 'cache': 'none', 'imageID': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'truesize': '52011008', 'type': 'disk', 'domainID': '142bbde6-ef9d-4a52-b9da-2de533c1f1bd', 'reqsize': '0', 'format': 'cow', 'poolID': '00000001-0001-0001-0001-000000000311', 'device': 'disk', 'path':
'/rhev/data-center/00000001-0001-0001-0001-000000000311/142bbde6-ef9d-4a52-b9da-2de533c1f1bd/images/b6af2856-a164-484a-afe5-9836bbdd14e8/f3125f62-c909-472f-919c-844e0b8c156d',
'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1', 'volumeID': 'f3125f62-c909-472f-919c-844e0b8c156d', 'diskType': 'file', 'alias': 'ua-b6af2856-a164-484a-afe5-9836bbdd14e8', 'discard': False}.
I see also the following error:
VDSM command CopyImageVDS failed: Image is not a legal chain: (u'b6af2856-a164-484a-afe5-9836bbdd14e8',)
This error appears a few more times in the list's archive, all of which seem to be related to rather-old bugs (3.5/3.6 times) or storage problems. I assume you use 4.2. Are you sure the corruption happened only now? Did working with snapshots worked well before the upgrade?
Seems as a corrupt VM disk?
Seems so to me, but I am not a storage expert.
The VM had 3 snapshots. I was able to delete one from GUI then am not able to delete the other two as the task fails. Generally I am not allowed to clone, export or do sth to the VM.
Have you encountered sth similar. Any advice?
The lastest post, from 2016, included a workaround, you might (very carefully!) try that.
I suggest to also open a bug and attach all relevant logs (engine, vdsm from all relevant hosts, including SPMs at time of snapshot operations and any other host that ran the VM), and try to give accurate reproduction steps.
Best regards, --
Didi
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/VYR4KMOZVEQOJ5...
Thank you Nicolas. I will keep this for future reference in case I encounter this again. Now I don't have the corrupted VM to experiment. Thanx, Alex On Wed, Jul 11, 2018 at 11:28 AM, <nicolas@devels.es> wrote:
Hi Alex,
We had a bigger problem recently which involved the error you mention. I sent it to the mail list and you can find the final solution we chose at [1]. Not the cleanest solution of course, but we managed to recover all VMs... I think in your case the relevant part is the one that mention the "vdsClient setVolumeLegality" command, although I don't know the root reason why you're getting the error (might be a corrupt snapshot, as in our case)...
Hope this helps.
[1]: https://www.mail-archive.com/users@ovirt.org/msg49300.html
El 2018-07-11 09:16, Alex K escribió:
Due to urgency of the case, I fetched the backup copy from weekend and proceeded to push missing data to VM (the VM is a git repo). I lost few notes, though not much damage was done... I'm starting to feel uncomfortable with this solution though and might switch (at least the production VMs) to plain KVM where I had never experienced such issues.
Alex
On Wed, Jul 11, 2018 at 7:27 AM, Yedidyah Bar David <didi@redhat.com> wrote:
(Changing subject, adding Freddy)
On Tue, Jul 10, 2018 at 8:06 PM, Alex K <rightkicktech@gmail.com> wrote:
Hi all,
I did a routine maintenance today (updating the hosts) to ovirt cluster (4.2) and I have one VM that was complaining about an invalid snapshot. After shutdown of VM the VM is not able to start again, giving the error:
VM Gitlab is down with error. Exit message: Bad volume specification {'serial': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'index': 0, 'iface': 'virtio', 'apparentsize': '51838976', 'specParams': {}, 'cache': 'none', 'imageID': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'truesize': '52011008', 'type': 'disk', 'domainID': '142bbde6-ef9d-4a52-b9da-2de533c1f1bd', 'reqsize': '0', 'format': 'cow', 'poolID': '00000001-0001-0001-0001-000000000311', 'device': 'disk', 'path':
'/rhev/data-center/00000001-0001-0001-0001-000000000311/142b
bde6-ef9d-4a52-b9da-2de533c1f1bd/images/b6af2856-a164-484a- afe5-9836bbdd14e8/f3125f62-c909-472f-919c-844e0b8c156d',
'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1',
'volumeID': 'f3125f62-c909-472f-919c-844e0b8c156d', 'diskType': 'file', 'alias': 'ua-b6af2856-a164-484a-afe5-9836bbdd14e8', 'discard': False}.
I see also the following error:
VDSM command CopyImageVDS failed: Image is not a legal chain: (u'b6af2856-a164-484a-afe5-9836bbdd14e8',)
This error appears a few more times in the list's archive, all of which seem to be related to rather-old bugs (3.5/3.6 times) or storage problems. I assume you use 4.2. Are you sure the corruption happened only now? Did working with snapshots worked well before the upgrade?
Seems as a corrupt VM disk?
Seems so to me, but I am not a storage expert.
The VM had 3 snapshots. I was able to delete one from GUI then am
not able to delete the other two as the task fails. Generally I am not allowed to clone, export or do sth to the VM.
Have you encountered sth similar. Any advice?
The lastest post, from 2016, included a workaround, you might (very carefully!) try that.
I suggest to also open a bug and attach all relevant logs (engine, vdsm from all relevant hosts, including SPMs at time of snapshot operations and any other host that ran the VM), and try to give accurate reproduction steps.
Best regards, --
Didi
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/messag e/VYR4KMOZVEQOJ5CBTFJMRYY7XAF5YAPP/
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/communit y/about/community-guidelines/ List Archives: https://lists.ovirt.org/archiv es/list/users@ovirt.org/message/Q4TUEIMBIHO6BX5JQQQSVC2RFD6IRAQG/
Hi all, coming back on this I did experience again the issue and the referred link was useful to restore the VMs. https://www.mail-archive.com/users@ovirt.org/msg49300.html I followed the steps to delete the illegal snapshot while the VM was down. I now have again the issue with other 3 VMs which are currently running (critical production ones). Is it possible to follow the steps and delete the illegal snapshot while the VMs are running? Thanx, Alex On Thu, Jul 12, 2018 at 10:40 AM Alex K <rightkicktech@gmail.com> wrote:
Thank you Nicolas. I will keep this for future reference in case I encounter this again. Now I don't have the corrupted VM to experiment.
Thanx, Alex
On Wed, Jul 11, 2018 at 11:28 AM, <nicolas@devels.es> wrote:
Hi Alex,
We had a bigger problem recently which involved the error you mention. I sent it to the mail list and you can find the final solution we chose at [1]. Not the cleanest solution of course, but we managed to recover all VMs... I think in your case the relevant part is the one that mention the "vdsClient setVolumeLegality" command, although I don't know the root reason why you're getting the error (might be a corrupt snapshot, as in our case)...
Hope this helps.
[1]: https://www.mail-archive.com/users@ovirt.org/msg49300.html
El 2018-07-11 09:16, Alex K escribió:
Due to urgency of the case, I fetched the backup copy from weekend and proceeded to push missing data to VM (the VM is a git repo). I lost few notes, though not much damage was done... I'm starting to feel uncomfortable with this solution though and might switch (at least the production VMs) to plain KVM where I had never experienced such issues.
Alex
On Wed, Jul 11, 2018 at 7:27 AM, Yedidyah Bar David <didi@redhat.com> wrote:
(Changing subject, adding Freddy)
On Tue, Jul 10, 2018 at 8:06 PM, Alex K <rightkicktech@gmail.com> wrote:
Hi all,
I did a routine maintenance today (updating the hosts) to ovirt cluster (4.2) and I have one VM that was complaining about an invalid snapshot. After shutdown of VM the VM is not able to start again, giving the error:
VM Gitlab is down with error. Exit message: Bad volume specification {'serial': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'index': 0, 'iface': 'virtio', 'apparentsize': '51838976', 'specParams': {}, 'cache': 'none', 'imageID': 'b6af2856-a164-484a-afe5-9836bbdd14e8', 'truesize': '52011008', 'type': 'disk', 'domainID': '142bbde6-ef9d-4a52-b9da-2de533c1f1bd', 'reqsize': '0', 'format': 'cow', 'poolID': '00000001-0001-0001-0001-000000000311', 'device': 'disk', 'path':
'/rhev/data-center/00000001-0001-0001-0001-000000000311/142bbde6-ef9d-4a52-b9da-2de533c1f1bd/images/b6af2856-a164-484a-afe5-9836bbdd14e8/f3125f62-c909-472f-919c-844e0b8c156d',
'propagateErrors': 'off', 'name': 'vda', 'bootOrder': '1',
'volumeID': 'f3125f62-c909-472f-919c-844e0b8c156d', 'diskType': 'file', 'alias': 'ua-b6af2856-a164-484a-afe5-9836bbdd14e8', 'discard': False}.
I see also the following error:
VDSM command CopyImageVDS failed: Image is not a legal chain: (u'b6af2856-a164-484a-afe5-9836bbdd14e8',)
This error appears a few more times in the list's archive, all of which seem to be related to rather-old bugs (3.5/3.6 times) or storage problems. I assume you use 4.2. Are you sure the corruption happened only now? Did working with snapshots worked well before the upgrade?
Seems as a corrupt VM disk?
Seems so to me, but I am not a storage expert.
The VM had 3 snapshots. I was able to delete one from GUI then am
not able to delete the other two as the task fails. Generally I am not allowed to clone, export or do sth to the VM.
Have you encountered sth similar. Any advice?
The lastest post, from 2016, included a workaround, you might (very carefully!) try that.
I suggest to also open a bug and attach all relevant logs (engine, vdsm from all relevant hosts, including SPMs at time of snapshot operations and any other host that ran the VM), and try to give accurate reproduction steps.
Best regards, --
Didi
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives:
https://lists.ovirt.org/archives/list/users@ovirt.org/message/VYR4KMOZVEQOJ5...
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Q4TUEIMBIHO6BX...
participants (3)
-
Alex K -
nicolas＠devels.es -
Yedidyah Bar David