How to re-initialize storage?
Hi, I inherited a small oVirt cluster with 3 nodes using iSCSI storage. The LUNs used by the oVirt nodes were also accessible by some VMware nodes. By mistake an action was started on VMware to initialize and start using the same LUNs, which it happily did. This rendered the storage no longer usable by oVirt nodes. All VMs went down, their image files no longer accessible, the whole cluster became unusable without access to the shared storage. We have recovered from backups and moved the VMs to another cluster and now I want to get this oVirt cluster back up and running. I'm not an oVirt expert so everything I tried was through the web interface and everything failed because storage is not accessible. I can't remove VMs, I can't remove nodes, I can't remove the storage domain. What is the right approach to destroy the cluster and rebuild it and re-initialize the storage? Even better if I can just remove all VMs from the inventory and somehow tell it to make the storage LUNs usable again? Any help is highly appreciated? Thank you, -- Peter
I am not an expert but storage is tied to your datacenter so you need to make sure your datacenter is removed so that you are able to remove storage On Mon, Mar 27, 2017 at 8:53 PM, Peter Wood <peterwood.sd@gmail.com> wrote:
Hi,
I inherited a small oVirt cluster with 3 nodes using iSCSI storage.
The LUNs used by the oVirt nodes were also accessible by some VMware nodes.
By mistake an action was started on VMware to initialize and start using the same LUNs, which it happily did. This rendered the storage no longer usable by oVirt nodes. All VMs went down, their image files no longer accessible, the whole cluster became unusable without access to the shared storage.
We have recovered from backups and moved the VMs to another cluster and now I want to get this oVirt cluster back up and running.
I'm not an oVirt expert so everything I tried was through the web interface and everything failed because storage is not accessible. I can't remove VMs, I can't remove nodes, I can't remove the storage domain.
What is the right approach to destroy the cluster and rebuild it and re-initialize the storage?
Even better if I can just remove all VMs from the inventory and somehow tell it to make the storage LUNs usable again?
Any help is highly appreciated?
Thank you,
-- Peter
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Hi Peter, You can take your hosts down to maintenance and then right click on your data center -> force remove. That will remove your storage and vms along with your datacenter. Regards, Idan On Mon, Mar 27, 2017 at 9:53 PM, Peter Wood <peterwood.sd@gmail.com> wrote:
Hi,
I inherited a small oVirt cluster with 3 nodes using iSCSI storage.
The LUNs used by the oVirt nodes were also accessible by some VMware nodes.
By mistake an action was started on VMware to initialize and start using the same LUNs, which it happily did. This rendered the storage no longer usable by oVirt nodes. All VMs went down, their image files no longer accessible, the whole cluster became unusable without access to the shared storage.
We have recovered from backups and moved the VMs to another cluster and now I want to get this oVirt cluster back up and running.
I'm not an oVirt expert so everything I tried was through the web interface and everything failed because storage is not accessible. I can't remove VMs, I can't remove nodes, I can't remove the storage domain.
What is the right approach to destroy the cluster and rebuild it and re-initialize the storage?
Even better if I can just remove all VMs from the inventory and somehow tell it to make the storage LUNs usable again?
Any help is highly appreciated?
Thank you,
-- Peter
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
On Tue, Mar 28, 2017 at 9:34 AM, Idan Shaby <ishaby@redhat.com> wrote:
Hi Peter,
You can take your hosts down to maintenance and then right click on your data center -> force remove. That will remove your storage and vms along with your datacenter.
Regards, Idan
Hi, I'd start off with backing up your db so you'll have a way to access your old configuration. I didn't understand how did you move the vms to another cluster - can you elaborate? Generally speaking, you can take a look at the import storage domain feature ( http://www.ovirt.org/develop/release-management/features/storage/importstora... ). Regards, Liron
On Mon, Mar 27, 2017 at 9:53 PM, Peter Wood <peterwood.sd@gmail.com> wrote:
Hi,
I inherited a small oVirt cluster with 3 nodes using iSCSI storage.
The LUNs used by the oVirt nodes were also accessible by some VMware nodes.
By mistake an action was started on VMware to initialize and start using the same LUNs, which it happily did. This rendered the storage no longer usable by oVirt nodes. All VMs went down, their image files no longer accessible, the whole cluster became unusable without access to the shared storage.
We have recovered from backups and moved the VMs to another cluster and now I want to get this oVirt cluster back up and running.
I'm not an oVirt expert so everything I tried was through the web interface and everything failed because storage is not accessible. I can't remove VMs, I can't remove nodes, I can't remove the storage domain.
What is the right approach to destroy the cluster and rebuild it and re-initialize the storage?
Even better if I can just remove all VMs from the inventory and somehow tell it to make the storage LUNs usable again?
Any help is highly appreciated?
Thank you,
-- Peter
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
Thank you all for the replies. When I said "moved the VMs" I meant "rebuilt the VMs". We just created new VMs and restored services from backups. I'm sorry for the confusion. I followed Idan's suggestions and was able to remove the datacenter. Thank you. On Tue, Mar 28, 2017 at 1:07 AM, Liron Aravot <laravot@redhat.com> wrote:
On Tue, Mar 28, 2017 at 9:34 AM, Idan Shaby <ishaby@redhat.com> wrote:
Hi Peter,
You can take your hosts down to maintenance and then right click on your data center -> force remove. That will remove your storage and vms along with your datacenter.
Regards, Idan
Hi, I'd start off with backing up your db so you'll have a way to access your old configuration. I didn't understand how did you move the vms to another cluster - can you elaborate? Generally speaking, you can take a look at the import storage domain feature (http://www.ovirt.org/develop/release-management/features/ storage/importstoragedomain/ ).
Regards, Liron
On Mon, Mar 27, 2017 at 9:53 PM, Peter Wood <peterwood.sd@gmail.com> wrote:
Hi,
I inherited a small oVirt cluster with 3 nodes using iSCSI storage.
The LUNs used by the oVirt nodes were also accessible by some VMware nodes.
By mistake an action was started on VMware to initialize and start using the same LUNs, which it happily did. This rendered the storage no longer usable by oVirt nodes. All VMs went down, their image files no longer accessible, the whole cluster became unusable without access to the shared storage.
We have recovered from backups and moved the VMs to another cluster and now I want to get this oVirt cluster back up and running.
I'm not an oVirt expert so everything I tried was through the web interface and everything failed because storage is not accessible. I can't remove VMs, I can't remove nodes, I can't remove the storage domain.
What is the right approach to destroy the cluster and rebuild it and re-initialize the storage?
Even better if I can just remove all VMs from the inventory and somehow tell it to make the storage LUNs usable again?
Any help is highly appreciated?
Thank you,
-- Peter
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (5)
-
Idan Shaby -
Liron Aravot -
martin chamambo -
Peter Wood -
Wood Peter