Hi, We have a 4 node cluster (4.4.10) and a standalone manager. We attached to this cluster 4 volumes on FC. Last week we lost the storage canisters (both), but recovered one of them from the storage disks. Unfortunatly only 3 storages came back from offline. I see the 4th FC ID, but it's on "Can't Extend" status. If i check the storages on nodes (lsblk), i can see only the ID not more on this 4th storage. Do you know any recovery tool or any idea for this situation? Thanks csabany
I see the 4th FC ID, but it's on "Can't Extend" status. I have no idea what this one means. Can you give a screenshot ? Best Regards,Strahil Nikolov
Hi, We have a 4 node cluster (4.4.10) and a standalone manager. We attached to this cluster 4 volumes on FC. Last week we lost the storage canisters (both), but recovered one of them from the storage disks. Unfortunatly only 3 storages came back from offline. I see the 4th FC ID, but it's on "Can't Extend" status. If i check the storages on nodes (lsblk), i can see only the ID not more on this 4th storage. Do you know any recovery tool or any idea for this situation? Thanks csabany _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MSTFARNGJ4STMD...
Hi,I have attached 2 pictures.On the fcerrorovirt,jpg, you can see this "status", on the other shows so on the storage is "everyting is ok". There are 4 volumes and the first one (50TB) is the "problem" storage. Thankscsabany-------- Eredeti levél --------Feladó: Strahil Nikolov via Users <users@ovirt.org>Dátum: 2022 május 3 20:24:11Tárgy: [ovirt-users] Re: FC Storage recoverCímzett: csabany@freemail.hu users@ovirt.org>I see the 4th FC ID, but it's on "Can't Extend" status.I have no idea what this one means. Can you give a screenshot ?Best Regards,Strahil NikolovHi,We have a 4 node cluster (4.4.10) and a standalone manager.We attached to this cluster 4 volumes on FC.Last week we lost the storage canisters (both), but recovered one of them from the storage disks. Unfortunatly only 3 storages came back from offline.I see the 4th FC ID, but it's on "Can't Extend" status.If i check the storages on nodes (lsblk), i can see only the ID not more on this 4th storage.Do you know any recovery tool or any idea for this situation?Thanks csabany _______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/MSTFARNGJ4STMD...
What is the status of your hosts ? Do you have a host designated as SPM ? Best Regards,Strahil Nikolov On Wed, May 4, 2022 at 8:03, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UXRIXG5ZAC7OEU...
All host status is "up".Yes i do.The node26617 (on the fcerrorovirt.jpg ) is the SPM.thanks,csabany-------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 4 08:52:24Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: Nyika Csaba <csabany@freemail.hu> users@ovirt.org <users@ovirt.org>What is the status of your hosts ? Do you have a host designated as SPM ?Best Regards,Strahil NikolovOn Wed, May 4, 2022 at 8:03, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UXRIXG5ZAC7OEU...
You will have to provide some logs from the SPM. If you don't use the storage domain (at least this is the impression), try to set it into maintenance and activate it again (UI -> Storage domain -> Select domain -> Datacenter -> Maintenance). Best Regards,Strahil Nikolov On Wed, May 4, 2022 at 18:00, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XAHLBEVJTUUQZQ...
That was i tried first but i got "Error while executing action: Cannot deactivate Storage. The relevant Storage Domain's status is Inactive." message.Which logfile should i send from the SPM host? I downloded and zipped the whole /var/log but it's too large (105mb).Thankscsabany-------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 4 17:23:03Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: Nyika Csaba <csabany@freemail.hu> users@ovirt.org <users@ovirt.org>You will have to provide some logs from the SPM.If you don't use the storage domain (at least this is the impression), try to set it into maintenance and activate it again (UI -> Storage domain -> Select domain -> Datacenter -> Maintenance).Best Regards,Strahil NikolovOn Wed, May 4, 2022 at 18:00, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/XAHLBEVJTUUQZQ...
If it's inactive, is the 'Activate' button greyed out ? Best Regards,Strahil Nikolov On Thu, May 5, 2022 at 8:41, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZKRDNS6X75HWVR...
No, the 'Activate' button is active, but activate job failed.So if i check the lsblk on a node, in this FC ID i hava got only a simpe line like this:sdao 66:128 0 51T 0 disk └─36005076300808894600000000000000a 253:119 0 51T 0 mpathThe others (for example):sdan 66:112 0 25T 0 disk └─360050763008088946000000000000008 253:15 0 25T 0 mpath ├─1831603c--e583--412a--b20e--f97b31ad9a55-metadata 253:39 0 512M 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-ids 253:50 0 128M 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-leases 253:84 0 2G 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-outbox 253:97 0 128M 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-inbox 253:98 0 128M 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-master 253:99 0 1G 0 lvm ├─1831603c--e583--412a--b20e--f97b31ad9a55-xleasesI think on this storge has a data corruption maybe.Best,csabany------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 5 14:43:41Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: csabany@freemail.hu users@ovirt.org <users@ovirt.org>If it's inactive, is the 'Activate' button greyed out ?Best Regards,Strahil NikolovOn Thu, May 5, 2022 at 8:41, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/ZKRDNS6X75HWVR...
Login over ssh to the SPM, then trigger the Activate button and then copy the vdsm log (last one) from the host.Also check with 'multipath -ll 36005076300808894600000000000000a' if all hosts have the multipath device and it's status is good.oVirt refuses to activate if it can't mount a file-based storage (NFS, Gluster) or access the LV on the block-based storage domains (FC, iSCSI,etc). Also in the vdsm logs of the SPM should have a clue why it fails to activate. I'm not sure if oVirt uses SCSI reservations or not, but it's worth checking). Best Regards,Strahil Nikolov On Thu, May 5, 2022 at 16:36, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YVZYWT2KSKDHS...
Hi,I have attached the relevant log.The multipath output:36005076300808894600000000000000a dm-119 IBM ,2145 size=51T features='1 queue_if_no_path' hwhandler='0' wp=rw|-+- policy='service-time 0' prio=50 status=active| |- 1:0:1:3 sdak 66:64 active ready running| `- 2:0:7:3 sdao 66:128 active ready running`-+- policy='service-time 0' prio=10 status=enabled |- 2:0:6:3 sdy 65:128 active ready running `- 1:0:0:3 sdx 65:112 active ready runningThankscsabany-------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 5 19:13:39Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: Nyika Csaba <csabany@freemail.hu> users@ovirt.org <users@ovirt.org>Login over ssh to the SPM, then trigger the Activate button and then copy the vdsm log (last one) from the host.Also check with 'multipath -ll 36005076300808894600000000000000a' if all hosts have the multipath device and it's status is good.oVirt refuses to activate if it can't mount a file-based storage (NFS, Gluster) or access the LV on the block-based storage domains (FC, iSCSI,etc).Also in the vdsm logs of the SPM should have a clue why it fails to activate.I'm not sure if oVirt uses SCSI reservations or not, but it's worth checking).Best Regards,Strahil NikolovOn Thu, May 5, 2022 at 16:36, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YVZYWT2KSKDHS...
Sorry, incorrect log was what i have sent before.Attached corrtect.Thanks,csabany-------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 5 19:13:39Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: Nyika Csaba <csabany@freemail.hu> users@ovirt.org <users@ovirt.org>Login over ssh to the SPM, then trigger the Activate button and then copy the vdsm log (last one) from the host.Also check with 'multipath -ll 36005076300808894600000000000000a' if all hosts have the multipath device and it's status is good.oVirt refuses to activate if it can't mount a file-based storage (NFS, Gluster) or access the LV on the block-based storage domains (FC, iSCSI,etc).Also in the vdsm logs of the SPM should have a clue why it fails to activate.I'm not sure if oVirt uses SCSI reservations or not, but it's worth checking).Best Regards,Strahil NikolovOn Thu, May 5, 2022 at 16:36, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/6YVZYWT2KSKDHS...
Do all hosts have 'active ready running' ? P.S.: It will take me some time to look in the logs as it's hard to check them on the move. Best Regards,Strahil Nikolov On Thu, May 5, 2022 at 22:09, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/3N2O34VFWX3F22...
Yes. All 27 nodes up and running.Thankscsabany-------- Eredeti levél --------Feladó: Strahil Nikolov <hunter86_bg@yahoo.com>Dátum: 2022 május 6 08:37:45Tárgy: Re: [ovirt-users] Re: FC Storage recoverCímzett: Nyika Csaba <csabany@freemail.hu> users@ovirt.org <users@ovirt.org>Do all hosts have 'active ready running' ?P.S.: It will take me some time to look in the logs as it's hard to check them on the move.Best Regards,Strahil NikolovOn Thu, May 5, 2022 at 22:09, Nyika Csaba<csabany@freemail.hu> wrote:_______________________________________________Users mailing list -- users@ovirt.orgTo unsubscribe send an email to users-leave@ovirt.orgPrivacy Statement: https://www.ovirt.org/privacy-policy.htmloVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/3N2O34VFWX3F22...
According to vdsm, there are some failed multipath paths -> you will need to clear them (pure LinuxSystem Administration) as they might cause the real symptom. According to the log there are read errors. You have to figure out why vdsm fails to read the LUN.Can you try to 'dd if=/dev/mapper/multipath-wwid of=/dev/null bs=10M count=200' and see if it errors out. Best Regards,Strahil Nikolov On Fri, May 6, 2022 at 15:37, Nyika Csaba<csabany@freemail.hu> wrote: _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/privacy-policy.html oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/Z7QQGPX6GX5SX3...
participants (3)
-
csabany＠freemail.hu -
Nyika Csaba
-
Strahil Nikolov