oVirt host deploy: disable multipath

https://bugzilla.redhat.com/show_bug.cgi?id=1472356 There are 2 drives in my install that oVirt is claiming as multipath. I have tried blacklisting at /etc/multipath/ but that hasn't worked. Any better solutions? I need these 2 drives to be non-multipathed.

On Wed, Jun 6, 2018 at 7:51 PM femi adegoke <ovirt@fateknollogee.com> wrote:
https://bugzilla.redhat.com/show_bug.cgi?id=1472356
There are 2 drives in my install that oVirt is claiming as multipath. I have tried blacklisting at /etc/multipath/ but that hasn't worked.
We hoped that we have a good way to blacklist local drives in EL 7.5 but it seems that we don't have such way yet. You have to manually blacklist the devices. The best way to do this is to add a new file in: /etc/multipath/conf.d/local.conf And put there: blacklist { wwid xxxyyy wwid yyyzzz } Or many other options documented in: See https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/7/htm... You have to reload multipathd after that: systemctl reload multipathd If the devices are already used by the rest of the stack above multipath I guess you will have to reboot. We are tracking this issue in: https://bugzilla.redhat.com/1016535 Nir
Any better solutions? I need these 2 drives to be non-multipathed. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JCTFCH2IV45ZDR...

Thanks Nir for the update. What if the drive is also a boot/os drive, are we still able to disable multipath?

On Thu, Jun 7, 2018 at 1:29 AM femi adegoke <ovirt@fateknollogee.com> wrote:
Thanks Nir for the update.
What if the drive is also a boot/os drive, are we still able to disable multipath?
I think you need to copy /etc/multipath/conf.d/local.conf into initrd, so multipathd will find it during early boot. See https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm... Ben: does it work for /etc/multipath/conf.d/ drop-in files? Nir
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/X3RELHXWVEN7IA...

The SSD that is used boot/os will not get away from multipath even though it is listed int blacklist. I've also ran this cmd a few times, nothing has changed "dracut --force --add multipath --include /etc/multipath /initramfs-3.10.0-862.3.2.el7.x86_64.img" Any ideas?

Also tried "dracut --force --add multipath --include /etc/multipath /etc/multipath" Same results, the boot/os SSD is still a multipath device.

Please try to keep relevant content of the previous mail when you reply. On Thu, Jun 7, 2018 at 3:43 AM femi adegoke <ovirt@fateknollogee.com> wrote:
The SSD that is used boot/os will not get away from multipath even though it is listed int blacklist.
I've also ran this cmd a few times, nothing has changed "dracut --force --add multipath --include /etc/multipath /initramfs-3.10.0-862.3.2.el7.x86_64.img"
Please check https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/6/htm... to make sure you configured the blacklist correctly. If does not work, please add more info: - contents of /etc/multipath.conf - contents of /etc/multiapth/conf.d/local.conf (or whatever file you used to blacklist) - output of "multipath -ll" - output "lsblk" - output of "lsinitrd" Nir

Noted, regarding your reply about content. - contents of /etc/multipath.conf = conf.d (folder), bindings, wwids - contents of /etc/multiapth/conf.d/local.conf blacklist { wwid INTEL_SSDSCKHB120G4_BTWM65160025120B devnode "*" wwid eui.0025385171b04d62 wwid SAMSUNG_MZ7GE960HMHP-000AZ_S1P8NYAG123827 } - output of "multipath -ll" [root@ovirt-node00 ~]# multipath -ll INTEL_SSDSCKHB120G4_BTWM65160025120B dm-1 , size=112G features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=0 status=active `- #:#:#:# sda 8:0 active undef running - output "lsblk" [root@ovirt-node00 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 111.8G 0 disk ├─sda1 8:1 0 1G 0 part ├─sda2 8:2 0 110.8G 0 part └─INTEL_SSDSCKHB120G4_BTWM65160025120B 253:1 0 111.8G 0 mpath ├─INTEL_SSDSCKHB120G4_BTWM65160025120B1 253:2 0 1G 0 part /boot └─INTEL_SSDSCKHB120G4_BTWM65160025120B2 253:3 0 110.8G 0 part ├─onn_ovirt--node00-swap 253:4 0 11.2G 0 lvm [SWAP] ├─onn_ovirt--node00-pool00_tmeta 253:5 0 1G 0 lvm │ └─onn_ovirt--node00-pool00-tpool 253:7 0 77.4G 0 lvm │ ├─onn_ovirt--node00-pool00 253:0 0 77.4G 0 lvm │ ├─onn_ovirt--node00-ovirt--node--ng--4.2.3.1--0.20180530.0+1 253:8 0 50.4G 0 lvm / │ ├─onn_ovirt--node00-home 253:10 0 1G 0 lvm /home │ ├─onn_ovirt--node00-tmp 253:11 0 1G 0 lvm /tmp │ ├─onn_ovirt--node00-var 253:12 0 15G 0 lvm /var │ ├─onn_ovirt--node00-var_log 253:13 0 8G 0 lvm /var/log │ ├─onn_ovirt--node00-var_log_audit 253:14 0 2G 0 lvm /var/log/audit │ └─onn_ovirt--node00-var_crash 253:16 0 10G 0 lvm /var/crash └─onn_ovirt--node00-pool00_tdata 253:6 0 77.4G 0 lvm └─onn_ovirt--node00-pool00-tpool 253:7 0 77.4G 0 lvm ├─onn_ovirt--node00-pool00 253:0 0 77.4G 0 lvm ├─onn_ovirt--node00-ovirt--node--ng--4.2.3.1--0.20180530.0+1 253:8 0 50.4G 0 lvm / ├─onn_ovirt--node00-home 253:10 0 1G 0 lvm /home ├─onn_ovirt--node00-tmp 253:11 0 1G 0 lvm /tmp ├─onn_ovirt--node00-var 253:12 0 15G 0 lvm /var ├─onn_ovirt--node00-var_log 253:13 0 8G 0 lvm /var/log ├─onn_ovirt--node00-var_log_audit 253:14 0 2G 0 lvm /var/log/audit └─onn_ovirt--node00-var_crash 253:16 0 10G 0 lvm /var/crash sdb 8:16 0 894.3G 0 disk nvme0n1 259:0 0 465.8G 0 disk - output of "lsinitrd"

On Thu, Jun 7, 2018 at 4:55 PM femi adegoke <ovirt@fateknollogee.com> wrote:
Noted, regarding your reply about content.
- contents of /etc/multipath.conf
I mean contents of the file /etc/multipath.conf
= conf.d (folder), bindings, wwids
- contents of /etc/multiapth/conf.d/local.conf blacklist { wwid INTEL_SSDSCKHB120G4_BTWM65160025120B devnode "*"
This looks wrong, this is not a valid regular expression. Not sure what are you trying to do here.
wwid eui.0025385171b04d62 wwid SAMSUNG_MZ7GE960HMHP-000AZ_S1P8NYAG123827
I guess these are sdb and nvme0n1? we see in lsblk? }
- output of "multipath -ll" [root@ovirt-node00 ~]# multipath -ll INTEL_SSDSCKHB120G4_BTWM65160025120B dm-1 , size=112G features='0' hwhandler='0' wp=rw `-+- policy='service-time 0' prio=0 status=active `- #:#:#:# sda 8:0 active undef running
- output "lsblk" [root@ovirt-node00 ~]# lsblk NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT sda 8:0 0 111.8G 0 disk ├─sda1 8:1 0 1G 0 part ├─sda2 8:2 0 110.8G 0 part └─INTEL_SSDSCKHB120G4_BTWM65160025120B 253:1 0 111.8G 0 mpath ├─INTEL_SSDSCKHB120G4_BTWM65160025120B1 253:2 0 1G 0 part /boot └─INTEL_SSDSCKHB120G4_BTWM65160025120B2 253:3 0 110.8G 0 part ├─onn_ovirt--node00-swap 253:4 0 11.2G 0 lvm [SWAP] ├─onn_ovirt--node00-pool00_tmeta 253:5 0 1G 0 lvm │ └─onn_ovirt--node00-pool00-tpool 253:7 0 77.4G 0 lvm │ ├─onn_ovirt--node00-pool00 253:0 0 77.4G 0 lvm │ ├─onn_ovirt--node00-ovirt--node--ng--4.2.3.1--0.20180530.0+1 253:8 0 50.4G 0 lvm / │ ├─onn_ovirt--node00-home 253:10 0 1G 0 lvm /home │ ├─onn_ovirt--node00-tmp 253:11 0 1G 0 lvm /tmp │ ├─onn_ovirt--node00-var 253:12 0 15G 0 lvm /var │ ├─onn_ovirt--node00-var_log 253:13 0 8G 0 lvm /var/log │ ├─onn_ovirt--node00-var_log_audit 253:14 0 2G 0 lvm /var/log/audit │ └─onn_ovirt--node00-var_crash 253:16 0 10G 0 lvm /var/crash └─onn_ovirt--node00-pool00_tdata 253:6 0 77.4G 0 lvm └─onn_ovirt--node00-pool00-tpool 253:7 0 77.4G 0 lvm ├─onn_ovirt--node00-pool00 253:0 0 77.4G 0 lvm ├─onn_ovirt--node00-ovirt--node--ng--4.2.3.1--0.20180530.0+1 253:8 0 50.4G 0 lvm / ├─onn_ovirt--node00-home 253:10 0 1G 0 lvm /home ├─onn_ovirt--node00-tmp 253:11 0 1G 0 lvm /tmp ├─onn_ovirt--node00-var 253:12 0 15G 0 lvm /var ├─onn_ovirt--node00-var_log 253:13 0 8G 0 lvm /var/log ├─onn_ovirt--node00-var_log_audit 253:14 0 2G 0 lvm /var/log/audit └─onn_ovirt--node00-var_crash 253:16 0 10G 0 lvm /var/crash sdb 8:16 0 894.3G 0 disk nvme0n1 259:0 0 465.8G 0 disk
- output of "lsinitrd"
Based on your next email, looks ok. Next step: 1. Change the blacklist to: blacklist { wwid INTEL_SSDSCKHB120G4_BTWM65160025120B wwid eui.0025385171b04d62 wwid SAMSUNG_MZ7GE960HMHP-000AZ_S1P8NYAG123827 } 2. Update initrd dracut --force --add multipath --include /etc/multipath /etc/multipath 3. Reboot The manual suggest to remove the multiapth device using multipath -f INTEL_SSDSCKHB120G4_BTWM65160025120B but I don't think this can work when your root file system is on top of logical volumes using this device. If multipath still grabs the device, we will need output of "journalctl -b" to understand what happened during early boot process. Nir
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/NGN6FNI4RY4NJ3...

After making the changes & rebooting, multipath is still grabbing those devices. Here is "journalctl -b" https://paste.fedoraproject.org/paste/3K3S4wQ~~qAlhTmgazRDAw

On Thu, Jun 7, 2018 at 6:12 PM femi adegoke <ovirt@fateknollogee.com> wrote:
After making the changes & rebooting, multipath is still grabbing those devices. Here is "journalctl -b"
https://paste.fedoraproject.org/paste/3K3S4wQ~~qAlhTmgazRDAw
It seems that it ignores the blacklist during boot, and once the map is used, it cannot be removed. Can you file a bug? Please attach all the file you sent in this thread, and log from the last boot you attach here. Nir
_______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/UMBINR7TXOL7IS...

Ok, Nir thanks for the update. I have one more identical host (using the same hardware), let me install oVirt node & verify that I'm seeing the same behavior. I will update you before I file the bug.

Nir, I am not 100% sure this is a bug. I repeated the install on the same hardware & the results are the same - multipath devices that will not go away. Both installs were done on a SuperMicro X10SDV-8C-TLN4F I then did another install on a SuperMicro X10DRi-T, this install has no problem & it doesn't need a blacklist. Here is some fpaste info for both installs. https://paste.fedoraproject.org/paste/lA0jwRkv32AqaV3pC53YuQ Let me know if you want me to gather some more data.

On Fri, Jun 8, 2018 at 11:01 AM femi adegoke <ovirt@fateknollogee.com> wrote:
Nir,
I am not 100% sure this is a bug.
I repeated the install on the same hardware & the results are the same - multipath devices that will not go away. Both installs were done on a SuperMicro X10SDV-8C-TLN4F
I then did another install on a SuperMicro X10DRi-T, this install has no problem & it doesn't need a blacklist.
The fact that you don't need a blacklist with one device does not mean that failing to respect the blacklist with another device that needs to be blacklisted is not a bug. Please file a multipath bug for your distribution, and include all the logs you shared here. Please also include the contents of /etc/multipath/wwids file. I suspect that the issue that blacklist is ignored if a device is listed in /etc/mutipath/wwids. Nir
Here is some fpaste info for both installs. https://paste.fedoraproject.org/paste/lA0jwRkv32AqaV3pC53YuQ
Let me know if you want me to gather some more data. _______________________________________________ Users mailing list -- users@ovirt.org To unsubscribe send an email to users-leave@ovirt.org Privacy Statement: https://www.ovirt.org/site/privacy-policy/ oVirt Code of Conduct: https://www.ovirt.org/community/about/community-guidelines/ List Archives: https://lists.ovirt.org/archives/list/users@ovirt.org/message/JAZ3YWEU7YWFIC...


Hi Femi, I'm having the same/similar issue that I just posted about - did you ever resolve it? Regards Simon...

The bugzilla URL says that's a private/internal only bug. You are not authorized to access bug #1016535. Most likely the bug has been restricted for internal development processes and we cannot grant access. If you are a Red Hat customer with an active subscription, please visit the Red Hat Customer Portal for assistance with your issue If you are a Fedora Project user and require assistance, please consider using one of the mailing lists we host for the Fedora Project
participants (4)
-
femi adegoke
-
Nir Soffer
-
simon@justconnect.ie
-
tadavis@lbl.gov