[ovirt-users] can not use iscsi storage type onovirtandGlusterfshyper-converged environment
Fred Rolland
frolland at redhat.com
Mon Dec 5 08:44:56 UTC 2016
Hi,
After changing the code, you need to restart the VDSM service to take
effect.
Regards,
Fred
On Mon, Dec 5, 2016 at 10:28 AM, 胡茂荣 <maorong.hu at horebdata.cn> wrote:
>
> Thanks for Yaniv Kaul , change code need build vdsm source code , and
> if only change /usr/share/vdsm/storage/devicemapper.py will not really
> take effect .
>
> could this problem as a bug , and correct it to ovirt vdsm source code
> ?
>
> ------------------ Original ------------------
> *From: * "Yaniv Kaul"<ykaul at redhat.com>;
> *Date: * Sun, Dec 4, 2016 07:07 PM
> *To: * "胡茂荣"<maorong.hu at horebdata.cn>;
> *Cc: * "胡晓宇"<samuel.xhu at horebdata.cn>; "users"<users at ovirt.org>; "Sahina
> Bose"<sabose at redhat.com>; "Jeff Nelson"<jenelson at redhat.com>;
> *Subject: * Re: [ovirt-users] can not use iscsi storage type
> onovirtandGlusterfshyper-converged environment
>
>
>
> On Dec 2, 2016 11:53 AM, "胡茂荣" <maorong.hu at horebdata.cn> wrote:
>
> I find supervdsm used " /usr/sbin/dmsetup status" :
>
> MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02
> 17:12:16,372::supervdsmServer::92::SuperVdsm.ServerCallback::(wrapper)
> call getPathsStatus with () {}
> MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02
> 17:12:16,373::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus)
> /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status (cwd None)
> MainProcess|jsonrpc.Executor/2::DEBUG::2016-12-02
> 17:12:16,377::devicemapper::154::Storage.Misc.excCmd::(_getPathsStatus)
> SUCCESS: <err> = ''; <rc> = 0
> MainProcess|jsonrpc.Executor/2::ERROR::2016-12-02
> 17:12:16,378::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper) Error
> in getPathsStatus
>
> problem :
> how can I change Storage.Misc.excCmd _getPathsStatus " /usr/bin/taskset
> --cpu-list 0-7 /usr/sbin/dmsetup status" to :
>
> /usr/bin/taskset --cpu-list 0-7 /usr/sbin/dmsetup status --target
> multipah
>
> I think add iscsi type storage , if supervdsm scan mulitpah will solve
> my problem .(my environment have other dm devices, use "dmsetup status"
> will show them, and vdsm get dm path status will occur error )
>
> ============================================================
> ====================
> so I changed some as follow :
> （1）
> I define EXT_DMSETUP_STATUS in /usr/lib/python2.7/site-packages/vdsm/constants.py
> :
>
> /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP =
> '/usr/sbin/dmsetup'
> /usr/lib/python2.7/site-packages/vdsm/constants.py:EXT_DMSETUP_STATUS =
> "/usr/sbin/dmsetup status --target multipath"
>
> (2)
> /usr/share/vdsm/storage/devicemapper.py add :
> from vdsm.constants import EXT_DMSETUP_STATUS
>
> and changed getPathsStatus cmd to " EXT_DMSETUP_STATUS" :
>
> def _getPathsStatus():
> cmd = [EXT_DMSETUP_STATUS] ##### before : cmd=[
> EXT_DMSETUP,"status"]
>
>
> Why not change this to:
> cmd = [EXT_DMSETUP, "status", "--target", "multipath"]
>
> Y.
>
> rc, out, err = misc.execCmd(cmd)
> ============================================================
> ===========================
>
> but log in supervdsm log also not change . Please help me ，how to change
> code to let supervdsm exec "/usr/sbin/dmsetup status --target multipath"
> in function getPathsStatus（） 。
>
>
>
>
> ------------------ Original ------------------
> *From: * "胡茂荣"<maorong.hu at horebdata.cn>;
> *Date: * Fri, Nov 25, 2016 05:44 PM
> *To: * "Sahina Bose"<sabose at redhat.com>;
> *Cc: * "Maor Lipchuk"<mlipchuk at redhat.com>; "Jeff Nelson"<
> jenelson at redhat.com>; "users"<users at ovirt.org>;
> *Subject: * Re: [ovirt-users] can not use iscsi storage type on
> ovirtandGlusterfshyper-converged environment
>
>
> ===================================---
>
> ###vdsm or supervdsm log report :
>
> MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01
> 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper)
> Error in getPathsStatus
>
> MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01
> 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper)
> Error in getPathsStatus
> ============================ some code info ------------>
> [root at horeba storage]# pwd
> /usr/share/vdsm/storage
>
> [root at horeba storage]# grep "getPathsStatus" -R ./
> ./devicemapper.py:def _getPathsStatus():
> ./devicemapper.py:def getPathsStatus():
> ./devicemapper.py: return getProxy().getPathsStatus()
> ./multipath.py: pathStatuses = devicemapper.getPathsStatus()
>
> def _getPathsStatus():
> cmd = [EXT_DMSETUP, "status"]
> rc, out, err = misc.execCmd(cmd)
> if rc != 0:
> raise Exception("Could not get device statuses")
>
> res = {}
> for statusLine in out:
> try:
> devName, statusLine = statusLine.split(":", 1)
> except ValueError:
> if len(out) == 1:
> # return an empty dict when status output is: No devices
> found
> return res
> else:
> raise
>
> for m in PATH_STATUS_RE.finditer(statusLine):
> devNum, status = m.groups()
> physdevName = findDev(*[int(i) for i in devNum.split(":")])
> res[physdevName] = {"A": "active", "F": "failed"}[status]
>
> return res
> def getPathsStatus():
> return getProxy().getPathsStatus()
> =============================================
> and flashcache dm device will error when use getPathsStatus() function
> . could change code not check flashcache dm device ?
> ========================================dmsetup info ----------->
> [root at horebc ~]# dmsetup status
> flash_sdb: 0 976771072 flashcache stats:
> reads(1388761), writes(15548965)
> read hits(1235671), read hit percent(88)
> write hits(6539144) write hit percent(42)
> dirty write hits(21372) dirty write hit percent(0)
> replacement(147711), write replacement(524881)
> write invalidates(0), read invalidates(1)
> pending enqueues(810), pending inval(810)
> metadata dirties(15196370), metadata cleans(15196322)
> metadata batch(30087377) metadata ssd writes(305315)
> cleanings(15196322) fallow cleanings(48187)
> no room(337139) front merge(716153) back merge(14391395)
> force_clean_block(0)
> disk reads(153093), disk writes(15530535) ssd reads(16431974) ssd
> writes(15672221)
> uncached reads(3714), uncached writes(334235), uncached IO
> requeue(0)
> disk read errors(0), disk write errors(0) ssd read errors(0) ssd
> write errors(0)
> uncached sequential reads(0), uncached sequential writes(0)
> pid_adds(0), pid_dels(0), pid_drops(0) pid_expiry(0)
> lru hot blocks(12158976), lru warm blocks(12158976)
> lru promotions(0), lru demotions(0)
> dedbd337--ca66--43ff--b78c--4e9347682a9c-metadata: 0 1048576 linear
> dedbd337--ca66--43ff--b78c--4e9347682a9c-leases: 0 4194304 linear
> 23137643634356633: 0 2147483648 multipath 2 0 0 0 1 1 A 0 1 2 8:128 A 0 0
> 1
> dedbd337--ca66--43ff--b78c--4e9347682a9c-master: 0 2097152 linear
> dedbd337--ca66--43ff--b78c--4e9347682a9c-inbox: 0 262144 linear
> dedbd337--ca66--43ff--b78c--4e9347682a9c-outbox: 0 262144 linear
>
> [root at horebc ~]# dmsetup info -C
> Name Maj Min Stat Open Targ
> Event UUID
> flash_sdb 253 0 L--w 1 1
> 0
> dedbd337--ca66--43ff--b78c--4e9347682a9c-metadata 253 4 L--w 0 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGIGwifAPyYj9GmjFzCmJkIf9vF
> FFHn9n7V
> dedbd337--ca66--43ff--b78c--4e9347682a9c-leases 253 6 L--w 0 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGIVSCllWEYYKziY1bSeiTL0dAK
> Ad27JqDT
> 23137643634356633 253 3 L--w 6 1
> 0 mpath-23137643634356633
> dedbd337--ca66--43ff--b78c--4e9347682a9c-master 253 9 L--w 0 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGIiEnFZklRhZfFZ4YRdYWFImKW
> sUGr5pHg
> dedbd337--ca66--43ff--b78c--4e9347682a9c-inbox 253 8 L--w 0 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGILobNK8KRD4SzDWyg50aG7jGd
> cNAi3KNw
> dedbd337--ca66--43ff--b78c--4e9347682a9c-outbox 253 5 L--w 0 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGIAvbT8CLegbVL802bG3QgLtH7
> I7llmS6R
> flash_sdf 253 2 L--w 1 1
> 0
> dedbd337--ca66--43ff--b78c--4e9347682a9c-ids 253 7 L--w 1 1
> 0 LVM-zSZKhQXbs2Ms5ceM2Wlt3p46BgsS8gGIIkeaFaWvxa9wbHy7WrfiXNsP
> 4F2J3gg0
> flash_sdd 253 1 L--w 1 1
> 0
>
> ====================================================================
> and flashcache dm have no UUID ，I think it can export it before checking
> .
>
>
> humaorong
> 2016-11-25
>
>
>
> ------------------ Original ------------------
> *From: * "胡茂荣"<maorong.hu at horebdata.cn>;
> *Date: * Fri, Nov 25, 2016 01:18 PM
> *To: * "Sahina Bose"<sabose at redhat.com>;
> *Cc: * "Maor Lipchuk"<mlipchuk at redhat.com>; "Jeff Nelson"<
> jenelson at redhat.com>; "users"<users at ovirt.org>;
> *Subject: * Re: [ovirt-users] can not use iscsi storage type on
> ovirtandGlusterfshyper-converged environment
>
>
> I find this problem more info :
> I use flashcache on ovirt hosts , have dm device in /dev/mapper/ :
>
> [root at horeba init.d]# dmsetup info -C （and I set them in multipath
> blacklist ）
> Name Maj Min Stat Open Targ Event UUID
>
> flash_sdb 253 0 L--w 0 1 0
>
> flash_sdf 253 2 L--w 0 1 0
>
> flash_sdd 253 1 L--w 0 1 0
> [root at horeba init.d]# multipath -l
> [root at horeba init.d]#
>
> [root at horeba init.d]# ll /dev/mapper/
> total 0
> crw------- 1 root root 10, 236 Nov 25 10:09 control
> lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdb -> ../dm-0
> lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdd -> ../dm-1
> lrwxrwxrwx 1 root root 7 Nov 25 12:51 flash_sdf -> ../dm-2
>
> on this condition , ovirt UI add iscsi type storage will fail .
>
> If I delete the flashcache device /dev/mapper/flash_* , ovirt UI add
> iscsi type storage have no problem .
>
> I need flashcache for using ssd cache on my environment , how can I
> use iscsi type storage on this environment , please help me , Thanks！
>
>
> ------------------ Original ------------------
> *From: * "Sahina Bose"<sabose at redhat.com>;
> *Date: * Thu, Nov 3, 2016 05:54 PM
> *To: * "胡茂荣"<maorong.hu at horebdata.cn>;
> *Cc: * "Maor Lipchuk"<mlipchuk at redhat.com>; "Jeff Nelson"<
> jenelson at redhat.com>; "users"<users at ovirt.org>;
> *Subject: * Re: [ovirt-users] can not use iscsi storage type on
> ovirtandGlusterfshyper-converged environment
>
> A wild guess, not sure if it is related - can you check if multipathd
> service is enabled. If you set up your oVirt-Gluster hyperconverged
> environment via gdeploy, multipathd service is disabled and the
> /etc/multipath.conf is edited to blacklist all devices - this was to fix
> https://bugzilla.redhat.com/show_bug.cgi?id=1303940
>
> Since you mention you're unable to add iscsi storage only in this
> environment, thought it's worth checking.
>
> On Thu, Nov 3, 2016 at 6:40 AM, 胡茂荣 <maorong.hu at horebdata.cn> wrote:
>
>>
>> my environment rpm are :
>> [root at horeba ~]# rpm -q vdsm
>> vdsm-4.18.13-1.el7.centos.x86_64
>>
>> [root at horeba ~]# rpm -aq | grep ovirt
>> ovirt-vmconsole-host-1.0.4-1.el7.centos.noarch
>> ovirt-imageio-common-0.4.0-1.el7.noarch
>> ovirt-hosted-engine-setup-2.0.2.2-1.el7.centos.noarch
>> ovirt-imageio-daemon-0.4.0-1.el7.noarch
>> ovirt-engine-appliance-4.0-20160928.1.el7.centos.noarch
>> ovirt-vmconsole-1.0.4-1.el7.centos.noarch
>> ovirt-host-deploy-1.5.2-1.el7.centos.noarch
>> ovirt-hosted-engine-ha-2.0.4-1.el7.centos.noarch
>> ovirt-release40-4.0.4-1.noarch
>> ovirt-setup-lib-1.0.2-1.el7.centos.noarch
>> ovirt-engine-sdk-python-3.6.9.1-1.el7.centos.noarch
>>
>> and I test if not on ' ovirt and Glusterfs hyper-converged environment '
>> , ovirt WEB UI add iscsi storage work ok .
>>
>>
>> ------------------ Original ------------------
>> *From: * "Maor Lipchuk"<mlipchuk at redhat.com>;
>> *Date: * Wed, Nov 2, 2016 07:37 PM
>> *To: * "胡茂荣"<maorong.hu at horebdata.cn>;
>> *Cc: * "users"<users at ovirt.org>; "Jeff Nelson"<jenelson at redhat.com>;
>> "Nir Soffer"<nsoffer at redhat.com>;
>> *Subject: * Re: [ovirt-users] can not use iscsi storage type on
>> ovirtandGlusterfs hyper-converged environment
>>
>> Thanks for the logs,
>>
>> What kind of VDSM version are you using?
>> "rpm -q vdsm"
>> There seems to be a similar issue which was reported recently in the VDSM
>> area
>> (see https://bugzilla.redhat.com/show_bug.cgi?id=1197292)
>> It should be fixed in later versions of VDSM vdsm-4.16.12-2.el7ev.x86_64
>> Adding also Nir and Jeff to the thread, if they have any insights
>>
>> Regards,
>> Maor
>>
>> On Wed, Nov 2, 2016 at 4:11 AM, 胡茂荣 <maorong.hu at horebdata.cn> wrote:
>>
>>>
>>> Hi Maor:
>>> vdsm/supervdsm/engine log on attachment . I mkfs.xfs the lun
>>> block device and mount to /mnt , dd write ,dmesg not report error ,dd
>>> result is ok :
>>>
>>> /dev/sdi 50G 33M 50G 1% /mnt
>>>
>>> [root at horebc mnt]# for i in `seq 3`; do dd if=/dev/zero of=./file
>>> bs=1G count=1 oflag=direct ; done
>>> 1+0 records in
>>> 1+0 records out
>>> 1073741824 bytes (1.1 GB) copied, 13.3232 s, 80.6 MB/s
>>> 1+0 records in
>>> 1+0 records out
>>> 1073741824 bytes (1.1 GB) copied, 9.89988 s, 108 MB/s
>>> 1+0 records in
>>> 1+0 records out
>>> 1073741824 bytes (1.1 GB) copied, 14.0143 s, 76.6 MB/s
>>>
>>> my envirnment have three network segments (hosts have 3 network
>>> segments ) :
>>> engine and glusterfs mount : 192.168.11.X/24
>>> glusterfs brick : 192.168.10.x/24
>>> iscsi : 192.168.1.0/24
>>>
>>> and I add 192.168.1.0/24 to engine vm , ovirt web UI report the
>>> same error .
>>>
>>> humaorong
>>> 2016-11-2
>>>
>>> ------------------ Original ------------------
>>> *From: * "Maor Lipchuk"<mlipchuk at redhat.com>;
>>> *Date: * Tue, Nov 1, 2016 08:14 PM
>>> *To: * "胡茂荣"<maorong.hu at horebdata.cn>;
>>> *Cc: * "users"<users at ovirt.org>;
>>> *Subject: * Re: [ovirt-users] can not use iscsi storage type on ovirt
>>> andGlusterfs hyper-converged environment
>>>
>>> Hi 胡茂荣Can u please also add the VDSM and engine logs.
>>> If you try discover and connect to those luns directly from your Host
>>> does it work?
>>>
>>> Regards,
>>> Maor
>>>
>>>
>>> On Tue, Nov 1, 2016 at 6:12 AM, 胡茂荣 <maorong.hu at horebdata.cn> wrote:
>>>
>>>>
>>>>
>>>> on ovirt and Glusterfs hyper-converged environment , can not use
>>>> iscsi storage type , UI report error: "Could not retrieve LUNs, please
>>>> check your storage." , vdsm report :"VDSM hosted_engine_3 command
>>>> failed: Error block device action: ()" .
>>>> but this block device alse login on centos 7 host :
>>>> =============================================================
>>>>
>>>> ## lsscsi
>>>>
>>>> [7:0:0:0] disk SCST_BIO DEVFOR_OVIRT_rbd 221 /dev/sdi
>>>>
>>>> ## dmesg :
>>>>
>>>> [684521.131186] sd 7:0:0:0: [sdi] Attached SCSI disk
>>>>
>>>> ===================================---
>>>>
>>>> ###vdsm or supervdsm log report :
>>>>
>>>> MainProcess|jsonrpc.Executor/7::ERROR::2016-11-01
>>>> 11:07:00,178::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper)
>>>> Error in getPathsStatus
>>>>
>>>> MainProcess|jsonrpc.Executor/4::ERROR::2016-11-01
>>>> 11:07:20,964::supervdsmServer::96::SuperVdsm.ServerCallback::(wrapper)
>>>> Error in getPathsStatus
>>>>
>>>> jsonrpc.Executor/4::DEBUG::2016-11-01 11:07:04,251::iscsi::434::Storage.ISCSI::(rescan)
>>>> Performing SCSI scan, this will take up to 30 seconds
>>>>
>>>> jsonrpc.Executor/5::INFO::2016-11-01 11:07:19,413::iscsi::567::Stor
>>>> age.ISCSI::(setRpFilterIfNeeded) iSCSI iface.net_ifacename not
>>>> provided. Skipping.
>>>>
>>>> 11:09:15,753::iscsiadm::119::Storage.Misc.excCmd::(_runCmd)
>>>> /usr/bin/taskset --cpu-list 0-7 /usr/bin/sudo -n /usr/sbin/iscsiadm -m
>>>> session -R (cwd None)
>>>>
>>>> ======================================
>>>>
>>>> the other info please the attachment "bug-info.doc".
>>>>
>>>> this prolem on ovirt3.6 and 4.X ovirt and Glusterfs
>>>> hyper-converged environment . how can I use iscsi storage type on ovirt
>>>> and Glusterfs hyper-converged environment .Please help me !
>>>>
>>>> humaorong
>>>>
>>>> 2016-11-1
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Users mailing list
>>>> Users at ovirt.org
>>>> http://lists.ovirt.org/mailman/listinfo/users
>>>>
>>>>
>>>
>>
>> _______________________________________________
>> Users mailing list
>> Users at ovirt.org
>> http://lists.ovirt.org/mailman/listinfo/users
>>
>>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
>
> _______________________________________________
> Users mailing list
> Users at ovirt.org
> http://lists.ovirt.org/mailman/listinfo/users
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.ovirt.org/pipermail/users/attachments/20161205/52eafd68/attachment-0001.html>
More information about the Users
mailing list