[Kimchi-devel] [PATCH v5 4/5] storagepool: Support Creating iSCSI storagepool in model.py
Sheldon
shaohef at linux.vnet.ibm.com
Fri Jan 3 06:08:33 UTC 2014
Reviewed-by: ShaoHe Feng <shaohef at linux.vnet.ibm.com>
On 01/02/2014 05:58 PM, Zhou Zheng Sheng wrote:
> This patch implements creating iSCSI storagepool for libvirt.
> Each LUN in iSCSI storagepool can be used as a volume, but iSCSI
> storagepool does not provide ability to create volume. For now in kimchi
> we create volume for each newly created VM. Next is to implement
> attaching existing volume to a new VM.
>
> This patch also adds validation for the iSCSI back-end. It tries to
> login the target using the auth before actually creating the pool.
> On RedHat family distributions, libiscsi comes with a Python binding,
> but not on other distributions. So in this patch it invokes iscsiadm to
> check iSCSI target.
>
> As regard to iSCSI CHAP authentication, it's somewhat broken before
> libvirt 1.1.1. It was reworked in libvirt commit eb0d79, and this patch
> goes with libvirt 1.1.1, but it doesn't require libvirt in the spec
> file. This is because libvirt 1.1.1 is not available on Ubuntu 12.04
> LTS. The user can just ignore this feature.
>
> How to test it manually
>
> Prerequisite
> An running iSCSI target on the network.
>
> Create:
> curl -u root -H 'Content-type: application/json' \
> -H 'Accept: application/json' \
> -d '{"source": {
> "target": "iqn.YouriSCSITargetIQN.XXX",
> "host": "10.0.0.X"},
> "type": "iscsi",
> "name": "iscsipool"}' \
> http://127.0.0.1:8000/storagepools
>
> Show Info:
> curl -u root -H 'Accept: application/json' \
> http://127.0.0.1:8000/storagepools/iscsipool
>
> Activate:
> curl -u root -H 'Content-type: application/json' \
> -H 'Accept: application/json' \
> -d '' http://127.0.0.1:8000/storagepools/iscsipool/activate
>
> Examine:
> iscsiadm -m session
>
> Deactivate:
> curl -u root -H 'Content-type: application/json' \
> -H 'Accept: application/json' \
> -d '' http://127.0.0.1:8000/storagepools/iscsipool/deactivate
>
> Delete:
> curl -u root -X DELETE -H 'Accept: application/json' \
> http://127.0.0.1:8000/storagepools/iscsipool
>
> v2 -> v3
> 1) Support CHAP authentication.
> 2) Validate iSCSI target before creating the pool.
> 3) Add XML generation unit test with "port" and "auth".
>
> v3 -> v4
> 1) Update RPM spec file and Debian control file.
> 2) Update API.json.
> 3) Give a proper name for the iSCSI target client class, TargetClient.
>
> v4 -> v5
> Fix description string of 'auth' in API.json.
>
> Signed-off-by: Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
> ---
> Makefile.am | 1 +
> contrib/DEBIAN/control.in | 3 +-
> contrib/kimchi.spec.fedora.in | 1 +
> contrib/kimchi.spec.suse.in | 1 +
> docs/API.md | 12 +++++-
> src/kimchi/API.json | 24 ++++++++++++
> src/kimchi/Makefile.am | 1 +
> src/kimchi/iscsi.py | 89 +++++++++++++++++++++++++++++++++++++++++++
> src/kimchi/model.py | 89 +++++++++++++++++++++++++++++++++++++++++--
> tests/test_storagepool.py | 63 ++++++++++++++++++++++++++++++
> 10 files changed, 278 insertions(+), 6 deletions(-)
> create mode 100644 src/kimchi/iscsi.py
>
> diff --git a/Makefile.am b/Makefile.am
> index 01efa1b..b01a8e7 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -47,6 +47,7 @@ PEP8_WHITELIST = \
> src/kimchi/config.py.in \
> src/kimchi/disks.py \
> src/kimchi/featuretests.py \
> + src/kimchi/iscsi.py \
> src/kimchi/rollbackcontext.py \
> src/kimchi/root.py \
> src/kimchi/server.py \
> diff --git a/contrib/DEBIAN/control.in b/contrib/DEBIAN/control.in
> index 380584c..eecfb27 100644
> --- a/contrib/DEBIAN/control.in
> +++ b/contrib/DEBIAN/control.in
> @@ -17,7 +17,8 @@ Depends: python-cherrypy3 (>= 3.2.0),
> python-psutil (>= 0.6.0),
> python-ethtool,
> sosreport,
> - python-ipaddr
> + python-ipaddr,
> + open-iscsi
> Build-Depends:
> Maintainer: Aline Manera <alinefm at br.ibm.com>
> Description: Kimchi web server
> diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in
> index 3044fc8..75435b3 100644
> --- a/contrib/kimchi.spec.fedora.in
> +++ b/contrib/kimchi.spec.fedora.in
> @@ -25,6 +25,7 @@ Requires: python-ethtool
> Requires: sos
> Requires: python-ipaddr
> Requires: nfs-utils
> +Requires: iscsi-initiator-utils
>
> %if 0%{?rhel} == 6
> Requires: python-ordereddict
> diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in
> index 190b2be..bcfb6db 100644
> --- a/contrib/kimchi.spec.suse.in
> +++ b/contrib/kimchi.spec.suse.in
> @@ -20,6 +20,7 @@ Requires: python-jsonschema >= 1.3.0
> Requires: python-ethtool
> Requires: python-ipaddr
> Requires: nfs-client
> +Requires: iscsi-initiator-utils
>
> %if 0%{?sles_version} == 11
> Requires: python-ordereddict
> diff --git a/docs/API.md b/docs/API.md
> index a8b9ea0..c347340 100644
> --- a/docs/API.md
> +++ b/docs/API.md
> @@ -182,17 +182,25 @@ Represents a snapshot of the Virtual Machine's primary monitor.
> * **POST**: Create a new Storage Pool
> * name: The name of the Storage Pool.
> * type: The type of the defined Storage Pool.
> - Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical'
> + Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical', 'iscsi'
> * path: The path of the defined Storage Pool.
> For 'kimchi-iso' pool refers to targeted deep scan path.
> Pool types: 'dir', 'kimchi-iso'.
> * source: Dictionary containing source information of the pool.
> * host: IP or hostname of server for a pool backed from a remote host.
> - Pool types: 'netfs'.
> + Pool types: 'netfs', 'iscsi'.
> * path: Export path on NFS server for NFS pool.
> Pool types: 'netfs'.
> * devices: Array of devices to be used in the Storage Pool
> Pool types: 'logical'.
> + * target: Target IQN of an iSCSI pool.
> + Pool types: 'iscsi'.
> + * port *(optional)*: Listening port of a remote storage server.
> + Pool types: 'iscsi'.
> + * auth *(optional)*: Storage back-end authentication information.
> + Pool types: 'iscsi'.
> + * username: Login username of the iSCSI target.
> + * password: Login password of the iSCSI target.
>
> ### Resource: Storage Pool
>
> diff --git a/src/kimchi/API.json b/src/kimchi/API.json
> index 6519c21..6f1dd03 100644
> --- a/src/kimchi/API.json
> +++ b/src/kimchi/API.json
> @@ -44,6 +44,30 @@
> "description": "Full path of the block device node",
> "type": "string"
> }
> + },
> + "target": {
> + "description": "Target IQN of an iSCSI pool",
> + "type": "string"
> + },
> + "port": {
> + "description": "Listening port of a remote storage server",
> + "type": "integer",
> + "minimum": 1,
> + "maximum": 65535
> + },
> + "auth": {
> + "description": "Storage back-end authentication information",
> + "type": "object",
> + "properties": {
> + "username": {
> + "description": "Login username of the iSCSI target",
> + "type": "string"
> + },
> + "password": {
> + "description": "Login password of the iSCSI target",
> + "type": "string"
> + }
> + }
> }
> }
> }
> diff --git a/src/kimchi/Makefile.am b/src/kimchi/Makefile.am
> index 88ccbf7..261c331 100644
> --- a/src/kimchi/Makefile.am
> +++ b/src/kimchi/Makefile.am
> @@ -31,6 +31,7 @@ kimchi_PYTHON = \
> distroloader.py \
> exception.py \
> featuretests.py \
> + iscsi.py \
> isoinfo.py \
> mockmodel.py \
> model.py \
> diff --git a/src/kimchi/iscsi.py b/src/kimchi/iscsi.py
> new file mode 100644
> index 0000000..35c0b8a
> --- /dev/null
> +++ b/src/kimchi/iscsi.py
> @@ -0,0 +1,89 @@
> +#
> +# Project Kimchi
> +#
> +# Copyright IBM, Corp. 2013
> +#
> +# Authors:
> +# Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA
> +
> +import subprocess
> +
> +
> +from kimchi.exception import OperationFailed
> +
> +
> +class TargetClient(object):
> + def __init__(self, target, host, port=None, auth=None):
> + self.portal = host + ("" if port is None else ":%s" % port)
> + self.target = target
> + self.auth = auth
> + self.targetCmd = ['iscsiadm', '--mode', 'node', '--targetname',
> + self.target, '--portal', self.portal]
> +
> + def _update_db(self, Name, Value):
> + self._run_cmd(['--op=update', '--name', Name, '--value', Value])
> +
> + def _update_auth(self):
> + if self.auth is None:
> + items = (('node.session.auth.authmethod', 'None'),
> + ('node.session.auth.username', ''),
> + ('node.session.auth.password', ''))
> + else:
> + items = (('node.session.auth.authmethod', 'CHAP'),
> + ('node.session.auth.username', self.auth['username']),
> + ('node.session.auth.password', self.auth['password']))
> + for name, value in items:
> + self._update_db(name, value)
> +
> + def _run_cmd(self, cmd):
> + iscsiadm = subprocess.Popen(
> + self.targetCmd + cmd,
> + stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> + out, err = iscsiadm.communicate()
> + if iscsiadm.returncode != 0:
> + raise OperationFailed('Error executing iscsiadm: %s' % err)
> + return out
> +
> + def _discover(self):
> + iscsiadm = subprocess.Popen(
> + ['iscsiadm', '--mode', 'discovery', '--type', 'sendtargets',
> + '--portal', self.portal],
> + stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> + out, err = iscsiadm.communicate()
> + if iscsiadm.returncode != 0:
> + raise OperationFailed('Error executing iscsiadm: %s' % err)
> + return out
> +
> + def _run_op(self, op):
> + self._run_cmd(['--' + op])
> +
> + def login(self):
> + self._discover()
> + self._update_auth()
> + self._run_op('login')
> +
> + def logout(self):
> + self._run_op('logout')
> +
> + def validate(self):
> + try:
> + self.login()
> + except OperationFailed:
> + return False
> +
> + self.logout()
> + return True
> diff --git a/src/kimchi/model.py b/src/kimchi/model.py
> index 2b46121..17d5485 100644
> --- a/src/kimchi/model.py
> +++ b/src/kimchi/model.py
> @@ -65,6 +65,7 @@ from kimchi.distroloader import DistroLoader
> from kimchi.exception import InvalidOperation, InvalidParameter, MissingParameter
> from kimchi.exception import NotFoundError, OperationFailed
> from kimchi.featuretests import FeatureTests
> +from kimchi.iscsi import TargetClient
> from kimchi.networkxml import to_network_xml
> from kimchi.objectstore import ObjectStore
> from kimchi.scan import Scanner
> @@ -995,7 +996,7 @@ class Model(object):
> if params['type'] == 'kimchi-iso':
> task_id = self._do_deep_scan(params)
> poolDef = StoragePoolDef.create(params)
> - poolDef.prepare()
> + poolDef.prepare(conn)
> xml = poolDef.xml
> except KeyError, key:
> raise MissingParameter(key)
> @@ -1444,7 +1445,7 @@ class StoragePoolDef(object):
> def __init__(self, poolArgs):
> self.poolArgs = poolArgs
>
> - def prepare(self):
> + def prepare(self, conn):
> ''' Validate pool arguments and perform preparations. Operation which
> would cause side effect should be put here. Subclasses can optionally
> override this method, or it always succeeds by default. '''
> @@ -1487,7 +1488,7 @@ class NetfsPoolDef(StoragePoolDef):
> super(NetfsPoolDef, self).__init__(poolArgs)
> self.path = '/var/lib/kimchi/nfs_mount/' + self.poolArgs['name']
>
> - def prepare(self):
> + def prepare(self, conn):
> # TODO: Verify the NFS export can be actually mounted.
> pass
>
> @@ -1550,6 +1551,88 @@ class LogicalPoolDef(StoragePoolDef):
> return xml
>
>
> +class IscsiPoolDef(StoragePoolDef):
> + poolType = 'iscsi'
> +
> + def prepare(self, conn):
> + source = self.poolArgs['source']
> + if not TargetClient(**source).validate():
> + raise OperationFailed("Can not login to iSCSI host %s target %s" %
> + (source['host'], source['target']))
> + self._prepare_auth(conn)
> +
> + def _prepare_auth(self, conn):
> + try:
> + auth = self.poolArgs['source']['auth']
> + except KeyError:
> + return
> +
> + try:
> + virSecret = conn.secretLookupByUsage(
> + libvirt.VIR_SECRET_USAGE_TYPE_ISCSI, self.poolArgs['name'])
> + except libvirt.libvirtError:
> + xml = '''
> + <secret ephemeral='no' private='yes'>
> + <description>Secret for iSCSI storage pool {name}</description>
> + <auth type='chap' username='{username}'/>
> + <usage type='iscsi'>
> + <target>{name}</target>
> + </usage>
> + </secret>'''.format(name=self.poolArgs['name'],
> + username=auth['username'])
> + virSecret = conn.secretDefineXML(xml)
> +
> + virSecret.setValue(auth['password'])
> +
> + def _format_port(self, poolArgs):
> + try:
> + port = poolArgs['source']['port']
> + except KeyError:
> + return ""
> + return "port='%s'" % port
> +
> + def _format_auth(self, poolArgs):
> + try:
> + auth = poolArgs['source']['auth']
> + except KeyError:
> + return ""
> +
> + return '''
> + <auth type='chap' username='{username}'>
> + <secret type='iscsi' usage='{name}'/>
> + </auth>'''.format(name=poolArgs['name'], username=auth['username'])
> +
> + @property
> + def xml(self):
> + # Required parameters
> + # name:
> + # type:
> + # source[host]:
> + # source[target]:
> + #
> + # Optional parameters
> + # source[port]:
> + poolArgs = copy.deepcopy(self.poolArgs)
> + poolArgs['source'].update({'port': self._format_port(poolArgs),
> + 'auth': self._format_auth(poolArgs)})
> + poolArgs['path'] = '/dev/disk/by-id'
> +
> + xml = """
> + <pool type='iscsi'>
> + <name>{name}</name>
> + <source>
> + <host name='{source[host]}' {source[port]}/>
> + <device path='{source[target]}'/>
> + {source[auth]}
> + </source>
> + <target>
> + <path>{path}</path>
> + </target>
> + </pool>
> + """.format(**poolArgs)
> + return xml
> +
> +
> def _get_volume_xml(**kwargs):
> # Required parameters
> # name:
> diff --git a/tests/test_storagepool.py b/tests/test_storagepool.py
> index 1a7786f..8341537 100644
> --- a/tests/test_storagepool.py
> +++ b/tests/test_storagepool.py
> @@ -78,6 +78,69 @@ class storagepoolTests(unittest.TestCase):
> <path>/var/lib/kimchi/logical_mount/unitTestLogicalPool</path>
> </target>
> </pool>
> + """},
> + {'def':
> + {'type': 'iscsi',
> + 'name': 'unitTestISCSIPool',
> + 'source': {
> + 'host': '127.0.0.1',
> + 'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
> + 'xml':
> + """
> + <pool type='iscsi'>
> + <name>unitTestISCSIPool</name>
> + <source>
> + <host name='127.0.0.1' />
> + <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> + </source>
> + <target>
> + <path>/dev/disk/by-id</path>
> + </target>
> + </pool>
> + """},
> + {'def':
> + {'type': 'iscsi',
> + 'name': 'unitTestISCSIPoolPort',
> + 'source': {
> + 'host': '127.0.0.1',
> + 'port': 3266,
> + 'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
> + 'xml':
> + """
> + <pool type='iscsi'>
> + <name>unitTestISCSIPoolPort</name>
> + <source>
> + <host name='127.0.0.1' port='3266' />
> + <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> + </source>
> + <target>
> + <path>/dev/disk/by-id</path>
> + </target>
> + </pool>
> + """},
> + {'def':
> + {'type': 'iscsi',
> + 'name': 'unitTestISCSIPoolAuth',
> + 'source': {
> + 'host': '127.0.0.1',
> + 'target': 'iqn.2003-01.org.linux-iscsi.localhost',
> + 'auth': {'username': 'testUser',
> + 'password': 'ActuallyNotUsedInPoolXML'}}},
> + 'xml':
> + """
> + <pool type='iscsi'>
> + <name>unitTestISCSIPoolAuth</name>
> + <source>
> + <host name='127.0.0.1' />
> + <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> + <auth type='chap' username='testUser'>
> + <secret type='iscsi' usage='unitTestISCSIPoolAuth'/>
> + </auth>
> + </source>
> + <target>
> + <path>/dev/disk/by-id</path>
> + </target>
> + </pool>
> """}]
>
> for poolDef in poolDefs:
--
Thanks and best regards!
Sheldon Feng(冯少合)<shaohef at linux.vnet.ibm.com>
IBM Linux Technology Center
More information about the Kimchi-devel
mailing list