[Kimchi-devel] [PATCH v5 4/5] storagepool: Support Creating iSCSI storagepool in model.py

Aline Manera alinefm at linux.vnet.ibm.com
Thu Jan 2 18:34:58 UTC 2014


Reviewed-by: Aline Manera <alinefm at linux.vnet.ibm.com>

On 01/02/2014 07:58 AM, Zhou Zheng Sheng wrote:
> This patch implements creating iSCSI storagepool for libvirt.
> Each LUN in iSCSI storagepool can be used as a volume, but iSCSI
> storagepool does not provide ability to create volume. For now in kimchi
> we create volume for each newly created VM. Next is to implement
> attaching existing volume to a new VM.
>
> This patch also adds validation for the iSCSI back-end. It tries to
> login the target using the auth before actually creating the pool.
> On RedHat family distributions, libiscsi comes with a Python binding,
> but not on other distributions. So in this patch it invokes iscsiadm to
> check iSCSI target.
>
> As regard to iSCSI CHAP authentication, it's somewhat broken before
> libvirt 1.1.1. It was reworked in libvirt commit eb0d79, and this patch
> goes with libvirt 1.1.1, but it doesn't require libvirt in the spec
> file. This is because libvirt 1.1.1 is not available on Ubuntu 12.04
> LTS. The user can just ignore this feature.
>
> How to test it manually
>
> Prerequisite
> An running iSCSI target on the network.
>
> Create:
> curl -u root -H 'Content-type: application/json' \
>    -H 'Accept: application/json' \
>    -d '{"source": {
>            "target": "iqn.YouriSCSITargetIQN.XXX",
>            "host": "10.0.0.X"},
>         "type": "iscsi",
>         "name": "iscsipool"}' \
>    http://127.0.0.1:8000/storagepools
>
> Show Info:
> curl -u root -H 'Accept: application/json' \
>    http://127.0.0.1:8000/storagepools/iscsipool
>
> Activate:
> curl -u root -H 'Content-type: application/json' \
>    -H 'Accept: application/json' \
>    -d '' http://127.0.0.1:8000/storagepools/iscsipool/activate
>
> Examine:
> iscsiadm -m session
>
> Deactivate:
> curl -u root -H 'Content-type: application/json' \
>    -H 'Accept: application/json' \
>    -d '' http://127.0.0.1:8000/storagepools/iscsipool/deactivate
>
> Delete:
> curl -u root -X DELETE -H 'Accept: application/json' \
>    http://127.0.0.1:8000/storagepools/iscsipool
>
> v2 -> v3
>    1) Support CHAP authentication.
>    2) Validate iSCSI target before creating the pool.
>    3) Add XML generation unit test with "port" and "auth".
>
> v3 -> v4
>    1) Update RPM spec file and Debian control file.
>    2) Update API.json.
>    3) Give a proper name for the iSCSI target client class, TargetClient.
>
> v4 -> v5
>    Fix description string of 'auth' in API.json.
>
> Signed-off-by: Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
> ---
>   Makefile.am                   |  1 +
>   contrib/DEBIAN/control.in     |  3 +-
>   contrib/kimchi.spec.fedora.in |  1 +
>   contrib/kimchi.spec.suse.in   |  1 +
>   docs/API.md                   | 12 +++++-
>   src/kimchi/API.json           | 24 ++++++++++++
>   src/kimchi/Makefile.am        |  1 +
>   src/kimchi/iscsi.py           | 89 +++++++++++++++++++++++++++++++++++++++++++
>   src/kimchi/model.py           | 89 +++++++++++++++++++++++++++++++++++++++++--
>   tests/test_storagepool.py     | 63 ++++++++++++++++++++++++++++++
>   10 files changed, 278 insertions(+), 6 deletions(-)
>   create mode 100644 src/kimchi/iscsi.py
>
> diff --git a/Makefile.am b/Makefile.am
> index 01efa1b..b01a8e7 100644
> --- a/Makefile.am
> +++ b/Makefile.am
> @@ -47,6 +47,7 @@ PEP8_WHITELIST = \
>   	src/kimchi/config.py.in \
>   	src/kimchi/disks.py \
>   	src/kimchi/featuretests.py \
> +	src/kimchi/iscsi.py \
>   	src/kimchi/rollbackcontext.py \
>   	src/kimchi/root.py \
>   	src/kimchi/server.py \
> diff --git a/contrib/DEBIAN/control.in b/contrib/DEBIAN/control.in
> index 380584c..eecfb27 100644
> --- a/contrib/DEBIAN/control.in
> +++ b/contrib/DEBIAN/control.in
> @@ -17,7 +17,8 @@ Depends: python-cherrypy3 (>= 3.2.0),
>            python-psutil (>= 0.6.0),
>            python-ethtool,
>            sosreport,
> -         python-ipaddr
> +         python-ipaddr,
> +         open-iscsi
>   Build-Depends:
>   Maintainer: Aline Manera <alinefm at br.ibm.com>
>   Description: Kimchi web server
> diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in
> index 3044fc8..75435b3 100644
> --- a/contrib/kimchi.spec.fedora.in
> +++ b/contrib/kimchi.spec.fedora.in
> @@ -25,6 +25,7 @@ Requires:	python-ethtool
>   Requires:	sos
>   Requires:	python-ipaddr
>   Requires:	nfs-utils
> +Requires:	iscsi-initiator-utils
>
>   %if 0%{?rhel} == 6
>   Requires:	python-ordereddict
> diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in
> index 190b2be..bcfb6db 100644
> --- a/contrib/kimchi.spec.suse.in
> +++ b/contrib/kimchi.spec.suse.in
> @@ -20,6 +20,7 @@ Requires:	python-jsonschema >= 1.3.0
>   Requires:	python-ethtool
>   Requires:	python-ipaddr
>   Requires:	nfs-client
> +Requires:	iscsi-initiator-utils
>
>   %if 0%{?sles_version} == 11
>   Requires:       python-ordereddict
> diff --git a/docs/API.md b/docs/API.md
> index a8b9ea0..c347340 100644
> --- a/docs/API.md
> +++ b/docs/API.md
> @@ -182,17 +182,25 @@ Represents a snapshot of the Virtual Machine's primary monitor.
>   * **POST**: Create a new Storage Pool
>       * name: The name of the Storage Pool.
>       * type: The type of the defined Storage Pool.
> -            Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical'
> +            Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical', 'iscsi'
>       * path: The path of the defined Storage Pool.
>               For 'kimchi-iso' pool refers to targeted deep scan path.
>               Pool types: 'dir', 'kimchi-iso'.
>       * source: Dictionary containing source information of the pool.
>           * host: IP or hostname of server for a pool backed from a remote host.
> -                Pool types: 'netfs'.
> +                Pool types: 'netfs', 'iscsi'.
>           * path: Export path on NFS server for NFS pool.
>                   Pool types: 'netfs'.
>           * devices: Array of devices to be used in the Storage Pool
>                      Pool types: 'logical'.
> +        * target: Target IQN of an iSCSI pool.
> +                  Pool types: 'iscsi'.
> +        * port *(optional)*: Listening port of a remote storage server.
> +                             Pool types: 'iscsi'.
> +        * auth *(optional)*: Storage back-end authentication information.
> +                             Pool types: 'iscsi'.
> +            * username: Login username of the iSCSI target.
> +            * password: Login password of the iSCSI target.
>
>   ### Resource: Storage Pool
>
> diff --git a/src/kimchi/API.json b/src/kimchi/API.json
> index 6519c21..6f1dd03 100644
> --- a/src/kimchi/API.json
> +++ b/src/kimchi/API.json
> @@ -44,6 +44,30 @@
>                                   "description": "Full path of the block device node",
>                                   "type": "string"
>                               }
> +                        },
> +                        "target": {
> +                            "description": "Target IQN of an iSCSI pool",
> +                            "type": "string"
> +                        },
> +                        "port": {
> +                            "description": "Listening port of a remote storage server",
> +                            "type": "integer",
> +                            "minimum": 1,
> +                            "maximum": 65535
> +                        },
> +                        "auth": {
> +                            "description": "Storage back-end authentication information",
> +                            "type": "object",
> +                            "properties": {
> +                                "username": {
> +                                    "description": "Login username of the iSCSI target",
> +                                    "type": "string"
> +                                },
> +                                "password": {
> +                                    "description": "Login password of the iSCSI target",
> +                                    "type": "string"
> +                                }
> +                            }
>                           }
>                       }
>                   }
> diff --git a/src/kimchi/Makefile.am b/src/kimchi/Makefile.am
> index 88ccbf7..261c331 100644
> --- a/src/kimchi/Makefile.am
> +++ b/src/kimchi/Makefile.am
> @@ -31,6 +31,7 @@ kimchi_PYTHON = \
>   	distroloader.py    \
>   	exception.py       \
>   	featuretests.py    \
> +	iscsi.py           \
>   	isoinfo.py         \
>   	mockmodel.py       \
>   	model.py           \
> diff --git a/src/kimchi/iscsi.py b/src/kimchi/iscsi.py
> new file mode 100644
> index 0000000..35c0b8a
> --- /dev/null
> +++ b/src/kimchi/iscsi.py
> @@ -0,0 +1,89 @@
> +#
> +# Project Kimchi
> +#
> +# Copyright IBM, Corp. 2013
> +#
> +# Authors:
> +#  Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
> +#
> +# This library is free software; you can redistribute it and/or
> +# modify it under the terms of the GNU Lesser General Public
> +# License as published by the Free Software Foundation; either
> +# version 2.1 of the License, or (at your option) any later version.
> +#
> +# This library is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the GNU
> +# Lesser General Public License for more details.
> +#
> +# You should have received a copy of the GNU Lesser General Public
> +# License along with this library; if not, write to the Free Software
> +# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA
> +
> +import subprocess
> +
> +
> +from kimchi.exception import OperationFailed
> +
> +
> +class TargetClient(object):
> +    def __init__(self, target, host, port=None, auth=None):
> +        self.portal = host + ("" if port is None else ":%s" % port)
> +        self.target = target
> +        self.auth = auth
> +        self.targetCmd = ['iscsiadm', '--mode', 'node', '--targetname',
> +                          self.target, '--portal', self.portal]
> +
> +    def _update_db(self, Name, Value):
> +        self._run_cmd(['--op=update', '--name', Name, '--value', Value])
> +
> +    def _update_auth(self):
> +        if self.auth is None:
> +            items = (('node.session.auth.authmethod', 'None'),
> +                     ('node.session.auth.username', ''),
> +                     ('node.session.auth.password', ''))
> +        else:
> +            items = (('node.session.auth.authmethod', 'CHAP'),
> +                     ('node.session.auth.username', self.auth['username']),
> +                     ('node.session.auth.password', self.auth['password']))
> +        for name, value in items:
> +            self._update_db(name, value)
> +
> +    def _run_cmd(self, cmd):
> +        iscsiadm = subprocess.Popen(
> +            self.targetCmd + cmd,
> +            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> +        out, err = iscsiadm.communicate()
> +        if iscsiadm.returncode != 0:
> +            raise OperationFailed('Error executing iscsiadm: %s' % err)
> +        return out
> +
> +    def _discover(self):
> +        iscsiadm = subprocess.Popen(
> +            ['iscsiadm', '--mode', 'discovery', '--type', 'sendtargets',
> +             '--portal', self.portal],
> +            stdout=subprocess.PIPE, stderr=subprocess.PIPE)
> +        out, err = iscsiadm.communicate()
> +        if iscsiadm.returncode != 0:
> +            raise OperationFailed('Error executing iscsiadm: %s' % err)
> +        return out
> +
> +    def _run_op(self, op):
> +        self._run_cmd(['--' + op])
> +
> +    def login(self):
> +        self._discover()
> +        self._update_auth()
> +        self._run_op('login')
> +
> +    def logout(self):
> +        self._run_op('logout')
> +
> +    def validate(self):
> +        try:
> +            self.login()
> +        except OperationFailed:
> +            return False
> +
> +        self.logout()
> +        return True
> diff --git a/src/kimchi/model.py b/src/kimchi/model.py
> index 2b46121..17d5485 100644
> --- a/src/kimchi/model.py
> +++ b/src/kimchi/model.py
> @@ -65,6 +65,7 @@ from kimchi.distroloader import DistroLoader
>   from kimchi.exception import InvalidOperation, InvalidParameter, MissingParameter
>   from kimchi.exception import NotFoundError, OperationFailed
>   from kimchi.featuretests import FeatureTests
> +from kimchi.iscsi import TargetClient
>   from kimchi.networkxml import to_network_xml
>   from kimchi.objectstore import ObjectStore
>   from kimchi.scan import Scanner
> @@ -995,7 +996,7 @@ class Model(object):
>               if params['type'] == 'kimchi-iso':
>                   task_id = self._do_deep_scan(params)
>               poolDef = StoragePoolDef.create(params)
> -            poolDef.prepare()
> +            poolDef.prepare(conn)
>               xml = poolDef.xml
>           except KeyError, key:
>               raise MissingParameter(key)
> @@ -1444,7 +1445,7 @@ class StoragePoolDef(object):
>       def __init__(self, poolArgs):
>           self.poolArgs = poolArgs
>
> -    def prepare(self):
> +    def prepare(self, conn):
>           ''' Validate pool arguments and perform preparations. Operation which
>           would cause side effect should be put here. Subclasses can optionally
>           override this method, or it always succeeds by default. '''
> @@ -1487,7 +1488,7 @@ class NetfsPoolDef(StoragePoolDef):
>           super(NetfsPoolDef, self).__init__(poolArgs)
>           self.path = '/var/lib/kimchi/nfs_mount/' + self.poolArgs['name']
>
> -    def prepare(self):
> +    def prepare(self, conn):
>           # TODO: Verify the NFS export can be actually mounted.
>           pass
>
> @@ -1550,6 +1551,88 @@ class LogicalPoolDef(StoragePoolDef):
>           return xml
>
>
> +class IscsiPoolDef(StoragePoolDef):
> +    poolType = 'iscsi'
> +
> +    def prepare(self, conn):
> +        source = self.poolArgs['source']
> +        if not TargetClient(**source).validate():
> +            raise OperationFailed("Can not login to iSCSI host %s target %s" %
> +                                  (source['host'], source['target']))
> +        self._prepare_auth(conn)
> +
> +    def _prepare_auth(self, conn):
> +        try:
> +            auth = self.poolArgs['source']['auth']
> +        except KeyError:
> +            return
> +
> +        try:
> +            virSecret = conn.secretLookupByUsage(
> +                libvirt.VIR_SECRET_USAGE_TYPE_ISCSI, self.poolArgs['name'])
> +        except libvirt.libvirtError:
> +            xml = '''
> +            <secret ephemeral='no' private='yes'>
> +              <description>Secret for iSCSI storage pool {name}</description>
> +              <auth type='chap' username='{username}'/>
> +              <usage type='iscsi'>
> +                <target>{name}</target>
> +              </usage>
> +            </secret>'''.format(name=self.poolArgs['name'],
> +                                username=auth['username'])
> +            virSecret = conn.secretDefineXML(xml)
> +
> +        virSecret.setValue(auth['password'])
> +
> +    def _format_port(self, poolArgs):
> +        try:
> +            port = poolArgs['source']['port']
> +        except KeyError:
> +            return ""
> +        return "port='%s'" % port
> +
> +    def _format_auth(self, poolArgs):
> +        try:
> +            auth = poolArgs['source']['auth']
> +        except KeyError:
> +            return ""
> +
> +        return '''
> +        <auth type='chap' username='{username}'>
> +          <secret type='iscsi' usage='{name}'/>
> +        </auth>'''.format(name=poolArgs['name'], username=auth['username'])
> +
> +    @property
> +    def xml(self):
> +        # Required parameters
> +        # name:
> +        # type:
> +        # source[host]:
> +        # source[target]:
> +        #
> +        # Optional parameters
> +        # source[port]:
> +        poolArgs = copy.deepcopy(self.poolArgs)
> +        poolArgs['source'].update({'port': self._format_port(poolArgs),
> +                                   'auth': self._format_auth(poolArgs)})
> +        poolArgs['path'] = '/dev/disk/by-id'
> +
> +        xml = """
> +        <pool type='iscsi'>
> +          <name>{name}</name>
> +          <source>
> +            <host name='{source[host]}' {source[port]}/>
> +            <device path='{source[target]}'/>
> +            {source[auth]}
> +          </source>
> +          <target>
> +            <path>{path}</path>
> +          </target>
> +        </pool>
> +        """.format(**poolArgs)
> +        return xml
> +
> +
>   def _get_volume_xml(**kwargs):
>       # Required parameters
>       # name:
> diff --git a/tests/test_storagepool.py b/tests/test_storagepool.py
> index 1a7786f..8341537 100644
> --- a/tests/test_storagepool.py
> +++ b/tests/test_storagepool.py
> @@ -78,6 +78,69 @@ class storagepoolTests(unittest.TestCase):
>                    <path>/var/lib/kimchi/logical_mount/unitTestLogicalPool</path>
>                </target>
>                </pool>
> +             """},
> +            {'def':
> +                {'type': 'iscsi',
> +                 'name': 'unitTestISCSIPool',
> +                 'source': {
> +                     'host': '127.0.0.1',
> +                     'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
> +             'xml':
> +             """
> +             <pool type='iscsi'>
> +               <name>unitTestISCSIPool</name>
> +               <source>
> +                 <host name='127.0.0.1' />
> +                 <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> +               </source>
> +               <target>
> +                 <path>/dev/disk/by-id</path>
> +               </target>
> +             </pool>
> +             """},
> +            {'def':
> +                {'type': 'iscsi',
> +                 'name': 'unitTestISCSIPoolPort',
> +                 'source': {
> +                     'host': '127.0.0.1',
> +                     'port': 3266,
> +                     'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
> +             'xml':
> +             """
> +             <pool type='iscsi'>
> +               <name>unitTestISCSIPoolPort</name>
> +               <source>
> +                 <host name='127.0.0.1' port='3266' />
> +                 <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> +               </source>
> +               <target>
> +                 <path>/dev/disk/by-id</path>
> +               </target>
> +             </pool>
> +             """},
> +            {'def':
> +                {'type': 'iscsi',
> +                 'name': 'unitTestISCSIPoolAuth',
> +                 'source': {
> +                     'host': '127.0.0.1',
> +                     'target': 'iqn.2003-01.org.linux-iscsi.localhost',
> +                     'auth': {'username': 'testUser',
> +                              'password': 'ActuallyNotUsedInPoolXML'}}},
> +             'xml':
> +             """
> +             <pool type='iscsi'>
> +               <name>unitTestISCSIPoolAuth</name>
> +               <source>
> +                 <host name='127.0.0.1' />
> +                 <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
> +                 <auth type='chap' username='testUser'>
> +                   <secret type='iscsi' usage='unitTestISCSIPoolAuth'/>
> +                 </auth>
> +               </source>
> +               <target>
> +                 <path>/dev/disk/by-id</path>
> +               </target>
> +             </pool>
>                """}]
>
>           for poolDef in poolDefs:




More information about the Kimchi-devel mailing list