[Kimchi-devel] [PATCH v4 4/5] storagepool: Support Creating iSCSI storagepool in model.py
Zhou Zheng Sheng
zhshzhou at linux.vnet.ibm.com
Tue Dec 31 06:39:58 UTC 2013
This patch implements creating iSCSI storagepool for libvirt.
Each LUN in iSCSI storagepool can be used as a volume, but iSCSI
storagepool does not provide ability to create volume. For now in kimchi
we create volume for each newly created VM. Next is to implement
attaching existing volume to a new VM.
This patch also adds validation for the iSCSI back-end. It tries to
login the target using the auth before actually creating the pool.
On RedHat family distributions, libiscsi comes with a Python binding,
but not on other distributions. So in this patch it invokes iscsiadm to
check iSCSI target.
As regard to iSCSI CHAP authentication, it's somewhat broken before
libvirt 1.1.1. It was reworked in libvirt commit eb0d79, and this patch
goes with libvirt 1.1.1, but it doesn't require libvirt in the spec
file. This is because libvirt 1.1.1 is not available on Ubuntu 12.04
LTS. The user can just ignore this feature.
How to test it manually
Prerequisite
An running iSCSI target on the network.
Create:
curl -u root -H 'Content-type: application/json' \
-H 'Accept: application/json' \
-d '{"source": {
"target": "iqn.YouriSCSITargetIQN.XXX",
"host": "10.0.0.X"},
"type": "iscsi",
"name": "iscsipool"}' \
http://127.0.0.1:8000/storagepools
Show Info:
curl -u root -H 'Accept: application/json' \
http://127.0.0.1:8000/storagepools/iscsipool
Activate:
curl -u root -H 'Content-type: application/json' \
-H 'Accept: application/json' \
-d '' http://127.0.0.1:8000/storagepools/iscsipool/activate
Examine:
iscsiadm -m session
Deactivate:
curl -u root -H 'Content-type: application/json' \
-H 'Accept: application/json' \
-d '' http://127.0.0.1:8000/storagepools/iscsipool/deactivate
Delete:
curl -u root -X DELETE -H 'Accept: application/json' \
http://127.0.0.1:8000/storagepools/iscsipool
v2 -> v3
1) Support CHAP authentication.
2) Validate iSCSI target before creating the pool.
3) Add XML generation unit test with "port" and "auth".
v3 -> v4
1) Update RPM spec file and Debian control file.
2) Update API.json.
3) Give a proper name for the iSCSI target client class, TargetClient.
Signed-off-by: Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
---
Makefile.am | 1 +
contrib/DEBIAN/control.in | 3 +-
contrib/kimchi.spec.fedora.in | 1 +
contrib/kimchi.spec.suse.in | 1 +
docs/API.md | 12 +++++-
src/kimchi/API.json | 24 ++++++++++++
src/kimchi/Makefile.am | 1 +
src/kimchi/iscsi.py | 89 +++++++++++++++++++++++++++++++++++++++++++
src/kimchi/model.py | 88 ++++++++++++++++++++++++++++++++++++++++--
tests/test_storagepool.py | 63 ++++++++++++++++++++++++++++++
10 files changed, 277 insertions(+), 6 deletions(-)
create mode 100644 src/kimchi/iscsi.py
diff --git a/Makefile.am b/Makefile.am
index 33059a7..b87f5d0 100644
--- a/Makefile.am
+++ b/Makefile.am
@@ -43,6 +43,7 @@ PEP8_WHITELIST = \
src/kimchi/cachebust.py \
src/kimchi/config.py.in \
src/kimchi/disks.py \
+ src/kimchi/iscsi.py \
src/kimchi/root.py \
src/kimchi/server.py \
plugins/__init__.py \
diff --git a/contrib/DEBIAN/control.in b/contrib/DEBIAN/control.in
index 380584c..eecfb27 100644
--- a/contrib/DEBIAN/control.in
+++ b/contrib/DEBIAN/control.in
@@ -17,7 +17,8 @@ Depends: python-cherrypy3 (>= 3.2.0),
python-psutil (>= 0.6.0),
python-ethtool,
sosreport,
- python-ipaddr
+ python-ipaddr,
+ open-iscsi
Build-Depends:
Maintainer: Aline Manera <alinefm at br.ibm.com>
Description: Kimchi web server
diff --git a/contrib/kimchi.spec.fedora.in b/contrib/kimchi.spec.fedora.in
index 577516c..47aa31e 100644
--- a/contrib/kimchi.spec.fedora.in
+++ b/contrib/kimchi.spec.fedora.in
@@ -25,6 +25,7 @@ Requires: python-ethtool
Requires: sos
Requires: python-ipaddr
Requires: nfs-utils
+Requires: iscsi-initiator-utils
%if 0%{?rhel} == 6
Requires: python-ordereddict
diff --git a/contrib/kimchi.spec.suse.in b/contrib/kimchi.spec.suse.in
index 12d02ec..c8a4d31 100644
--- a/contrib/kimchi.spec.suse.in
+++ b/contrib/kimchi.spec.suse.in
@@ -20,6 +20,7 @@ Requires: python-jsonschema >= 1.3.0
Requires: python-ethtool
Requires: python-ipaddr
Requires: nfs-client
+Requires: iscsi-initiator-utils
%if 0%{?sles_version} == 11
Requires: python-ordereddict
diff --git a/docs/API.md b/docs/API.md
index a8b9ea0..c347340 100644
--- a/docs/API.md
+++ b/docs/API.md
@@ -182,17 +182,25 @@ Represents a snapshot of the Virtual Machine's primary monitor.
* **POST**: Create a new Storage Pool
* name: The name of the Storage Pool.
* type: The type of the defined Storage Pool.
- Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical'
+ Supported types: 'dir', 'kimchi-iso', 'netfs', 'logical', 'iscsi'
* path: The path of the defined Storage Pool.
For 'kimchi-iso' pool refers to targeted deep scan path.
Pool types: 'dir', 'kimchi-iso'.
* source: Dictionary containing source information of the pool.
* host: IP or hostname of server for a pool backed from a remote host.
- Pool types: 'netfs'.
+ Pool types: 'netfs', 'iscsi'.
* path: Export path on NFS server for NFS pool.
Pool types: 'netfs'.
* devices: Array of devices to be used in the Storage Pool
Pool types: 'logical'.
+ * target: Target IQN of an iSCSI pool.
+ Pool types: 'iscsi'.
+ * port *(optional)*: Listening port of a remote storage server.
+ Pool types: 'iscsi'.
+ * auth *(optional)*: Storage back-end authentication information.
+ Pool types: 'iscsi'.
+ * username: Login username of the iSCSI target.
+ * password: Login password of the iSCSI target.
### Resource: Storage Pool
diff --git a/src/kimchi/API.json b/src/kimchi/API.json
index 6519c21..c1b88b4 100644
--- a/src/kimchi/API.json
+++ b/src/kimchi/API.json
@@ -44,6 +44,30 @@
"description": "Full path of the block device node",
"type": "string"
}
+ },
+ "target": {
+ "description": "Target IQN of an iSCSI pool",
+ "type": "string"
+ },
+ "port": {
+ "description": "Listening port of a remote storage server",
+ "type": "integer",
+ "minimum": 1,
+ "maximum": 65535
+ },
+ "auth": {
+ "description": "IP or hostname of server for a pool backed from a remote host",
+ "type": "object",
+ "properties": {
+ "username": {
+ "description": "Login username of the iSCSI target",
+ "type": "string"
+ },
+ "password": {
+ "description": "Login password of the iSCSI target",
+ "type": "string"
+ }
+ }
}
}
}
diff --git a/src/kimchi/Makefile.am b/src/kimchi/Makefile.am
index 47a3bd2..02b6fee 100644
--- a/src/kimchi/Makefile.am
+++ b/src/kimchi/Makefile.am
@@ -28,6 +28,7 @@ kimchi_PYTHON = \
distroloader.py \
exception.py \
__init__.py \
+ iscsi.py \
isoinfo.py \
netinfo.py \
network.py \
diff --git a/src/kimchi/iscsi.py b/src/kimchi/iscsi.py
new file mode 100644
index 0000000..35c0b8a
--- /dev/null
+++ b/src/kimchi/iscsi.py
@@ -0,0 +1,89 @@
+#
+# Project Kimchi
+#
+# Copyright IBM, Corp. 2013
+#
+# Authors:
+# Zhou Zheng Sheng <zhshzhou at linux.vnet.ibm.com>
+#
+# This library is free software; you can redistribute it and/or
+# modify it under the terms of the GNU Lesser General Public
+# License as published by the Free Software Foundation; either
+# version 2.1 of the License, or (at your option) any later version.
+#
+# This library is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+# Lesser General Public License for more details.
+#
+# You should have received a copy of the GNU Lesser General Public
+# License along with this library; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301USA
+
+import subprocess
+
+
+from kimchi.exception import OperationFailed
+
+
+class TargetClient(object):
+ def __init__(self, target, host, port=None, auth=None):
+ self.portal = host + ("" if port is None else ":%s" % port)
+ self.target = target
+ self.auth = auth
+ self.targetCmd = ['iscsiadm', '--mode', 'node', '--targetname',
+ self.target, '--portal', self.portal]
+
+ def _update_db(self, Name, Value):
+ self._run_cmd(['--op=update', '--name', Name, '--value', Value])
+
+ def _update_auth(self):
+ if self.auth is None:
+ items = (('node.session.auth.authmethod', 'None'),
+ ('node.session.auth.username', ''),
+ ('node.session.auth.password', ''))
+ else:
+ items = (('node.session.auth.authmethod', 'CHAP'),
+ ('node.session.auth.username', self.auth['username']),
+ ('node.session.auth.password', self.auth['password']))
+ for name, value in items:
+ self._update_db(name, value)
+
+ def _run_cmd(self, cmd):
+ iscsiadm = subprocess.Popen(
+ self.targetCmd + cmd,
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ out, err = iscsiadm.communicate()
+ if iscsiadm.returncode != 0:
+ raise OperationFailed('Error executing iscsiadm: %s' % err)
+ return out
+
+ def _discover(self):
+ iscsiadm = subprocess.Popen(
+ ['iscsiadm', '--mode', 'discovery', '--type', 'sendtargets',
+ '--portal', self.portal],
+ stdout=subprocess.PIPE, stderr=subprocess.PIPE)
+ out, err = iscsiadm.communicate()
+ if iscsiadm.returncode != 0:
+ raise OperationFailed('Error executing iscsiadm: %s' % err)
+ return out
+
+ def _run_op(self, op):
+ self._run_cmd(['--' + op])
+
+ def login(self):
+ self._discover()
+ self._update_auth()
+ self._run_op('login')
+
+ def logout(self):
+ self._run_op('logout')
+
+ def validate(self):
+ try:
+ self.login()
+ except OperationFailed:
+ return False
+
+ self.logout()
+ return True
diff --git a/src/kimchi/model.py b/src/kimchi/model.py
index b51fc62..f9ac3ed 100644
--- a/src/kimchi/model.py
+++ b/src/kimchi/model.py
@@ -65,6 +65,7 @@ from kimchi.distroloader import DistroLoader
from kimchi.exception import InvalidOperation, InvalidParameter, MissingParameter
from kimchi.exception import NotFoundError, OperationFailed
from kimchi.featuretests import FeatureTests
+from kimchi.iscsi import TargetClient
from kimchi.networkxml import to_network_xml
from kimchi.objectstore import ObjectStore
from kimchi.scan import Scanner
@@ -995,7 +996,7 @@ class Model(object):
if params['type'] == 'kimchi-iso':
task_id = self._do_deep_scan(params)
poolDef = StoragePoolDef.create(params)
- poolDef.prepare()
+ poolDef.prepare(conn)
xml = poolDef.xml
except KeyError, key:
raise MissingParameter(key)
@@ -1445,7 +1446,7 @@ class StoragePoolDef(object):
self.poolArgs = poolArgs
self._xml = None
- def prepare(self):
+ def prepare(self, conn):
''' Validate pool arguments and perform preparations. Operation which
would cause side effect should be put here. Subclasses can optionally
override this method, or it always succeeds by default. '''
@@ -1491,7 +1492,7 @@ class NetfsPoolDef(StoragePoolDef):
super(NetfsPoolDef, self).__init__(poolArgs)
self.path = '/var/lib/kimchi/nfs_mount/' + self.poolArgs['name']
- def prepare(self):
+ def prepare(self, conn):
# TODO Verify the NFS export can be actually mounted.
pass
@@ -1552,6 +1553,87 @@ class LogicalPoolDef(StoragePoolDef):
return xml
+class IscsiPoolDef(StoragePoolDef):
+ poolType = 'iscsi'
+
+ def prepare(self, conn):
+ source = self.poolArgs['source']
+ if not TargetClient(**source).validate():
+ raise OperationFailed("Can not login to iSCSI host %s target %s" %
+ (source['host'], source['target']))
+ self._prepare_auth(conn)
+
+ def _prepare_auth(self, conn):
+ try:
+ auth = self.poolArgs['source']['auth']
+ except KeyError:
+ return
+
+ try:
+ virSecret = conn.secretLookupByUsage(
+ libvirt.VIR_SECRET_USAGE_TYPE_ISCSI, self.poolArgs['name'])
+ except libvirt.libvirtError:
+ xml = '''
+ <secret ephemeral='no' private='yes'>
+ <description>Secret for iSCSI storage pool {name}</description>
+ <auth type='chap' username='{username}'/>
+ <usage type='iscsi'>
+ <target>{name}</target>
+ </usage>
+ </secret>'''.format(name=self.poolArgs['name'],
+ username=auth['username'])
+ virSecret = conn.secretDefineXML(xml)
+
+ virSecret.setValue(auth['password'])
+
+ def _format_port(self, poolArgs):
+ try:
+ port = poolArgs['source']['port']
+ except KeyError:
+ return ""
+ return "port='%s'" % port
+
+ def _format_auth(self, poolArgs):
+ try:
+ auth = poolArgs['source']['auth']
+ except KeyError:
+ return ""
+
+ return '''
+ <auth type='chap' username='{username}'>
+ <secret type='iscsi' usage='{name}'/>
+ </auth>'''.format(name=poolArgs['name'], username=auth['username'])
+
+ def _get_xml(self):
+ # Required parameters
+ # name:
+ # type:
+ # source[host]:
+ # source[target]:
+ #
+ # Optional parameters
+ # source[port]:
+ poolArgs = copy.deepcopy(self.poolArgs)
+ poolArgs['source'].update({'port': self._format_port(poolArgs),
+ 'auth': self._format_auth(poolArgs)})
+ poolArgs['path'] = '/dev/disk/by-id'
+
+ xml = """
+ <pool type='iscsi'>
+ <name>{name}</name>
+ <source>
+ <host name='{source[host]}' {source[port]}/>
+ <device path='{source[target]}'/>
+ {source[auth]}
+ </source>
+ <target>
+ <path>{path}</path>
+ </target>
+ </pool>
+ """.format(**poolArgs)
+ return xml
+
+
def _get_volume_xml(**kwargs):
# Required parameters
# name:
diff --git a/tests/test_storagepool.py b/tests/test_storagepool.py
index 42be004..a66632b 100644
--- a/tests/test_storagepool.py
+++ b/tests/test_storagepool.py
@@ -78,6 +78,69 @@ class storagepoolTests(unittest.TestCase):
<path>/var/lib/kimchi/logical_mount/unitTestLogicalPool</path>
</target>
</pool>
+ """},
+ {'def':
+ {'type': 'iscsi',
+ 'name': 'unitTestISCSIPool',
+ 'source': {
+ 'host': '127.0.0.1',
+ 'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
+ 'xml':
+ """
+ <pool type='iscsi'>
+ <name>unitTestISCSIPool</name>
+ <source>
+ <host name='127.0.0.1' />
+ <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
+ </source>
+ <target>
+ <path>/dev/disk/by-id</path>
+ </target>
+ </pool>
+ """},
+ {'def':
+ {'type': 'iscsi',
+ 'name': 'unitTestISCSIPoolPort',
+ 'source': {
+ 'host': '127.0.0.1',
+ 'port': 3266,
+ 'target': 'iqn.2003-01.org.linux-iscsi.localhost'}},
+ 'xml':
+ """
+ <pool type='iscsi'>
+ <name>unitTestISCSIPoolPort</name>
+ <source>
+ <host name='127.0.0.1' port='3266' />
+ <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
+ </source>
+ <target>
+ <path>/dev/disk/by-id</path>
+ </target>
+ </pool>
+ """},
+ {'def':
+ {'type': 'iscsi',
+ 'name': 'unitTestISCSIPoolAuth',
+ 'source': {
+ 'host': '127.0.0.1',
+ 'target': 'iqn.2003-01.org.linux-iscsi.localhost',
+ 'auth': {'username': 'testUser',
+ 'password': 'ActuallyNotUsedInPoolXML'}}},
+ 'xml':
+ """
+ <pool type='iscsi'>
+ <name>unitTestISCSIPoolAuth</name>
+ <source>
+ <host name='127.0.0.1' />
+ <device path='iqn.2003-01.org.linux-iscsi.localhost'/>
+ <auth type='chap' username='testUser'>
+ <secret type='iscsi' usage='unitTestISCSIPoolAuth'/>
+ </auth>
+ </source>
+ <target>
+ <path>/dev/disk/by-id</path>
+ </target>
+ </pool>
"""}]
for poolDef in poolDefs:
--
1.7.11.7
More information about the Kimchi-devel
mailing list