Using ceph volumes with ovirt
This is a multi-part message in MIME format. --------------C7D3C76DD1E6339049812313 Content-Type: text/plain; charset=iso-8859-15; format=flowed Content-Transfer-Encoding: 7bit Hi, I'm happily using our research cluster in Italy via gluster, and now I'm trying to hotplug a ceph disk on a VM of my cluster, without success. The ceph cluster is managed via openstack cinder and I can create correctly the disk via ovirt (3.6.6.2-1 on CentOS 7.2). The problem comes when trying to hotplug, or start a machine with the given disk attached. In the vdsm log of the host where the VM is running or starting I see the following error: jsonrpc.Executor/5::INFO::2016-05-30 10:35:29,197::vm::2729::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug disk xml: <disk address="" device="disk" snapshot="no" type="network"> <source name="images/volume-9134b639-c23c-4ff1-91ca-0462c80026d2" protocol="rbd"> <host name="141.108.X.Y1" port="6789" transport="tcp"/> <host name="141.108.X.Y2" port="6789" transport="tcp"/> </source> <auth username="cinder"> <secret type="ceph" uuid="<base 64 ceph secret>"/> </auth> <target bus="virtio" dev="vdb"/> <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> </disk> jsonrpc.Executor/5::ERROR::2016-05-30 10:35:29,198::vm::2737::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug failed Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2735, in hotplugDisk self._dom.attachDevice(driveXml) File "/usr/share/vdsm/virt/virdomain.py", line 68, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 530, in attachDevice if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self) libvirtError: XML error: invalid auth secret uuid In fact the uuid of the secret used by ovirt to hotplug seems to be the ceph secret (masked here as <base 64 ceph secret>), while libvirt expects the uuid of the libvirt secret, by looking at the instructions http://docs.ceph.com/docs/jewel/rbd/libvirt/. Anyone got it working? Thanks, Alessandro --------------C7D3C76DD1E6339049812313 Content-Type: text/html; charset=iso-8859-15 Content-Transfer-Encoding: 8bit <html> <head> <meta http-equiv="content-type" content="text/html; charset=iso-8859-15"> </head> <body bgcolor="#FFFFFF" text="#000000"> <p> </p> <div class="moz-text-flowed" style="font-family: -moz-fixed; font-size: 12px;" lang="x-western">Hi, <br> I'm happily using our research cluster in Italy via gluster, and now I'm trying to hotplug a ceph disk on a VM of my cluster, without success. <br> The ceph cluster is managed via openstack cinder and I can create correctly the disk via ovirt (3.6.6.2-1 on CentOS 7.2). <br> The problem comes when trying to hotplug, or start a machine with the given disk attached. <br> In the vdsm log of the host where the VM is running or starting I see the following error: <br> <br> <br> jsonrpc.Executor/5::INFO::2016-05-30 10:35:29,197::vm::2729::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug disk xml: <disk address="" device="disk" snapshot="no" type="network"> <br> <source name="images/volume-9134b639-c23c-4ff1-91ca-0462c80026d2" protocol="rbd"> <br> <host name="141.108.X.Y1" port="6789" transport="tcp"/> <br> <host name="141.108.X.Y2" port="6789" transport="tcp"/> <br> </source> <br> <auth username="cinder"> <br> <secret type="ceph" uuid="<base 64 ceph secret>"/> <br> </auth> <br> <target bus="virtio" dev="vdb"/> <br> <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> <br> </disk> <br> <br> jsonrpc.Executor/5::ERROR::2016-05-30 10:35:29,198::vm::2737::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug failed <br> Traceback (most recent call last): <br> File "/usr/share/vdsm/virt/vm.py", line 2735, in hotplugDisk <br> self._dom.attachDevice(driveXml) <br> File "/usr/share/vdsm/virt/virdomain.py", line 68, in f <br> ret = attr(*args, **kwargs) <br> File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper <br> ret = f(*args, **kwargs) <br> File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper <br> return func(inst, *args, **kwargs) <br> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 530, in attachDevice <br> if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self) <br> libvirtError: XML error: invalid auth secret uuid <br> <br> <br> <br> In fact the uuid of the secret used by ovirt to hotplug seems to be the ceph secret (masked here as <base 64 ceph secret>), while libvirt expects the uuid of the libvirt secret, by looking at the instructions <a class="moz-txt-link-freetext" href="http://docs.ceph.com/docs/jewel/rbd/libvirt/">http://docs.ceph.com/docs/jewel/rbd/libvirt/</a>. <br> Anyone got it working? <br> Thanks, <br> <br> Alessandro <br> </div> </body> </html> --------------C7D3C76DD1E6339049812313--
This is a multi-part message in MIME format. --------------D0AC99C00ED1983052736348 Content-Type: text/plain; charset=windows-1252; format=flowed Content-Transfer-Encoding: 7bit Hi, just to answer myself, I found these instructions that solved my problem: http://7xqb88.com1.z0.glb.clouddn.com/Features_Cinder%20Integration.pdf Basically, I was missing the step to add the ceph key to the Authentication Keys tab of the cinder External Provider. It's all working now. Cheers, Alessandro Il 30/05/16 10:55, Alessandro De Salvo ha scritto:
Hi, I'm happily using our research cluster in Italy via gluster, and now I'm trying to hotplug a ceph disk on a VM of my cluster, without success. The ceph cluster is managed via openstack cinder and I can create correctly the disk via ovirt (3.6.6.2-1 on CentOS 7.2). The problem comes when trying to hotplug, or start a machine with the given disk attached. In the vdsm log of the host where the VM is running or starting I see the following error:
jsonrpc.Executor/5::INFO::2016-05-30 10:35:29,197::vm::2729::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug disk xml: <disk address="" device="disk" snapshot="no" type="network"> <source name="images/volume-9134b639-c23c-4ff1-91ca-0462c80026d2" protocol="rbd"> <host name="141.108.X.Y1" port="6789" transport="tcp"/> <host name="141.108.X.Y2" port="6789" transport="tcp"/> </source> <auth username="cinder"> <secret type="ceph" uuid="<base 64 ceph secret>"/> </auth> <target bus="virtio" dev="vdb"/> <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> </disk>
jsonrpc.Executor/5::ERROR::2016-05-30 10:35:29,198::vm::2737::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug failed Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2735, in hotplugDisk self._dom.attachDevice(driveXml) File "/usr/share/vdsm/virt/virdomain.py", line 68, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 530, in attachDevice if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self) libvirtError: XML error: invalid auth secret uuid
In fact the uuid of the secret used by ovirt to hotplug seems to be the ceph secret (masked here as <base 64 ceph secret>), while libvirt expects the uuid of the libvirt secret, by looking at the instructions http://docs.ceph.com/docs/jewel/rbd/libvirt/. Anyone got it working? Thanks,
Alessandro
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
--------------D0AC99C00ED1983052736348 Content-Type: text/html; charset=windows-1252 Content-Transfer-Encoding: 8bit <html> <head> <meta content="text/html; charset=windows-1252" http-equiv="Content-Type"> </head> <body bgcolor="#FFFFFF" text="#000000"> Hi,<br> just to answer myself, I found these instructions that solved my problem:<br> <br> <a class="moz-txt-link-freetext" href="http://7xqb88.com1.z0.glb.clouddn.com/Features_Cinder%20Integration.pdf">http://7xqb88.com1.z0.glb.clouddn.com/Features_Cinder%20Integration.pdf</a><br> <br> Basically, I was missing the step to add the ceph key to the Authentication Keys tab of the cinder External Provider.<br> It's all working now.<br> Cheers,<br> <br> Alessandro<br> <br> <div class="moz-cite-prefix">Il 30/05/16 10:55, Alessandro De Salvo ha scritto:<br> </div> <blockquote cite="mid:9e1c7728-c143-0835-33ce-10e761a2d9b0@roma1.infn.it" type="cite"> <meta http-equiv="content-type" content="text/html; charset=windows-1252"> <p> </p> <div class="moz-text-flowed" style="font-family: -moz-fixed; font-size: 12px;" lang="x-western">Hi, <br> I'm happily using our research cluster in Italy via gluster, and now I'm trying to hotplug a ceph disk on a VM of my cluster, without success. <br> The ceph cluster is managed via openstack cinder and I can create correctly the disk via ovirt (3.6.6.2-1 on CentOS 7.2). <br> The problem comes when trying to hotplug, or start a machine with the given disk attached. <br> In the vdsm log of the host where the VM is running or starting I see the following error: <br> <br> <br> jsonrpc.Executor/5::INFO::2016-05-30 10:35:29,197::vm::2729::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug disk xml: <disk address="" device="disk" snapshot="no" type="network"> <br> <source name="images/volume-9134b639-c23c-4ff1-91ca-0462c80026d2" protocol="rbd"> <br> <host name="141.108.X.Y1" port="6789" transport="tcp"/> <br> <host name="141.108.X.Y2" port="6789" transport="tcp"/> <br> </source> <br> <auth username="cinder"> <br> <secret type="ceph" uuid="<base 64 ceph secret>"/> <br> </auth> <br> <target bus="virtio" dev="vdb"/> <br> <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> <br> </disk> <br> <br> jsonrpc.Executor/5::ERROR::2016-05-30 10:35:29,198::vm::2737::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug failed <br> Traceback (most recent call last): <br> File "/usr/share/vdsm/virt/vm.py", line 2735, in hotplugDisk <br> self._dom.attachDevice(driveXml) <br> File "/usr/share/vdsm/virt/virdomain.py", line 68, in f <br> ret = attr(*args, **kwargs) <br> File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper <br> ret = f(*args, **kwargs) <br> File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper <br> return func(inst, *args, **kwargs) <br> File "/usr/lib64/python2.7/site-packages/libvirt.py", line 530, in attachDevice <br> if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self) <br> libvirtError: XML error: invalid auth secret uuid <br> <br> <br> <br> In fact the uuid of the secret used by ovirt to hotplug seems to be the ceph secret (masked here as <base 64 ceph secret>), while libvirt expects the uuid of the libvirt secret, by looking at the instructions <a moz-do-not-send="true" class="moz-txt-link-freetext" href="http://docs.ceph.com/docs/jewel/rbd/libvirt/">http://docs.ceph.com/docs/jewel/rbd/libvirt/</a>. <br> Anyone got it working? <br> Thanks, <br> <br> Alessandro <br> </div> <br> <fieldset class="mimeAttachmentHeader"></fieldset> <br> <pre wrap="">_______________________________________________ Users mailing list <a class="moz-txt-link-abbreviated" href="mailto:Users@ovirt.org">Users@ovirt.org</a> <a class="moz-txt-link-freetext" href="http://lists.ovirt.org/mailman/listinfo/users">http://lists.ovirt.org/mailman/listinfo/users</a> </pre> </blockquote> <br> </body> </html> --------------D0AC99C00ED1983052736348--
On Mon, May 30, 2016 at 1:44 PM, Alessandro De Salvo <Alessandro.DeSalvo@roma1.infn.it> wrote:
Hi, just to answer myself, I found these instructions that solved my problem:
http://7xqb88.com1.z0.glb.clouddn.com/Features_Cinder%20Integration.pdf
Basically, I was missing the step to add the ceph key to the Authentication Keys tab of the cinder External Provider. It's all working now.
Cool. You can also look in the wiki: http://www.ovirt.org/develop/release-management/features/storage/cinder-inte... (Looks dead now, but usually it is up). Please share your experience with this feature, we live to get feedback. Cheers, Nir
Cheers,
Alessandro
Il 30/05/16 10:55, Alessandro De Salvo ha scritto:
Hi, I'm happily using our research cluster in Italy via gluster, and now I'm trying to hotplug a ceph disk on a VM of my cluster, without success. The ceph cluster is managed via openstack cinder and I can create correctly the disk via ovirt (3.6.6.2-1 on CentOS 7.2). The problem comes when trying to hotplug, or start a machine with the given disk attached. In the vdsm log of the host where the VM is running or starting I see the following error:
jsonrpc.Executor/5::INFO::2016-05-30 10:35:29,197::vm::2729::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug disk xml: <disk address="" device="disk" snapshot="no" type="network"> <source name="images/volume-9134b639-c23c-4ff1-91ca-0462c80026d2" protocol="rbd"> <host name="141.108.X.Y1" port="6789" transport="tcp"/> <host name="141.108.X.Y2" port="6789" transport="tcp"/> </source> <auth username="cinder"> <secret type="ceph" uuid="<base 64 ceph secret>"/> </auth> <target bus="virtio" dev="vdb"/> <driver cache="none" error_policy="stop" io="threads" name="qemu" type="raw"/> </disk>
jsonrpc.Executor/5::ERROR::2016-05-30 10:35:29,198::vm::2737::virt.vm::(hotplugDisk) vmId=`c189472e-25d2-4df1-b089-590009856dd3`::Hotplug failed Traceback (most recent call last): File "/usr/share/vdsm/virt/vm.py", line 2735, in hotplugDisk self._dom.attachDevice(driveXml) File "/usr/share/vdsm/virt/virdomain.py", line 68, in f ret = attr(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/libvirtconnection.py", line 124, in wrapper ret = f(*args, **kwargs) File "/usr/lib/python2.7/site-packages/vdsm/utils.py", line 1313, in wrapper return func(inst, *args, **kwargs) File "/usr/lib64/python2.7/site-packages/libvirt.py", line 530, in attachDevice if ret == -1: raise libvirtError ('virDomainAttachDevice() failed', dom=self) libvirtError: XML error: invalid auth secret uuid
In fact the uuid of the secret used by ovirt to hotplug seems to be the ceph secret (masked here as <base 64 ceph secret>), while libvirt expects the uuid of the libvirt secret, by looking at the instructions http://docs.ceph.com/docs/jewel/rbd/libvirt/. Anyone got it working? Thanks,
Alessandro
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
_______________________________________________ Users mailing list Users@ovirt.org http://lists.ovirt.org/mailman/listinfo/users
participants (2)
-
Alessandro De Salvo -
Nir Soffer