[ovirt-devel] Re: [ovirt-users] Re: [=EXTERNAL=] Re: help using nvme/tcp storage with cinderlib and Managed Block Storage
On Thu, Feb 24, 2022 at 3:58 PM Nir Soffer <nsoffer(a)redhat.com> wrote:
On Wed, Feb 23, 2022 at 6:24 PM Muli Ben-Yehuda
<muli(a)lightbitslabs.com>
wrote:
>
> Thanks for the detailed instructions, Nir. I'm going to scrounge up some
hardware.
> By the way, if anyone else would like to work on NVMe/TCP support, for
NVMe/TCP target you can either use Lightbits (talk to me offline for
details) or use the upstream Linux NVMe/TCP target. Lightbits is a
clustered storage system while upstream is a single target, but the client
side should be close enough for vdsm/ovirt purposes.
I played with NVMe/TCP a little bit, using qemu to create a virtual
NVMe disk, and export
it using the kernel on one VM, and consume it on another VM.
https://futurewei-cloud.github.io/ARM-Datacenter/qemu/nvme-of-tcp-vms/
One question about device naming - do we always get the same name of the
device in all hosts?
No, we do not, see below how we handle migration in os_brick.
To support VM migration, every device must have unique name in the cluster.
With multipath we always have unique name, since we disable
"friendly
names",
so we always have:
/dev/mapper/{wwid}
With rbd we also do not use /dev/rbdN but a unique path:
/dev/rbd/poolname/volume-vol-id
How do we ensure cluster-unique device path? If os_brick does not handle
it, we
can to do in ovirt, for example:
/run/vdsm/mangedvolumes/{uuid} -> /dev/nvme7n42
but I think this should be handled in cinderlib, since openstack have
the same problem with migration.
Indeed. Both the Lightbits LightOS connector and the nvmeof connector do
this through the target provided namespace (LUN) UUID. After connecting to
the target, the connectors wait for the local friendly-named device file
that has the right UUID to show up, and then return the friendly name. So
different hosts will have different friendly names, but the VMs will be
attached to the right namespace since we return the friendly name on the
current host that has the right UUID. Does this also work for you?
Cheers,
Muli
--
*Lightbits Labs**
*Lead the cloud-native data center
transformation by
delivering *scalable *and *efficient *software
defined storage that is
*easy *to consume.
*This message is sent in confidence for the addressee
only. It
may contain legally privileged information. The contents are not
to be
disclosed to anyone other than the addressee. Unauthorized recipients
are
requested to preserve this confidentiality, advise the sender
immediately of
any error in transmission and delete the email from their
systems.*
Attachments:
- attachment.html (text/html — 4.2 KB)