d/numam-spdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

NVMf: Add a custom(VFIOUSER) transport target support

Browse Source 
VFIO is a framework that allows a physical device to be securely passed through
to a user space process. Typically, the user space process is a VM and the device
is passed through to it in order to achieve high performance. VFIO provides an API
and the required functionality in the kernel.

vfio-user is a framework that allows implementing PCI devices in userspace.

SPDK NVMf library provides the emulation of a NVMe device, combined with vfio-user
protocol, we can provide the emulated NVMe device to VM or client application, such
as SPDK NVMe perf.

This patch has dependency on libvfio-user library, users can start NVMe identify/perf
examples.

Usage:
1. Start nvmf_target.
2. Use RPC to create a subsystem with namespace bdev.

    mkdir -p /var/run/muser
    mkdir -p /var/run/muser/iommu_group
    mkdir -p /var/run/muser/domain/muser0/8
    mkdir -p /dev/shm/muser/muser0

    scripts/rpc.py nvmf_create_transport -t VFIOUSER
    scripts/rpc.py nvmf_create_subsystem -a nqn.2019-07.io.spdk.muser:/var/run/muser/domain/muser0/8
    scripts/rpc.py bdev_malloc_create -b muser0 $((512)) 512
    scripts/rpc.py nvmf_subsystem_add_ns -n 1 nqn.2019-07.io.spdk.muser:/var/run/muser/domain/muser0/8 muser0
    scripts/rpc.py nvmf_subsystem_add_listener -t VFIOUSER -a "/var/run/muser/domain/muser0/8" -s 0 nqn.2019-07.io.spdk.muser:/var/run/muser/domain/muser0/8

    ln -s /var/run/muser/domain/muser0/8 /var/run/muser/domain/muser0/8/iommu_group
    ln -s /var/run/muser/domain/muser0/8 /var/run/muser/iommu_group/8
    ln -s /var/run/muser/domain/muser0/8/bar0  /dev/shm/muser/muser0/bar0

Currently, file /var/run/muser/domain/muser0/8/ctrlr is the socket file which is used
to transfer socket messages between target and client.

[1] https://www.mail-archive.com/qemu-devel@nongnu.org/msg723794.html

Co-authored-by: Ben Walker <benjamin.walker@intel.com>
Co-authored-by: Changpeng Liu <changpeng.liu@intel.com>
Co-authored-by: Thanos Makatos <thanos.makatos@nutanix.com>

Signed-off-by: Thanos Makatos <thanos.makatos@nutanix.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change-Id: I5f2a8e77314a6a13aa6366dd4a6dc77b13434e7b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3838
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This commit is contained in:
Changpeng Liu 2020-10-26 23:37:10 +08:00 committed by Jim Harris
parent 6308a24f56
commit f4fd07c68c
2 changed files with 2382 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

7
lib/nvmf/Makefile
View File

@ -62,6 +62,13 @@ endif
endif
endif
ifeq ($(CONFIG_VFIO_USER), y)
C_SRCS += vfio_user.c
CFLAGS += -I$(VFIO_USER_INCLUDE_DIR)
LDFLAGS += -L$(VFIO_USER_LIBRARY_DIR)
LOCAL_SYS_LIBS += -Wl,-Bstatic -lvfio-user -Wl,-Bdynamic -ljson-c
endif
ifeq ($(CONFIG_FC),y)
C_SRCS += fc.c fc_ls.c
CFLAGS += -I$(CURDIR) $(ENV_CFLAGS)

2375
lib/nvmf/vfio_user.c Normal file
View File

File diff suppressed because it is too large Load Diff