Raid module:
============
- SPDK raid bdev module is a new bdev module which is
responsible for striping various NVMe devices and expose the raid bdev
to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
current SPDK configuration file for across reboot support. DDF will be
introduced later.
High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
relationship was returned in JSON RPC commands
Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Also remove libspdk_net from the list and add it to each app that
originally linked it via NET_MODULES.
This will allow the use of SOCK_MODULES in application Makefiles without
pulling in the RPC libraries required by spdk_net.
Change-Id: Iad68ddd25bea24bba53a681c78d41e6e703f866d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416476
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This separates the network interface management code (interface.c and
net_rpc.c) from the socket abstraction layer, which allows users that
only want the socket abstraction to avoid pulling in the JSON RPC
libraries.
Change-Id: I9b00285a70bac0c74c73353cfa900d4f3b2e465f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416475
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch integrates VPP userspace TCP/IP stack,
as optional network framework to be used in SPDK.
Support is done via VPP Communications Library.
Change-Id: I4c2945c76878dbf24ff2f8612e0a21615f7d87e6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/389566
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
At this time blobcli makes use of all the bdev modules
during compilation. Yet vbdev_lvol does examine,
that already creates bs_dev on a bdev and loads
blobstore for all devices where lvol store exists.
Since bs_dev cannot be created on device twice,
same as blobstore cannot be loaded twice.
It prevented blobcli from displaying information
for blobstore underlying lvol store.
This patch remove vbdev_lvol from list of bdev modules
for blobcli.
Change-Id: I144ba22ec28f7f315e768745f5dd0cbbe3bbad86
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/404832
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
NVML (Non Volatile Memory Library) changed its name to
PMDK (Persistent Memory Development Kit), so make the
necessary changes to the SPDK repository.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id07a87eafb4e9a3099603030c58c0e927bafa608
Reviewed-on: https://review.gerrithub.io/406256
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This uses libiscsi to implement an iSCSI initiator bdev
module for SPDK. Still a lots of work to do on this - posting
it in case anyone is interested in working on this further.
A number of todo items are listed in a README in the lib/bdev/iscsi
directory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Change-Id: I060e33de0cd6796246789bf0e1bb4f2df59d8f71
Reviewed-on: https://review.gerrithub.io/390313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Example showing the basics of a vbdev via a module that implements
the main functions, and a few extras, needed for a vbdev. The tests
also demonstrate how to configure an application to use a vbdev and
an underlying bdev
Change-Id: I67bf542143eb6532465cdb6af63c0f2ab7084c0e
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/399768
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Enable dynamic compilation of SPDK with VPP.
Change-Id: I9f5f7223625cbebec0d6209b5a2ce6bdcc1a04bf
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/401825
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id370631770cb2e090c3efbe95cd86db4f6168c5e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/403702
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Exported lib/bdev/rte_virtio as a separate
library not dependent on bdev.
Virtio is now accessible via spdk_internal/virtio.h.
The header is marked `internal`, as it's
not meant to be used by end users. It's
not handy to handle all backend-specific
(e.g. Virtio-SCSI) logic in a user code.
For now the Virtio interface is publicly
exposed only via bdev_virtio module. We
might want to consider adding a separate,
public Virtio-SCSI library in the future.
Note: this patch doesn't do any changes
to the virtio code. Everything is
moved 1:1.
Change-Id: I805e5d12d265d82b0bc5784c89fbadb81abdb278
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/388166
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Use SPDK bdev subsystem to allow basic IO to pmem pool
using pmemblk_* calls.
New pmem bdev can be constructed using RPC call
construct_pmem_bdev PMEM_FILE
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If5ca94161fe53644b8fccd3b77de5479da1b2e55
Reviewed-on: https://review.gerrithub.io/376973
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia96ae78ff9530d953181ac5f7255a38f3c8ec430
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Reviewed-on: https://review.gerrithub.io/375392
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Supports both PCI mode (for usage in guest VMs) and
vhost-user mode (for usage in host processes). The rte_virtio
subdirectory contains a lot of code lifted from the DPDK
virtio-net driver. Most of the PCI and vhost-user code is
reused almost exactly as-is, but the virtio code is drastically
rewritten as the DPDK code was very network specific.
Has been lightly tested with both the bdevio and bdevperf
applications in both PCI and vhost-user modes.
Still quite a bit of work needed - a list of todo
items is included in a README in the module's directory.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I85989d3de9ea89a87b719ececdb6d2ac16b77f53
Reviewed-on: https://review.gerrithub.io/374519
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I6a138e1c1d775e8c8bfeede9600f8b0f799ecdad
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/362445
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Also change the discovery/nvmf.sh test to use it.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I56bce9a84bd46f13b6d4f34da81abf23413f2598
This virtual block device takes an underlying block device and splits it
into several smaller equal-sized block devices.
Change-Id: I6f6e686c1177b2e4885f7e88809ad329caae55bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make it easier to use SPDK libraries by putting them all in a single
directory that can be added with -L rather than scattered around the
source tree.
Change-Id: I5c0f5dd6e7058b5f92fa9bc41548190ffc064761
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch will add a new bdev module, rbd.
It can make ceph rbd as the backend of iSCSI
target.
Change-Id: Id5eb3b159ee607052e3c33a2e59d721739fd9977
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
