57 Commits

Author SHA1 Message Date
paul luse
600341b6c5 bdev/compress: replace RPC parm name with pm_path
We decided to name compress bdevs with COMP_ and then the name of
the underlying bdev so the RPC parm name was stale.  Previously
hardcoded the path to PM for early dev so replaced name with pm_path.

Note that a sample create looks like this now:

rpc.py construct_compress_bdev -b NVMe0n1 -p ~/pm_files -d compress_isal

And that devs need to pay attention for the pathname they provide, its
easy to leave orphaned pm files around and they can get big.

Change-Id: Ifb5245c922461bbecec4bef266bdeb25e8b87f31
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-26 20:07:19 +00:00
Konrad Sztyber
a0cb5e9d77 lib/ftl: retrieve caching bdev from configuration
Added means to configure libftl to use (optionally) another bdev as
persistent write buffer cache.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97028a681be168d9386eac8a226631ff772f803b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448629
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-29 04:42:48 +00:00
Paul Luse
db541f8eb3 bdev/compress: add compression vbdev module
Just the starting point, base functions and structs.

Change-Id: I2ff2b672aa675a19b89e4449f1cc4aa664007f6f
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/435747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-03-27 14:51:22 +00:00
Vitaliy Mysak
bef0c6edd2 ocf: rpc: extend get_ocf_bdevs for multicore cases
Extend existing get_ocf_bdevs call to make it easier
  to get information about attached cores.

Implementation is based adding additional optional argument "name"
  to existing call. Based on "name" bdevs are filtered.
  Backward compatability of RPC interface is preserved.

This patch also adds tests for the case when name is given.

Change-Id: I4300ebe37e936bc5cca8e066b5f09db462a87cf7
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/444841
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-03-26 23:30:31 +00:00
Ben Walker
a741e34180 bdev/nvme: Add configuration parameter to slow down polling
For NVMe devices, in conjunction with the new batching options,
it can be advantageous to artificially delay between polling
for completions. Add an option to slow this rate down.

Change-Id: I0fc92709ff45ead0beb388dda60694bf1ed8b258
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Jim Harris
db524432dc bdev: remove delete_bdev RPC
This RPC was deprecated a couple of releases ago.
bdev modules now each have their own RPC for deleting
bdevs.  Due to how bdevs are created differently on
different modules, it is simply not possible to
have one delete_bdev RPC that would work for all bdev
types.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia46c95dce6e35f7557e8d41c41b8fea382924547

Reviewed-on: https://review.gerrithub.io/c/442615
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-01 08:50:07 +00:00
Shuhei Matsumoto
260f9a77c3 bdev/nvme: Set per-controller PRCHK options by JSON RPC
Add prchk_reftag and prchk_guard to construct_nvme_bdev RPC.
In spdk_rpc_construct_nvme_bdev, create prchk_flags based on them
and pass it to spdk_bdev_nvme_create, and in spdk_bdev_nvme_create,
pass it to create_ctrlr.

A single option enable_prchk may be enough but add separate options
for reftag and guard to clarify that apptag is not supported yet.

The next patch will make per-controller PRCHK options configurable
by .INI config file.

Change-Id: I370ebbe984ee83d133b7f50bdc648ea746c8d42d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-12 09:14:31 +00:00
paul luse
8a1acca65d bdev/raid: Add strip_size_kb rpc param for create
strip_size as an rpc param is now deprecated and can be removed
in a future release.  Either strip_size or strip_size_kb can be
used but only one of them or the rpc will fail.

Internally we maintain both fields because strip size always
comes in as KB but we convert it to blocks so having both elements
makes it clear for developers what they're looking at.

JSON output includes both strip_size and strip_size_kb.

Fixes #550

Change-Id: I5dc51e8af22eae3d56af8f8d37a564dbaae228fa
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-11 22:46:33 +00:00
Vitaliy Mysak
199080cfa2 OCF: rpc: add get_ocf_bdevs method
Add new RPC method for OCF bdev: get_ocf_bdevs
It is useful in respect to not registered OCF bdevs
  which do not appear in standard get_bdevs call

Change-Id: I8a5fc86a880b04c47d5f139aa5fa4d07ca39c853
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-28 18:53:31 +00:00
Vitaliy Mysak
27e0190b84 OCF: rpc: add stats method
Add get_ocf_stats rpc method that
  returns available OCF statistics for specific OCF bdev

Change-Id: If043a18c847fbeeddd8fdde1af9397e24bd90718
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-28 18:53:31 +00:00
Vitaliy Mysak
35366140d8 OCF: add RPC
Add rpc methods for vbdev_ocf:
  * construct_ocf_bdev
  * delete_ocf_bdev

Change-Id: Ie75a2b4bf4f88462c00651d195d22d156478abf8
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-28 18:53:31 +00:00
Wojciech Malikowski
7c233adeff bdev_ftl: Added FTL bdev module
This patch adds FTL bdev. RPC scripts have been updated to allow for
creation and removal of FTL bdevs.

Change-Id: I82a5c5033b65bbeb67c238cae969a68cff767dcc
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431329
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 18:55:56 +00:00
Piotr Pelplinski
e5e427c94b histograms: add rpc calls
This patch adds RPC calls for histograms in bdev layer.
Following calls are added:
 - enable_bdev_histogram - enable/disable histogram structures for specified bdev and each of its channels.
 - get_bdev_histogram - merges histograms from all channnels and encodes histogram as base64


Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib423a919dc1cde7dd7d92247db5482cfb9d66956
Reviewed-on: https://review.gerrithub.io/c/433573
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 14:14:32 +00:00
GangCao
05b43152b2 QoS/Bdev: add RPC support for Read/Write separate bandwidth limits
This patch adds the RPC support for the Read/Write separate
bandwidth limit controls. The basic usage as following:

usage:
rpc.py set_bdev_qos_limit [-h] [--rw_ios_per_sec RW_IOS_PER_SEC]
                               [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                               [--r_mbytes_per_sec R_MBYTES_PER_SEC]
                               [--w_mbytes_per_sec W_MBYTES_PER_SEC]
                               name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --r_mbytes_per_sec R_MBYTES_PER_SEC
             Read megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --w_mbytes_per_sec W_MBYTES_PER_SEC
             Write megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I822ec4814d21adff9826ce03a6af3783b1b98f44
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/417650
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
Seth Howell
cc7a099c0c rpc: reconcile options for construct_nvme_bdev
Change-Id: I7a13559312979f5019b42305ebe02ec102dda2a1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437574
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
088379cf74 nvme_rdma: Add source addr info to RPC
Change-Id: Id6b5aee4b36d828a0f9e5ddc85293a15342d2aae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Patrick Ohly
4cfae03606 bdev/rbd: full control over client configuration
When the caller of the RPC API has all the necessary information about
how to access a Ceph cluster, then having to create configuration
files before calling the RPC API is problematic (has to touch files
owned by a local admin, changes must be removed again).

But having to encode support for certain configuration options in SPDK
is also problematic, because that might change depending on the
librados version.

The approach taken here is to merely pass through arbitrary key/value
config options. Existing config files are ignored when that happens.
The caller of the RPC then has full control over the connection setup
and can be sure that he does not inherit settings from a local file
accidentally.

In addition, user management is supported now, with or without a
config. This is useful for accessing a volume with a less privileged
user. Previously, passing NULL to rados_create implicitly chose the
"admin" user.

Change-Id: I5e7f36092df663a3d7ac503c04fc624a8fe1208e
Signed-off-by: Patrick Ohly <patrick.ohly@intel.com>
Reviewed-on: https://review.gerrithub.io/430460
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-05 17:49:57 +00:00
paul luse
164d79143b passthru: add RPC testing
And change names of parms to be consistent with others used in
RPC testing.

Change-Id: I8331c6a22866d89a2a4ffb5fc8d41d74b4b7b07d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/428724
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-06 17:32:08 +00:00
Liu Xiaodong
98187ed9a7 rpc: revise comments for construct_nvme_bdev
Just considering the situation of multi-namespaces device.

Change-Id: I1b0d502874ab53891060c973a3ae75e81ec591f2
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/431698
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-02 19:02:43 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
Ben Walker
d705ab93e1 Check file permissions in the check_format script
Change-Id: I572b24b2f4f149333f2e4b04f6f4961de9c23373
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/427539
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-04 23:08:12 +00:00
paul luse
cafd537c7d test: exercise crypto rpc calls
Most vbdevs have complete test suites that include rpc calls
to set them up and teare them down.  With crypto there's not
a lot to do w/test other than make sure it's configured and
run IO so right now that amounts to tools that use a config
(ini) file.  Adding create/delete calls here to get basic rpc
coverage in CI.

The fixes/udpates in the vbdev module and the .py files were
fixes found in enabling testing of the rpc calls.

Change-Id: I5d7d25e1c96dcf76e0055f23d27781b858e290b0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425875
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-27 22:07:02 +00:00
Pawel Wodkowski
4dc0ef6f45 scripts: fix executable attribute for includes
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.

Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-18 16:34:45 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Pawel Wodkowski
5f9bdac004 bdev/nvme: add RPC to enable and disable hotplug
Change-Id: Iac7340dd0df427ba4e9df5b1c52cfb732d011ae8
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419093
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-26 22:12:29 +00:00
Pawel Wodkowski
4bef621e90 bdev/nvme: Add set_bdev_nvme_options
Change-Id: I5ab027d5204f9c4991eacaea631367783d1ad115
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/418723
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-26 22:12:29 +00:00
Tomasz Zawadzki
3a56fabce3 bdev: add delete_nvme_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for NVMe bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib9a0475d735af2616a3005d04530ca825ece8a52
Reviewed-on: https://review.gerrithub.io/416546
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-07-26 22:07:50 +00:00
Kunal Sablok
41586b0f1d bdev: add raid bdev module
Raid module:
============
- SPDK raid bdev module is a new bdev module which is
  responsible for striping various NVMe devices and expose the raid bdev
  to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
  tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
  supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
  are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
  current SPDK configuration file for across reboot support. DDF will be
introduced later.

High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
  file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
  tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
  Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
  are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
  are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
  raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
  relationship was returned in JSON RPC commands

Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-16 20:50:40 +00:00
Seth Howell
0398702483 bdev: add rpc for enabling queue_depth monitoring
Change-Id: Idb5e29b5d5b1d3431fbd1051ccaa6218f9a09600
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-12 23:27:27 +00:00
Maciej Szwed
e48f0bf17d bdev: add delete_passthru_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for passthru bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8ea20b3003dd6539d84123c3b5363bd8bdfd6f7f

Reviewed-on: https://review.gerrithub.io/416535
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-02 22:53:18 +00:00
Maciej Szwed
1a0ce4ded0 bdev: add delete_pmem_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for pmem bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ic526e9ae462f595c4668c2b2612ad074208a7c4e
Reviewed-on: https://review.gerrithub.io/416520
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-02 22:53:18 +00:00
Maciej Szwed
4fd967b22a bdev: add delete_rbd_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for rbd bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ia87459e0cc49a0c408de582bd1f5680c570d42d9
Reviewed-on: https://review.gerrithub.io/416529
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-02 22:53:18 +00:00
Maciej Szwed
2ce9836b17 bdev: add delete_null_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for null bdev.

Changes in spdkcli done accordingly.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I576322257e5cf70ec03d6b20f8ba43bfce222907
Reviewed-on: https://review.gerrithub.io/416505
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-02 22:53:18 +00:00
Tomasz Zawadzki
160850f0a6 rpc/bdev: construct RPC return a string instead array
Construct calls that create single new bdev,
now return a string instead of array with single element.

Change-Id: Iad841b8e5eab0265693efb6be472354ea616c562
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/416081
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-22 21:12:43 +00:00
Tomasz Zawadzki
6be654aad7 bdev: add delete_iscsi_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for iSCSI bdev.

No changes in spdkcli, since it is not implemented yet.
Tests unchanged, because test is only done by running
bdevperf and exiting. Without time to delete_iscsi_bdev.

Change-Id: I8341bfb65932a1deb1fb6463e11ef932d6f6d597
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/415678
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-22 17:24:18 +00:00
Tomasz Zawadzki
0847ad90fd bdev: add delete_error_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for error injection bdev.

Changes in tests and spdkcli done accordingly.

Change-Id: I7f0be09cfed582c4d1060871e40c41eff5af7129
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/415502
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-22 17:24:05 +00:00
Tomasz Zawadzki
59dee78bd6 bdev: add delete_malloc_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for malloc.

Changes in tests and spdkcli done accordingly.

Change-Id: I5296452f552ee6eaba4d012d47aea022f17b9d6e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/415259
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-22 15:17:44 +00:00
Tomasz Zawadzki
96e93bd8d5 bdev: add delete_aio_bdev call
Since delete_bdev should be used only for debug purpose,
this patch adds delete call specific for AIO.

Changes in tests and spdkcli done accordingly.

Change-Id: Ib732953a2c138dc25915ba97ce3a8acf771ad74f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/415462
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-19 20:43:12 +00:00
Jim Harris
3090f4012e bdev: add set_bdev_options RPC
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I060679165bd4215d010e74407aedd1aedc20fb9a

Reviewed-on: https://review.gerrithub.io/414713
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-14 14:57:11 +00:00
Pawel Wodkowski
fba68d61e1 bdev/iscsi: add RPC support and cofnig dump
Also, as we are here, switch to new spdk_json_write_named_* API in
bdev_iscsi_dump_info_json()

Change-Id: Ib29466eb5c6d6496dd1d8efc1f2064577bf56272
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/406635
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-05 17:52:06 +00:00
Pawel Niedzwiecki
88168dd8b8 rpc: fix construct_split_vbdev and destruct_split_vbdev
Change-Id: I6110b6bcbb66b7a7ae412f02247a1c604ee4c276
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/412456
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-30 00:37:15 +00:00
Daniel Verkamp
1ea9765460 scripts/rpc.py: pass named args to bdev.py
Also add docstrings to all bdev.py methods.

Change-Id: Ib980d6141b3933f0df26a09e5242f27afc562f96
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/411952
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-24 05:10:36 +00:00
Yanbo Zhou
64ccd4b95b bdev: a new public function to get the I/O statistics of bdev
Add a new function and its RPC caller. By using it, we can
get the statistics of all the bdevs or the specified bdev.

Meanwhile, with this patch, the open source tool 'sysstat/iostat'
can support for SPDK. The 'iostat' tool can call this function to
get the statistics of all the SPDK managed devices via the rpc
interface.

Change-Id: I135a7bbd49d923014bdf93720f78dd5a588d7afa
Signed-off-by: Yanbo Zhou <yanbo.zhou@intel.com>
Reviewed-on: https://review.gerrithub.io/393130
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-23 17:49:21 +00:00
GangCao
cb1c88d19f vbdev/passthru: add the construct_passthru_bdev RPC method
Add the RPC method to construct a passthru bdev.

The usage as following:

usage:
rpc.py construct_passthru_bdev [-h] -b BASE_BDEV_NAME -p PASSTHRU_BDEV_NAME

Change-Id: I87bcfde499a9f0c2f5758e36e6772e0fc2928d20
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/406891
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-20 14:58:13 -04:00
GangCao
ffba4fdbc3 bdev/qos: add RPC method to set QoS at runtime
This patch adds a new RPC method to configure QoS on bdev
at runtime.

For example:

set_bdev_qos_limit_iops Malloc0 20000 --> Enable QoS on this
block device with 20000 IOPS rate limiting.

Change-Id: I1ee8b313b769fb5a664820f4ba827e0230be4b5d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/393255
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-19 13:47:54 -04:00
Karol Latecki
c43dc1f1e3 scripts/rpc.py: calculate num_blocks with floor division
If called with python 3 num_blocks will be calculated
as float number with .0 decimal part.
Change to floor division so that is't always int.

Change-Id: Ic0aae404e21a1bdfe7db291532d80d4b4598d307
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/407547
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-13 15:44:46 -04:00
Pawel Wodkowski
a5dbccf02b bdev/split: add RPC support
Split bdevs now can be create using "create_split_bdev". To keep
backward compatibility, this call will not fail if base bdev is not
available yet instead will add base bdev name to its live configuration
and create splits when base bdev will be created (during examin
process).

Change-Id: Ie26ffc0e947f6d88ff56830dd50999795283df2e
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/404164
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-04 16:03:00 -04:00
Daniel Verkamp
96dc91d608 scripts/rpc.py: pass client as separate parameter
Split client out of the args object and pass it as the first parameter
to all RPC methods instead.  This is a step toward decoupling the
rpc/*.py interface from the argparse front end.

Change-Id: Ib030862e0c79112e5c9acdde295d68983126a987
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405502
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-30 16:48:47 -04:00
Daniel Verkamp
df897ed490 scripts/rpc.py: move printing into main rpc.py
The modules in scripts/rpc/*.py should be a generic, reusable library;
the printing should be done by the command-line frontend (rpc.py)
instead of the library code.

Change-Id: Ibeb022a3591f0a140fc43104d8dcf17d7041e48b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/404426
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-26 00:57:53 -04:00