Commit Graph

422 Commits

Author SHA1 Message Date
Ben Walker
5c0f6700aa CHANGELOG: Update env section
Change-Id: Ie0c9276f5321b103ef399f387106ec1789522392
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452690
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-30 19:33:48 +00:00
Ben Walker
13c416c027 CHANGELOG: Update thread section
Change-Id: Ie8f167e8ea9a72bfb559f644c6556c66ec0256b4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452689
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-30 19:33:48 +00:00
Ben Walker
0017ba8b4b CHANGELOG: Update nvme section
Change-Id: I4fc9e597281323baf8f03f5977db264efff77fbb
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452688
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-30 19:33:48 +00:00
Darek Stojaczyk
ae628a844e CHANGELOG: mention AIO completion reaping
Change-Id: I4bf17559eeb372211bc32d0a2d1cb72db9f99f17
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-30 19:33:48 +00:00
Darek Stojaczyk
ef0093adf2 CHANGELOG: add missing vhost updates
Change-Id: Ie5a3defd8bc12b724799b9a4801574f8831732ff
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-30 19:33:48 +00:00
Darek Stojaczyk
59688f4db7 CHANGELOG: merge two "thread" sections
Change-Id: I573baddeb52537459083b92e78eab612dc0e0f70
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/452636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-30 19:33:48 +00:00
Chunyang Hui
6b48e743a3 Opal: Add revert tper cmd option
Reset the device to its factory defaults.

Change-Id: I43f7dc8fb7bd5226283a4762beac0e2cf016f698
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-25 18:20:42 +00:00
Chunyang Hui
7250ec64db Opal: add take ownership cmd options
Also make some modification for invoking process in
nvme_manage tool.

Change-Id: Ib54db43d7336d3e839e7d9317c292b9a57b38f80
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-24 22:40:04 +00:00
Shuhei Matsumoto
f74643ef0f CHANGELOG: Support APIs for metadata and DIF in bdev layer
Change-Id: Ib47a854de93bd4b130ab7f663848db651eb873dd
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-24 16:55:23 +00:00
Shuhei Matsumoto
462ff07b37 CHANGELOG: Support DIF insert and strip feature in iSCSI target
Change-Id: Ie4f78688a0c6767eb48401c75f58502616b0c201
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-04-24 16:55:23 +00:00
Shuhei Matsumoto
d4b5edae66 CHANGELOG: Support DIF passthrough feature in NVMe-oF target
Change-Id: I3baf48a4caad0616e74d46d2b25eb97010c54ed0
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451643
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 16:55:23 +00:00
Shuhei Matsumoto
31158c0dd0 CHANGELOG: Fix the wrong location for NVMe-oF target section
Merge nvmf section into NVMe-oF target section.

Change-Id: I0279392e655b719ff6229b00838fcc6406265113
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451787
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-24 16:55:23 +00:00
Ben Walker
84850dacd7 bdev: Add a zero copy I/O path
Add a ZCOPY operation to obtain buffers that represent
data regions on the backing block device.

Change-Id: Ie941c16ee051d0009e3888b52b8f41773bba47b3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/386166
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-23 18:38:39 +00:00
Gregory Shapiro
14032a984c NVMF: Add model number as parameter to construct_nvmf_subsystem (-d option).
Change-Id: Ia1a458a0ac1c5a17d2955a3f31c6dfe77538eb17
Signed-off-by: Gregory Shapiro <gregory.shapiro@kaminario.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/438562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-04-23 16:51:16 +00:00
Darek Stojaczyk
81523d9dd2 env/dpdk: register VFIO hotremove callback
This is an attempt to fix device hotremove with VFIO.

A soft device hotremove request through sysfs [1] would
currently just block until the SPDK process manually
releases that device - e.g. upon an RPC request.

VFIO won't get unbound from the device untill userspace
releases all its resources. VFIO can signal a pending
hotremove request by kicking any file descriptor provided
by the userspace - and DPDK does provide such descriptor -
but SPDK does not listen on it.

DPDK does offer handy API to listen and in this patch
we make use of it inside our env/pci layer. Within
a DPDK callback we set an internal per-device hotremove
flag, which upper-layer SPDK drivers can poll with a new
env API - spdk_pci_device_is_removed().

The VFIO hotremove event will be sent to primary
processes only, so that's where we listen.

We make use of this new API in the NVMe hotplug poller,
which will process it just like any other supported
hotremove event.

Fixes #595
Fixes #690

[1] # echo 1 > /sys/bus/pci/devices/<bdf>/remove

Change-Id: I03d88271c2089c740e232056d9340e5a640d442c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448927
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-18 22:49:30 +00:00
Hui, Chunyang
55f5b205c5 Opal: Add Opal library
First submission. Implemented part of the Opal library
and "scan" function. Can be invoked by nvme_manage.

Change-Id: Iba86d86dd3af06a06b6805120ee5005af8183459
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/439335
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-04-08 17:04:25 +00:00
Xiaodong Liu
efdd6edba2 nbd: optional nbd_device in start_nbd_disk
Enhance RPC method start_nbd_disk to take nbd_device as
one optional parameter. If it is not assigned, automaticly
choose an available nbd device path from /dev/nbd0 to
/dev/nbdN.
For github issue #324:
https://github.com/spdk/spdk/issues/324

Change-Id: I72c064d8bd476df342f5aa0af4d6120eb021c7ed
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/440453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-04-04 08:35:50 +00:00
Jim Harris
bfd1e46e6e bdev: deprecate spdk_vbdev_register
This API had good intentions, but as more complicated
use cases came up where base bdevs could come and go,
we've realized that the bdev layer will need another
mechanism to query bdev modules on these types of
relationships between a virtual bdev and its base
bdevs.  We removed all code related to tracking
the array of base bdevs a long time ago.

Change all existing callers to use spdk_bdev_register.
Document spdk_vbdev_register as deprecated for now,
and change its implementation to just call
spdk_bdev_register for simplicity sake.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b40ed96480c0fa7184db42953a9f4e4c167fed1

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/450076
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-04-04 08:28:31 +00:00
Chunyang Hui
38902a5a27 rpc: add get_spdk_version rpc method
Change-Id: Iacc1766a60fb1e15a1013475f83f7f13cd45fe12
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448613
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-26 22:54:28 +00:00
Darek Stojaczyk
f373369a9c env: add spdk_env_fini()
The function now has to be called before application
exit. At the moment it only frees the dynamically
allocated DPDK command line option strings - something
that was previously done from an atexit() callback -
but there's more to free there.

Note: the function descriptions were partially copied
from equivalent DPDK functions.

Change-Id: I5f4a6607fdfadff9325917259f58fcbc2cedba1a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-20 21:05:33 +00:00
Darek Stojaczyk
6f48bf7cb6 env/dpdk: drop support for DPDK < 17.11
DPDK 17.11 is the oldest version still supported by DPDK,
so drop support for DPDKs older than that in SPDK. This
lets us remove a huge amount of ifdefs.

Change-Id: I500987648e388cd5418a25845b6cccf4b55a4e5b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447674
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-20 21:05:33 +00:00
Darek Stojaczyk
55656fb19f env: deprecate phys_addr param in spdk_*malloc()
Historically, all memory returned from spdk_*malloc()
used to be physically contiguous, hence it could be
addressed by offsetting just a single physical address.

Since DPDK dynamic memory management came along, the
above is no longer true. Memory returned from spdk_*malloc()
doesn't have to be physically contiguous anymore. The
phys_addr returned from spdk_*malloc() only applies to
the beginning of the allocated buffer and user can't
possibly know how big that "beginning" is.

The phys_addr parameter in spdk_*malloc() is useless on
its own in most cases and only suggests that the returned
buffer is physically contiguous, which is wrong.

This information can be returned from spdk_vtophys(),
which is the only safe way to retrieve physical addresses.
That's why phys_addr param in spdk_*malloc() is now
deprecated.

Change-Id: I934292f7db28b869b05caca4cb5c68c436e228d4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/448168
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-20 01:06:09 +00:00
Ben Walker
cf0eac8c66 nvme: Add qpair option to batch command submissions
Avoid ringing the submission queue doorbell until the
call to spdk_nvme_qpair_process_completions().

Change-Id: I7b3cd952e5ec79109eaa1c3a50f6537d7aaea51a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-19 07:27:44 +00:00
Seth Howell
01201d3e87 rdma: remove compile time config for SRQ
Change-Id: I44af3ee4dc6ec76045e1d0614910402487098a3d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447120
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-18 23:32:21 +00:00
Evgeniy Kochetov
ed0b611fc5 nvmf/rdma: Add shared receive queue support
This is a new feature for NVMEoF RDMA target, that is intended to save
resource allocation (by sharing them) and utilize the
locality (completions and memory) to get the best performance with
Shared Receive Queues (SRQs). We'll create a SRQ per core (poll
group), per device and associate each created QP/CQ with an
appropriate SRQ.

Our testing environment has 2 hosts.
Host 1:
  CPU: Intel(R) Xeon(R) CPU E5-2609 0 @ 2.40GHz dual socket (8 cores total)
  Network: ConnectX-5, ConnectX-5 VPI , 100GbE, single-port QSFP28, PCIe3.0 x16
  Disk: Intel Optane SSD 900P Series
  OS: Fedora 27 x86_64
Host 2:
  CPU: Intel(R) Xeon(R) CPU E5-2630 v2 @ 2.60GHz dual-socket (24 cores total)
  Network: ConnectX-4 VPI , 100GbE, dual-port QSFP28
  Disk: Intel Optane SSD 900P Series
  OS : CentOS 7.5.1804 x86_64
Hosts are connected via Spectrum switch.
Host 1 is running SPDK NVMeoF target.
Host 2 is used as initiator running fio with SPDK plugin.

Configuration:
- SPDK NVMeoF target: cpu mask 0x0F (4 cores), max queue depth 128,
  max SRQ depth 1024, max QPs per controller 1024
- Single NVMf subsystem with single namespace backed by physical SSD disk
- fio with SPDK plugin: randread pattern, 1-256 jobs, block size 4k,
  IO depth 16, cpu_mask 0xFFF0, IO rate 10k, rate process “poisson”

Here is a full fio command line:
fio  --name=Job --stats=1 --group_reporting=1 --idle-prof=percpu \
--loops=1 --numjobs=1 --thread=1 --time_based=1 --runtime=30s \
--ramp_time=5s --bs=4k --size=4G --iodepth=16 --readwrite=randread \
--rwmixread=75 --randrepeat=1 --ioengine=spdk --direct=1 \
--gtod_reduce=0 --cpumask=0xFFF0 --rate_iops=10k \
--rate_process=poisson \
--filename='trtype=RDMA adrfam=IPv4 traddr=1.1.79.1 trsvcid=4420 ns=1'

SPDK allocates the following entities for every work request in
receive queue (shared or not): reqs (1024 bytes), recvs (96 bytes),
cmds (64 bytes), cpls (16 bytes), in_capsule_buffer. All except the
last one are fixed size. In capsule data size is configured to 4096.
Memory consumption calculation (target):
- Multiple SRQ: core_num * ib_devs_num * SRQ_depth * (1200 +
  in_capsule_data_size)
- Multiple RQ: queue_num * RQ_depth * (1200 + in_capsule_data_size)
We ignore admin queues in calculations for simplicity.

Cases:
1. Multiple SRQ with 1024 entries:
   - Mem = 4 * 1 * 1024 * (1200 + 4096) = 20.7 MiB
     (Constant number – does not depend on initiators number)
2. RQ with 128 entries for 64 initiators:
   - Mem = 64 * 128 * (1200 + 4096) = 41.4 MiB

Results:
FIO_JOBS   kIOPS     Bandwidth,MiB/s  AvgLatency,us  MaxResidentSize,kiB
       RQ       SRQ     RQ      SRQ    RQ       SRQ      RQ       SRQ
1      8.623    8.623   33.7    33.7   13.89    14.03    144376   155624
2      17.3     17.3    67.4    67.4   14.03    14.1     145776   155700
4      34.5     34.5    135     135    14.15    14.23    146540   156184
8      69.1     69.1    270     270    14.64    14.49    148116   156960
16     138      138     540     540    14.84    15.38    151216   158668
32     276      276     1079    1079   16.5     16.61    157560   161936
64     513      502     2005    1960   1673     1612     170408   168440
128    535      526     2092    2054   3329     3344     195796   181524
256    571      571     2232    2233   6854     6873     246484   207856

We can see the benefit in memory consumption.

Change-Id: I40c70f6ccbad7754918bcc6cb397e955b09d1033
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/428458
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-15 19:19:17 +00:00
Changpeng Liu
84245b7202 nvme: add spdk_nvme_connect_async() API
Users may want to connect specified controller at running time,
so this API will connect to the controller and return probe context
to users, users must call spdk_nvme_probe_poll_async() to initialize
the controller to the READY state before using it.

Change-Id: I232886b000454ee826ea73c4e1043d0d18ee0ec6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-14 22:37:02 +00:00
Shuhei Matsumoto
fb16799191 bdev: Add spdk_bdev_get_data_block_size API.
iSCSI target will need to get data block size except for metadata.
Other SPDK application may need to get the same information
future. Hence this patch adds an new API spdk_bdev_get_data_block_size
to bdev layer. In the header file, spdk_bdev_get_data_block_size
is located next to spdk_bdev_is_md_interleaved to avoid confusion
by new users.

Change-Id: I0fd2a6d0bcf6a4c18c583f70d96cc5035fc57fe9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/445082
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-03-07 07:03:27 +00:00
Chunyang Hui
51ab378862 nvme: Add getting supported flag for controllers
New API added for upper level to get controllers'
supported flags.

Change-Id: I51e9d0e57c355fa37f092602a94f4c08deb8898c
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446091
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-07 00:03:34 +00:00
Ben Walker
362879570a event: Remove arg2 from spdk_app_start()
We never used this anywhere, and I need to move to a model where
the start up action is a thread message instead

Change-Id: I6b21ba9afb93a3245aceca2fe24713ffd16d0933
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446986
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-03-05 08:43:12 +00:00
Tomasz Zawadzki
154eb3399a thread: add spdk_thread_is_idle()
This function add possibility to check if there are any scheduled operations
on particular thread.

Return from spdk_thread_poll() will be used as a way to load-balance and
signify if any work was performed during the single iteration.
A poller could return 0, but still be registered.

This helps especially in fio_plugin that only checked active_pollers or
messages via spdk_thread_poll().

Change-Id: Id6237278eb3b4bd4922b2abaa3c8ebd5e434d45d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-01 21:38:02 +00:00
Tomasz Zawadzki
7173e9bd02 thread: add spdk_thread_has_pollers()
This function add possibility to check if there are registered pollers
on particular thread.

Change-Id: I80af06a10c5c1b54fed5bb28a3aa769a52d8a206
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/446624
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-01 21:38:02 +00:00
Tomasz Zawadzki
ca87060dcc lvol: add option to change clear method for lvol store creation
Default 'unmap' option stays as it was.

'Write_zeroes' comes useful when one wants to make sure
that data presented from lvol bdevs on initial creation presents 0's.

'None' will be used for performance tests,
when whole device is preconditioned before creating lvol store.
Instead of performing preconditioning on each lvol bdev after its creation.

Change-Id: Ic5a5985e42a84f038a882bbe6f881624ae96242c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-28 20:50:27 +00:00
Changpeng Liu
30bbf3d944 nvme: move probe context as a internal data structure
Users should not access the internal probe context fields when
using the asynchronous probe API, so change spdk_nvme_probe_async()
to let it can only return the probe context pointer.

Change-Id: I0413c2d8db6cbe4539ad80919ed34dd621a9df70
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445870
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-22 18:13:39 +00:00
Changpeng Liu
bad30d5366 nvme: add the asynchronous controllers probe/poll APIs
User can create a probe context to probe and attach controllers
asynchronously, the controllers will be added to the context list
for the first step, then users can poll the context until the list
becomes empty.

Change-Id: I3a96e2d8a9724332ff15542f78f9553fdab505e2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442664
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-15 03:14:20 +00:00
paul luse
8a1acca65d bdev/raid: Add strip_size_kb rpc param for create
strip_size as an rpc param is now deprecated and can be removed
in a future release.  Either strip_size or strip_size_kb can be
used but only one of them or the rpc will fail.

Internally we maintain both fields because strip size always
comes in as KB but we convert it to blocks so having both elements
makes it clear for developers what they're looking at.

JSON output includes both strip_size and strip_size_kb.

Fixes #550

Change-Id: I5dc51e8af22eae3d56af8f8d37a564dbaae228fa
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-11 22:46:33 +00:00
Tomasz Zawadzki
1982d4b1ad version: 19.04 pre
Change-Id: I8a89febac0af1e2faa98f1e299ce03edef9f8e59
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442572
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-02-01 09:29:12 +00:00
Tomasz Zawadzki
e3e35a85ea SPDK 19.01
Change-Id: I819755deb35b3deca836e3cb22882de7bfc8b6a0
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442571
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-01 08:45:56 +00:00
Jim Harris
eca42c6609 CHANGELOG: add note on vhost vulnerability
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id47256ecfc5d774e7d8054423cda32a90f0c4f76

Reviewed-on: https://review.gerrithub.io/c/442929
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-02-01 08:42:28 +00:00
Tomasz Zawadzki
4a6f45520c changelog: added missing items for 19.01
Change-Id: I1168ac5317ef1142f14e19156c4a8df0dbf9fb00
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442570
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-30 21:42:13 +00:00
Darek Stojaczyk
63c64676bf CHANGELOG: add trace_record
Change-Id: I805259dca4e5242307a43ce5a9a8bde79d8769ba
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442729
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
d555d6c2d0 configure: remove deprecated RAID options
Change-Id: Ibbb53fefad5fa117c9b630301fb50bc04c87c3ea
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442728
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
b52c6ea9d0 CHANGELOG: add deprecation-related notices
Change-Id: Ic5bc07145bc4c9723a41c2cc02d4f33d2efab42b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
1d30b77ef7 CHANGELOG: all hugepages are now reserved dynamically
Change-Id: I8d6c3b4c7b1ff1e82782db05d5c14cddc06d1a6a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
2c8c80f336 CHANGELOG: drop DPDK 16.07 support
Change-Id: I39910e37fc8a23ca57dcd393f2c3c21e89cb31b4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442725
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-30 17:58:10 +00:00
Darek Stojaczyk
a2e522a3c5 CHANGELOG: add DPDK 18.11
Change-Id: I3f7b2025f6ddb11dac5e23cc9c2b4df5c4cea860
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442631
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-01-30 08:37:00 +00:00
Darek Stojaczyk
a0999fd283 CHANGELOG: add multiple connections per vhost device
Change-Id: I13065d0e12bf96f9192cf9e8aa896c7f6dd51e2a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442630
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-01-30 08:37:00 +00:00
Darek Stojaczyk
c14cd59f1b CHANGELOG: add env changes
Change-Id: I3ea1719439a84b60e3a2a8c56a7f5fbebe6be861
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-01-30 08:37:00 +00:00
Tomasz Zawadzki
0c0d9d4e0a doc/lvol: add description for clear-method when creating lvol bdevs
Change-Id: I93647d2633f88522e3c0826eb7a379b9cfa0126d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442563
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-29 20:28:52 +00:00
Ziye Yang
552e21cce6 spdk: Add ISA-L support with related crc32 function
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.

Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.

Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
2019-01-29 08:31:00 +00:00
Konrad Sztyber
6f62f0a187 ftl: usage documentation
Added FTL module user guide including setting up QEMU for Open Channel
drive emulation.

Change-Id: Ic9b46552ba99a2180a9f72e8c08e876edcecd39d
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441736
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-25 19:59:05 +00:00
Tomasz Zawadzki
3bb815ae9f lvol: exposed marking lvol bdev as read only
Added set_read_only_lvol_bdev() RPC that marks
lvol bdev as read only.
This enables to create clones off of a base lvol,
without having to create additional lvol bdev with snapshot.

Change-Id: Ic20bbcd8fbebcef157acce44bfa1da035b0da459
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 19:02:36 +00:00
Shuhei Matsumoto
f75128a943 CHANGELOG: DIF and DIX APIs were added to utilility
Change-Id: Ia98c3a364ee28e43c69cf2c890bc7cac990dd3f5
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/440606
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-18 14:01:09 +00:00
Chunyang Hui
01e5610da8 parse: add return value to app_parse callback
Need to check user input and return status of parsing
to prevent app or target from crashing.
Input checking function will be added in the future.

Change-Id: I8167ac13306ae4f81e2cacb80edd9dcf9382c374
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-18 10:08:09 +00:00
Xiaodong Liu
81551144be env: add spdk_env_dpdk_post_init
Based on requirement of initializing SPDK env
from a DPDK application, relative to spdk_env_init,
add spdk_env_dpdk_post_init for calling after rte_env_init.
More details, visit
https://github.com/spdk/spdk/issues/529

Change-Id: I6fda1593e0296ef93b705e31cc76bcd0d248673a
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-08 13:28:31 +00:00
Chunyang Hui
19feb4e181 nvme: add security receive and security send wrapper
Change-Id: Id25040d62f89d4e8f2268bb3383c5665c0508f5a
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-07 05:51:07 +00:00
Ziye Yang
58f1624497 nvmf: add the transport shared buffer num configuration option.
Previously, we allocate the buffer size according
to the MaxQueueDepth info, however this is not exactly
a good way for customers to configure, we should provided
a shared buffer number configuration for the transport.

Change-Id: Ic6ff83076a65e77ec7376688ffb3737fd899057c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 19:55:57 +00:00
Seth Howell
088379cf74 nvme_rdma: Add source addr info to RPC
Change-Id: Id6b5aee4b36d828a0f9e5ddc85293a15342d2aae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436220
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
675c5592e7 nvme_bdev: add parsing for hostaddr and hostsvcid
This allows us to specify the host-side configuration for each
controller to which we connect.

Change-Id: Iac2aed3934d4a326f45546f2f541e374308e2589
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436219
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Shuhei Matsumoto
e303567bc1 util/crc16: Add init_crc parameter as seed value to spdk_crc16_t10dif
Add init_crc parameter as seed value to spdk_crc16_t10dif API to generate
a CRC value spanning multiple separate buffers.

This will be necessary for upcoming DIF/DIX patches.

Having init_crc parameter is general, and so change the existing API
without adding seed version of the existing API.

Change-Id: I0ac7919b18013967e41829dcedd3e4e73204d5d6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-18 17:28:56 +00:00
Liu Xiaodong
ae0aae1532 lib/trace: add 3 RPC method for tpoint_group_mask
Now trace facility can be enabled/disabled at runtime,
besides app starting parameters

Change-Id: I086564665ba4375b946a5339d5921d6584e9bfda
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/435470
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-14 21:29:12 +00:00
Lance Hartmann
e865a52415 nvme: Eliminate identify errors to Discovery ctrlr
The nvme/identify cmd issued some cmds to a ctrlr irrespective
of its type, and when the target was a Discovery ctrlr which only
accepts a very limited cmd set, that would result in errors observable
both on the initiator side (from nvme/identify) and in the output on
the target (nvmf_tgt).  Introduce new API, spdk_nvme_ctrlr_is_discovery(),
and alter identify to make use of that in determining which commands
to send to the target.

Change-Id: I974a569843f1d2b9e1ece7bd3bf9ceee1bfae872
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/436225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-11 17:39:52 +00:00
Ziye Yang
be4fbb2141 nvme_tcp: Make the header and data digest configurable.
Change-Id: Ia65e235a85207c128ba274e1bab38d6c35344239
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 23:24:12 +00:00
Changpeng Liu
31e8fb4025 util/uuid: add a new uuid copy API.
Change-Id: I03042c4a7030eaac406e3c3afe6fe2f69bd9db36
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/436301
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: Jim Harris <james.r.harris@intel.com>
2018-12-06 22:25:09 +00:00
Jim Harris
870b0e5b29 log: remove "trace" from log flag RPCs
We will keep the RPCs for now but mark them deprecated.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0407dcb392ea0c9e89c0f26cd5670aed2dbfadef

Reviewed-on: https://review.gerrithub.io/435345
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-03 19:50:15 +00:00
Andrey Kuzmin
af034b6de4 bdev: unregister bdevs top-down during shutdown.
There are some use cases such as multipath and RAID expansion where a
vbdev could have been registered before one of its base bdevs.

Currently we unregister bdevs at shutdown in reverse order of their
registration.  Continue to do that in general, but skip any bdev that
is still claimed.  Any bdevs skipped in this way will eventually be
unregistered once any bdevs that have claimed it have completed
unregistration.

Change-Id: Iafde9558430bc5ce56e8608ef50bcb2b5fbfbf71
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/432136
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 22:49:23 +00:00
Ziye Yang
e956be96eb nvme: Add the NVMe over fabrics TCP/IP transport support
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:

Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).

Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 20:36:05 +00:00
Changpeng Liu
2706cd4238 nvme: add timeout for Admin commands when initialization
Currently there are no timeout mechanism for Admin commands
when initialization, the NVMe driver may enter infinite loop.
While here, add a new parameter to the controller initialization
options, NVMe controller will report an error when timeout
happens during initialization.

Change-Id: Id0c6b6fa15abe5227b486bee95c8e02914b0d358
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-16 15:29:33 +00:00
Seth Howell
962ba4e89a nvmf: remove tgt_opts from nvmf_tgt
This option is deprecated. Also, rename the rpc and configuration
options for setting the opts to reflect that they now only set the max
number of subsystems

Change-Id: Iaabcbf33dd0a0dc489d81233fda74e9e7f3e0d2e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-08 23:08:26 +00:00
Tomasz Zawadzki
ae45b7ec38 version: 19.01 pre
Change-Id: I00cf8919c25e67f9771327b1802b64f24ef2cb82
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431164
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-31 21:45:09 +00:00
Tomasz Zawadzki
f14c7b3b06 SPDK 18.10
Change-Id: Idf71750aac404884384ec749b3136d2f341f76e7
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431163
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-31 16:55:17 +00:00
Tomasz Zawadzki
338b038f5b CHANGELOG: bdev crypto is not considered experimental for this release
Change-Id: I9c01bfc05417f07b9265692a02b6aaba37c507f2
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431529
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-31 16:48:34 +00:00
Darek Stojaczyk
81c0e62feb CHANGELOG: mention I/OAT improved error handling
Change-Id: I62ba6697599c99791104e9fb46a5b275a47174ea
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-30 20:26:21 +00:00
Darek Stojaczyk
dba21f69d9 CHANGELOG: mention long name cmdline parameters
Change-Id: Iaf4323190c5f4e0e19fb1311d35bc2347a5b70da
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
450717384c CHANGELOG: iSCSI and NVMe-oF support for SPDKCLI
Change-Id: I622cb78656d2da6b9c410c3acd862d4870289ee0
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431160
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
c32c2a4212 CHANGELOG: add info on blobstore io_units
Change-Id: Ief376d26ec33507e458693487b892e00bab1682e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431159
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
6c01916cab CHANGELOG: add info on send_nvme_cmd RPC
Change-Id: Iad65dc8b3863601bb8b061a0f903d4ab39201180
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431162
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
e3b55a6efa CHANGELOG: add info on new jsonrpc-client library
Change-Id: Ieb699a3d59711cb0ab6b8683decdcea290a6fb80
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431203
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
73372bad33 CHANGELOG: added info on new API for iteration over JSON object
Change-Id: Ia82a6a5315e430e22cd0cec471ff45a5f5458eec
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431201
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
b57a439171 CHANGELOG: add info on spdk_file_get_id()
Change-Id: Ic5e4c5560481f93db349696373d295737e1bf2dc
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431200
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Tomasz Zawadzki
abd726ca5a CHANGELOG: info on get_bdevs_config removal
Change-Id: Iab4b2e0a3f5d646f0bde228ef966950a8a450017
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/431199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-30 18:08:16 +00:00
Darek Stojaczyk
3e2297140c doc: describe dynamic memory management
Change-Id: I0ffa5a8ea0ffb46113c6a000546a445784ce7b2f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431094
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-29 23:43:45 +00:00
Jim Harris
b391eadc94 doc: adjust CHANGELOG entry for iSCSI initiator
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iddf357ad9a2a5d078591312ee545264d5775756b

Reviewed-on: https://review.gerrithub.io/429974
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-19 14:28:38 +00:00
Darek Stojaczyk
b237c1b5c3 bdev/virtio: deprecate remove_virtio_scsi_bdev RPC
`remove_virtio_bdev` RPC can be used instead. It deletes
both Virtio Block and SCSI devices.

Change-Id: I3504f8d27eb13bb7522c05041c21042153a4fa18
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-18 22:02:48 +00:00
Darek Stojaczyk
f3096dace8 bdev/virtio: deprecate old bdev construct RPCs
The 4 different RPC commands to create Virtio bdevs
have been unified to a single, yet another RPC command
in hopes the previous ones will be eventually removed.
That happened in 18.07 and now in order to remove the
old ones we mark them deprecated.

The unified RPC command was introduced in 18.07 and
has already replaced the old, separate RPCs in our
test scripts some time ago.

This patch also removes the old RPCs from doc/bdev.md
and while here, rephrases the Virtio section a bit to
make it more readable after some alphabetization
changes from the past.

jsonrpc.md already described the old RPCs as about to
be deprecated, so there are no changes required in
there.

Change-Id: I77ebf2dde56b1f4a6e4090ed76de16a97310c1c5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-18 22:02:48 +00:00
Darek Stojaczyk
c6eb4da6c0 doc/bdev: document iSCSI bdev module
Most to-do items in lib/bdev/iscsi/README have been
done a long time ago, leaving this file greatly outdated.
It is removed now and the iSCSI bdev description is
added to doc/bdev.md.

Since most of the to-do items have been done, this
patch also adds a CHANGELOG entry announcing that
iSCSI bdev is no longer in experimental state.

Change-Id: I186c9e1489547152655e1e036b27ab6bc17fcf11
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429896
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-18 22:02:48 +00:00
Ziye Yang
5c263046db sock: Update the spdk_sock_getaddr function.
Purpose: We need to get the port info in other applications
(e.g., NVMe-oF TCP/IP transport)

Change-Id: I3a4636e764e44425436bb064cb0062c6f3e44035
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/428313
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-11 04:58:49 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
Chunyang Hui
99ca58e082 NVMe: Add support for security send/receive command
Change-Id: I61f6a8fac4938d6af3ac6c84bcf11f99036ed856
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/426657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-09-29 03:13:55 +00:00
Seth Howell
4e06bb5e6d env: pass an spdk_mem_map_ops structure to mem_map_alloc
This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.

Change-Id: I90da6d4d31c669a3bf046f7721923dd743c5ef21
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-26 20:57:57 +00:00
Jim Harris
1d64773c2a raid: enable by default
We kept this disabled by default originally because it lacked
multiple iov support needed for vhost and nvmf in-capsule data.
Now that support has been added, there is no reason to not
enable it automatically - it has no extra package dependencies,
so requiring users to specify --with-raid to configure is just
an extra unneeded step.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia9796370e12a6f93993e70083de36114a37851bd

Reviewed-on: https://review.gerrithub.io/426003
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-19 17:22:31 +00:00
Pawel Kaminski
ca12cb3097 scripts: add config converter
Add a new script to enable converting
old INI config files to the new JSON-RPC
config file format.
This prepares for deprecating the INI
config file file format in the future.

Change-Id: I0f2bfa9a585ce3537772b662d8e4028ab08a554d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/420457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-09-18 15:24:40 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Seth Howell
d288c41242 env_dpdk: change behavior of spdk_mem_map_translate
The function now takes a pointer as it's last argument, and copies the
size of the memory region for which the translation is validinto that
pointer.
For now, that will always be 2MB. However that behavior can change in
the future.

This series of changes is aimed at enabling spdk_mem_map_translate to
report back to the user the length of the valid mem_map up to the
function that requested the translation.
This will be useful when retrieving memory regions associated with I/O
buffers in NVMe-oF. For large I/O it will be possible that the buffer is
split over multiple MRs and the I/O will have to be split into multiple
SGLs.

Change-Id: I8686c166ec956507f5ae55cf602341281482cb89
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424888
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-15 00:28:23 +00:00
Pawel Wodkowski
b96f97cf29 scripts: use python3 in all scripts
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.

Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.

Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:24:30 +00:00
Jim Harris
ff6299f831 build: remove DPDK_DIR
This is an artifact from before SPDK had a configure
script or a DPDK submodule.  Make configure the
only supported way for specifying the location of the
DPDK installation to use with SPDK.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c197c46220928bb18b97c8807755967d76ea42c

Reviewed-on: https://review.gerrithub.io/424893
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-07 16:01:28 +00:00
Jim Harris
b6ee6f9241 iscsi, nvmf, vhost: remove default config file pathnames
Config file pathnames have been possible on the command line
for a very long time, and all of the SPDK testing infrastructure
uses it.  The default pathnames were added a very long time ago,
and now can cause weird issues like conflicting with --wait-for-rpc
when an INI config file is not passed on the command line but is
pulled in by the default pathname instead.

So remove the default pathnames.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia914ff1c9321235f79f8cc9a054ed7f099340151

Reviewed-on: https://review.gerrithub.io/424606
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Jim Harris
87d2cd9b69 app: deprecate ReactorMask from ini config file
This is not used anywhere, and is not something we can
set over RPC.  So make this a command-line only option
(as it should be).

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2df862cdce2f17992d2324312d5c0e98b38a8acd
Reviewed-on: https://review.gerrithub.io/423930
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-30 03:41:01 +00:00
Shuhei Matsumoto
ad323b8064 iscsi&scrpts/rpc: Require to specify CHAP secret file explicitly to load it
Previous patches enabled users to configure CHAP secrets dynamically
by RPCs. Subsequent patches will enable users to load CHAP secrets
from JSON config file.

Loading CHAP secret file is done by default and this will conflict to
JSON config file.

Hence the path to CHAP secret file is required to specify in the config
file or JSON RPC set_iscsi_options explicitly after this patch.

Users who have used CHAP secret file are expected to specify it explicitly
and this will be no harm for them.

Besides, CHAP secret file is not oly for discovery sessions but also for
login to iSCSI targets. However there were wrong description to make user
misunderstand. Hence remove these wrong description in this patch too.

Change-Id: Ic4093cabc0c14b87e26baef4bba6b0d292e40c06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-29 21:40:31 +00:00
Shuhei Matsumoto
a62d80f98f CHANGELOG: Load CHAP secrets from file once only at boot and then configure them by RPCs
This feature change is done by a few patches and it is difficult
to do CHANGELOG in a patch of them. Hence use this dedicated patch.

Change-Id: I4b3957640828f8c4652a2cc0db84c37fc2a04063
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423051
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 21:13:49 +00:00
Chen Wang
1f813ec3da doc: fix typos in the doc directory
Change-Id: Ifff553ed70ce5aa8e7bdf6d8a8e9e9afb73e8a64
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/423497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 19:38:10 +00:00
Shuhei Matsumoto
844735c928 iscsi: Add set_iscsi_target_node_auth RPC to configure CHAP dynamically
An new RPC set_iscsi_target_node_auth enables users to configure CHAP
authentication for the target node dynamically.

New setting is used for new iSCSI sessions. For existing iSCSI sessions,
new setting is not used until user logout and login again.

Try to use descriptive message in the RPC when error occurs.

Change-Id: I0bd40d92262d708c1f7de0effb208078bdf8cc41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420974
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-22 22:39:26 +00:00
Shuhei Matsumoto
6cd1714a99 iscsi: Add set_iscsi_discovery_auth RPC to configure CHAP dynamically
An new RPC set_iscsi_discovery_auth enables users to configure
CHAP authentication for discovery sessions dynamically.
Try to use descriptive message in the RPC when error occurs.

Change-Id: I8883d5e054539cf6db091a5b79d0479e62119811
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420976
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-22 22:39:26 +00:00
Shuhei Matsumoto
66723f7f92 rpc/iscsi: Use chap to parameter names for CHAP in discovery session
Current parameter names of `set_iscsi_options` and `get_iscsi_global_params`
RPC method for CHAP in discovery session may not be clear and long
compared with CHAP in login to target nodes.

This patch changes parameter names of `set_iscsi_options` and
`get_iscsi_global_params` RPC method for CHAP in discovery session
from no_discovery_auth, req_discovery_auth, req_discovery_auth_mutual,
and discovery_auth_group to disable_chap, require_chap, mutual_chap,
and chap_group, respectively.

Old parameters are still usable because decoder of them are not removed.

Change-Id: If3102f1233d57ee3cadfb733a6fc1fca14a0e972
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/423050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-22 22:39:26 +00:00
wuzhouhui
6deac3e660 bdev/lvol: using spdk_bdev_alias_del_all() to delete all alias on destroy
So we don't need to allocate memory (maybe failed) just for free other
memory.

Change-Id: I2c83f6acc2aa6ed79455bff90f952a2e70b44d59
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/422203
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-21 00:53:31 +00:00
Lance Hartmann
0923734c9f build: Conditionally builds single and separate shared libs
Adds option to configure script (default is disabled)
to build SPDK shared libraries, and enhances build
correspondingly to do so.  This includes the single, combined
shared library encompassing all of the SPDK static libs,
as well as production of individual SPDK shared libraries
corresponding to each of the static ones.  Although the
SPDK shared libs conform with shared lib naming conventions
complete with version numbering suffixes, this does not
confer any ABI compatibility claims whatsoever at this time.
It's merely conformance with the naming conventions.
Finally, there are no built-in dependencies between SPDK
shared libs, thus requiring one to specify all needed SPDK
shared libs to resolve all symbols in the final link of an
executable wishing to use said SPDK shared libs.

Change-Id: Ia375c0e3fb6d150d45a39e84e3b226fbaea48545
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/422307
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-17 14:23:31 +00:00
Jim Harris
5b900148e5 nvme: add spdk_nvme_ns_get_extended_sector_size
spdk_nvme_ns_get_sector_size returns the sector
size of the data only.  This new function adds
in the metadata size, if any.

While here, modify the bdev/nvme driver to use this
function - this is needed for ongoing extended sector
size work through the bdev layer.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic8070a7f8d29b0b2ac2a2d65e0df5f4736488351

Reviewed-on: https://review.gerrithub.io/422445
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-16 04:38:10 +00:00
Ben Walker
5a27b6b442 version: 18.10 pre
Change-Id: I2d86f5aa440d971e4852d8b175abca830f42867b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421165
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-03 00:42:36 +00:00
Ben Walker
716ad0e389 SPDK 18.07
Change-Id: I9b9c8f0c213dcdd4d233171daf41e96033324af6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/421164
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-02 21:03:33 +00:00
Jim Harris
cf90380720 CHANGELOG.md: remove duplicated BlobFS entry
Reported-by: Shuhei Matsumoto
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I253ad84d6705550585dcd1da48560300b0eb8377

Reviewed-on: https://review.gerrithub.io/420939
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-01 16:33:44 +00:00
Ben Walker
812a38d497 Reorganize 18.07 Changelog entries.
Fill in some missing things based on the diff

Change-Id: I5dadd6ac31dac7e9cf483c514080f88b1429a405
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/420922
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-07-31 23:05:14 +00:00
Dariusz Stojaczyk
120c4f3c05 CHANGELOG: add DPDK 18.05 note
Change-Id: I17e1663ee2a231a41ca3747afe83a0a57054b8e7
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/420728
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 16:47:45 +00:00
Ziye Yang
703d1f80a8 blobfs: change the return type of spdk_file_truncate
Change-Id: I29347d7b7be0237d7d604c7cd7363fa08a0cd025
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/420133
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-26 01:42:17 +00:00
Ziye Yang
b1da5cecaa nvme: change the type of paramter for timeout_callback set
Change the type of nvme_timeout parameter in
spdk_nvme_ctrlr_register_timeout_callback from uin32_t to uint64_t.

Reason: This will make the timeout trigger test more flexible and
will not affect the original meanings.

Also for the configuration file, still maintain the compatability
support

Change-Id: I94c90f67b2e9c57220ab82ecea11a1590d62aed4
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/419326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-19 07:18:17 +00:00
Daniel Verkamp
6118b85333 build: add intel-ipsec-mb submodule
This is a dependency of the DPDK crypto framework.

The submodule is currently set to our SPDK fork of the ipsec library
which is based off of the library v0.48 tag, as required by DPDK plus
one minor Makefile change on the 'spdk' branch of the fork.

Also added is a configure option '--with-crypto' and the associated
conditionals for building DPDK with or without the library dependency.

The crypto vbdev patch will use this same configure option to determine
DPDK opts as well as whether the crypto vbdev should be built or not.

Change-Id: Ib1cfd15b63b29acf3c2f7345d0d7edfd94716620
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/404983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-07-19 00:06:55 +00:00
Kunal Sablok
41586b0f1d bdev: add raid bdev module
Raid module:
============
- SPDK raid bdev module is a new bdev module which is
  responsible for striping various NVMe devices and expose the raid bdev
  to bdev layer which would enhance the performance and capacity.
- It can support theoretically 256 base devices (currently it is being
  tested max upto 8 base devices)
- Multiple strip sizes like 32KB, 64KB, 128KB, 256KB, 512KB etc is
  supported. Most of the current testing is focused on 64KB strip size.
- New RPC commands like "create raid bdev", "destroy raid bdev" and "get raid bdevs"
  are introduced to configure raid bdev dynamically in a running
SPDK system.
- Currently raid bdev configuration parameters are persisted in the
  current SPDK configuration file for across reboot support. DDF will be
introduced later.

High level testing done:
=======================
- Raid bdev is created with 8 base NVMe devices via configuration
  file and is exposed to initiator via existing methods. Initiator is
able to see a single NVMe namespace with capacity equal to sum of the
minimum capacities of 8 devices. Initiator was able to run raw
read/write workload, file system workload etc (tested with XFS file
system workload).
- Multiple raid bdevs are also created and exposed to initiator and
  tested with file system and other workloads for read/write IO.
- LVS / LVOL are created over raid bdev and exposed to initiator.
  Testing was done for raw read/write workloads and XFS file system
workloads.
- RPC testing is done where on the running SPDK system raid bdevs
  are created out of NVMe base devices. These raid bdevs (and LVOLs
over raid bdevs) are then exposed to initiator and IO workload was
tested for raw read/write and XFS file system workload.
- RPC testing is done for delete raid bdevs where all raid bdevs
  are deleted in running SPDK system.
- RPC testing is done for get raid bdevs where existing list of
  raid bdev names is printed (it can be all raid bdevs or only
online or only configuring or only offline).
- RPC testing is done where raid bdevs and underlying NVMe devices
  relationship was returned in JSON RPC commands

Change-Id: I10ae1266f8f2cca3c106e4df8c1c0993ddf435d8
Signed-off-by: Kunal Sablok <kunal.sablok@intel.com>
Reviewed-on: https://review.gerrithub.io/410484
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-16 20:50:40 +00:00
Seth Howell
6b7609b639 bdev.c: encapsulate members of spdk_bdev_part.
Change-Id: Ia3d04905d319c075a12726ca85c3aa1d3135db44
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/417176
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-05 19:12:53 +00:00
Daniel Verkamp
6f8ab39534 changelog: add note about -t to -L option rename
This was changed in commit 8f26b74e02 ("examples: make debug flag
option consistent").

Change-Id: Id47eff11321063b684fdd4e9d2187e459065e98e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/416890
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-28 16:28:33 +00:00
Changpeng Liu
c391fed3e1 nvme: add nvme error injection support
Users can set specified Admin commands or IO commands with
error status, when submitting new commands which are already
set with error status, the commands will return to the caller
with specified error code. So that users can emulate some error
status for their error condition code path.

Change-Id: I4b93c7e4f2b15a659da73b39e26bfa162eb5214e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/410870
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-22 17:26:34 +00:00
Daniel Verkamp
6f46e272c3 build: add combined shared library (libspdk.so)
Change-Id: If48df087c145a3fd278c0ae04558d8352b01cc1a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/414687
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-14 03:36:03 +00:00
Shuhei Matsumoto
239e63be7d copy/ioat: Change the default of IOAT from Enabled to Disabled
This patch changes the default setting of IOAT copy engine from
Enabled to Disabled. Accordingly this patch changes the config file
specification for IOAT copy engine from "Disable Yes/No" to
"Enable No/Yes".

Subsequent patches will add a new JSON RPC to configure IOAT
copy engine dynamically.

Change-Id: I754990cbb6ecc096953dd2fb9d34366b91111bf8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/410757
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-12 17:41:29 +00:00
Ben Walker
c1dc8cd584 thread: Rename io_channel.h to thread.h
Leave io_channel.h in the repository and make it simply
include thread.h for backward compatibility.

Change-Id: I5320fb8e9e5bc6e39ee9ce44cd79570194a8b841
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414699
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-12 15:24:07 +00:00
Ben Walker
8c9508f9b3 nvmf: Correctly unregister nvmf tgt as an I/O device
Change-Id: I9552354d620986c2a3d6b071f32b587e9e683198
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/413861
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-06-07 16:07:38 +00:00
Daniel Verkamp
5eb33f0a5f nbd: return path to /dev/nbd* in start_nbd_disk
This will allow us to make the start_nbd_disk method choose an available
/dev/nbd* node automatically in an upcoming commit.

Change-Id: I3c6eb67e7e0c513212eba17139fd0cc96bf9b6dd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413847
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-07 16:05:59 +00:00
zkhatami88
0cdb08b0e0 env: add size parameter to spdk_mem_map_translate
Change-Id: I808101edaf4d75613baf19a950915f1d8e75b1af
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/413154
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Zahra Khatami <zahra.k.khatami@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-05 18:36:00 +00:00
Daniel Verkamp
2ee95af0fd bdev: stop zeroing data in spdk_bdev_get_io_stat()
This will allow multiple independent callers to request I/O statistics
without stepping on each other (previously the stats would be reset on
each request, which only works with a single caller).

This also means that we can now allow requesting stats while the VTune
integration is enabled.

Change-Id: Ia9d4d6fd37fa66b3671cd33b3183c90524f955bd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/412257
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-24 22:05:48 +00:00
John Meneghini
22898a91b9 git: add .githooks to repository
- Add git pre-commit and pre-push hooks
 - To enable type 'git config core.hooksPath .githooks'
 - For for additional example hooks see the .git/hooks directory

Change-Id: I92155f4083c8547759bfbbfe4df64923548fa4d5
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/409577
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-04 22:17:35 +00:00
Daniel Verkamp
080a301d8f version: SPDK 18.07-pre
Change-Id: I712fb4fce02821bc7ae483d06aa905f54b4998aa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/409361
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-30 16:44:19 +00:00
Daniel Verkamp
145b2f7bd3 changelog: add bdev QoS and iSCSI VPP support
Also tweak some wording and add other minor feature additions.

Change-Id: I37f4c97c260e55c62b112b72fb47a316772da3f7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/409223
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-26 23:56:38 +00:00
Maciej Szwed
ee3213da3e doc: changelog update
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ia2d0c596725c5df239f0e01ac9f10e5b65b02764
Reviewed-on: https://review.gerrithub.io/408825
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-25 17:25:49 +00:00
Dariusz Stojaczyk
addd386d81 doc/virtio: mention 2MB hugepage support
Also updated the CHANGELOG.

Change-Id: I9040505f689aab51fc26ab47d5a10438e35f9fc4
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/408830
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-25 17:25:49 +00:00
Dariusz Stojaczyk
30f369f703 doc/virtio: add Virtio Block doc
I decided to use 'Virtio Block' as the title. Virtio
spec uses the same name alongside SCSI.

```
5 Device Types
[...]
 5.2 Block Device
 5.6 SCSI Host Device
```

Also added a CHANGELOG note.

Change-Id: Ie2eabd053213da776c78a8d938a621b42e6d5558
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/408829
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-25 17:25:49 +00:00
zkhatami88
a77cd3f75c env: add malloc variants with DMA/shareable flags
This will allow environment abstraction layers to provide different
types of memory depending on whether it needs to be DMA-able and/or
shared across multi-process boundaries.  For the DPDK environment, the
flags can be ignored, since rte_malloc() supports both DMA-able and
shared memory.

Change-Id: I5ee894337dd9d6e24418848c0a35f131184383c8
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/402334
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Tested-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-24 21:38:16 +00:00
Daniel Verkamp
f9d52c9ebd lvol: add destroy_lvol_bdev RPC method
This is the first step toward fixing the behavior of the lvol bdev
module so that it handles spdk_bdev_unregister() correctly.

Currently, when an lvol bdev is unregistered, this causes the associated
lvol to be deleted; this isn't the desired behavior, since
spdk_bdev_unregister() is just meant to drop the currently-exposed bdev,
as in a hot-unplug event, not destroy the backing storage.

The current implementation of the new destroy_lvol_bdev method still
calls into the exsting (broken) spdk_bdev_unregister() path, but this at
least lets us fix the existing delete_bdev RPC callers that really
intended to destroy the lvol.

The next step is to make vbdev_lvol_destroy() call spdk_lvol_destroy()
instead of unregistering the bdev, which will require cleanup in the
generic lvol layer to remove the close_only flag.

Change-Id: Ic4cb51305d31f6d7366a82ba7bf7b19ebbcd3e9c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408084
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-24 21:33:56 +00:00
Jim Harris
60622c61f9 CHANGELOG.md: document DPDK CVE-2018-1059 fixes
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I748be6adeb83cfa99f4359db68acdec7ea5ab523

Reviewed-on: https://review.gerrithub.io/408736
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-23 20:30:20 +00:00
Paul Luse
2536874e85 env: Add SPDK wrapper for rte_mempool_get_bulk()
Needed for crypto vbdev work.

Change-Id: Ib18ce3aaecf2388cf2cdc4dea110db514c8c1f1b
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/408256
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-20 16:41:28 -04:00
Seth Howell
4ca87a01b4 nvmf: make spdk_nvmf_subsystem_remove_ns asynchronous
Update the thread-local caches with new namespace data during each call
to spdk_nvmf_subsystem_remove_ns to handle the case where the user
requested to remove a namespace and then immediately add a different
one at the same namespace id. This makes the call asynchronous.

Change-Id: I8fd1968f7da78966386de18506b98d403b82d80e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/408220
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-18 15:41:00 -04:00
Daniel Verkamp
0a162815d6 Revert "subsystem.c: make subsystem_remove_ns asynchronous"
This reverts commit 498f9add11.

Making the subsystem removal asynchronous seems to be triggering an
intermittent failure in the NVMe AER test.  Let's revert this for now
until we can diagnose the issue.

Change-Id: Ie1d598f0d5cce07e6869d87cd8388848caa78e46
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408118
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-17 20:17:27 -04:00
Seth Howell
498f9add11 subsystem.c: make subsystem_remove_ns asynchronous
Performing a subsystem update on each poll group after the addition or
removal of a namespace allows us to avoid the case where we pause a
subsystem, perform a removal, then an addition, resume the subsystem and
don't update the channel for that subsystem. This, however requires that
spdk_nvmf_subsystem_remove_ns be asynchronous.

Change-Id: I856572c02e6267f708da3a956accbbedae7260fb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/407012
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-17 18:23:19 -04:00
paul luse
a4a497d5b0 bdev: add new optional bdev i/f entry point
Add new optional bdev module interface function, init_complete, to notify bdev modules
when the bdev subsystem initialization is complete. Useful for virtual bdevs that require
notification that the set of initialization examine() calls is complete.

Change-Id: I0997fb5749d430f2fd3a40172ec8a1d5caa96964
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/407222
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-04-13 17:46:33 -04:00
Jim Harris
218dcd840a bdev/pmem: change all NVML strings to PMDK
NVML (Non Volatile Memory Library) changed its name to
PMDK (Persistent Memory Development Kit), so make the
necessary changes to the SPDK repository.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id07a87eafb4e9a3099603030c58c0e927bafa608

Reviewed-on: https://review.gerrithub.io/406256
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Jim Harris
463925ff0f blob: make spdk_blob_resize an async operation
To support resize operations during I/O, we will need
to send messages to each thread to quiesce I/O while
the resize operation is in progress to guard against
the cluster map memory changing while another thread
is accessing the cluster map.

Therefore, spdk_blob_resize needs to be asynchronous.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ida037334739b4b80a1dbc76e8f1c70bca8b73582

Reviewed-on: https://review.gerrithub.io/404616
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-05 18:43:11 -04:00
Daniel Verkamp
fccb94622a changelog: tweak wording of NVMe active NS entry
This allows Doxygen to recognize and autolink the
spdk_nvme_ctrlr_get_next_active_ns() reference.

Change-Id: I74ec95b1b7120c5359b4543260d7c5fe2ae88fba
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/406224
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-03 18:09:12 -04:00
Ed Rodriguez
06fbf4b34b nvme: Use active namespace list for enumerating namespaces
- Add support for multi page CNS 0x2
 - Use CNS value 0x02 (SPDK_NVME_IDENTIFY_ACTIVE_NS_LIST)
   to query active namespaces
 - Add an API to iterate the active list

Change-Id: Iea524881fa6e3610a7d85ab02a2005a92fd633df
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/401957
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-02 17:17:40 -04:00
Daniel Verkamp
258845450d env: deprecate spdk_pci_get_device()
This isn't possible to implement using the current public API of DPDK,
and all of the in-tree users have been removed.  Replace the
implementation with a stub that always returns NULL and mark it
deprecated so that any users have a release to update their code.

Change-Id: I4bc71f0a9fd518923484e862333b0c5e86883980
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405710
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-29 01:46:43 -04:00
Daniel Verkamp
15bd6999db nvme: add spdk_nvme_ns_get_uuid() function
This function retrieves the UUID from a namespace, if available.

Change-Id: I98c55375948b92eaf429b41fb36dfea4e2b780a2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/404734
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-27 02:43:46 -04:00
Jim Harris
2af15344ab bdev: add iSCSI initiator bdev module
This uses libiscsi to implement an iSCSI initiator bdev
module for SPDK.  Still a lots of work to do on this - posting
it in case anyone is interested in working on this further.

A number of todo items are listed in a README in the lib/bdev/iscsi
directory.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Change-Id: I060e33de0cd6796246789bf0e1bb4f2df59d8f71
Reviewed-on: https://review.gerrithub.io/390313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-21 20:35:42 -04:00
Roman Sudarikov
c3bc40a6ef io_channel: add return value to pollers
This will be used to track time used in pollers - each poller can now
indicate if it found any work to do or not.

For cases where it was obvious and the infrastructure was already in
place, existing pollers have been modified to return 0 or a positive
value to indicate whether work was done.  Other pollers have been
modified to return -1 by default, indicating that the poller isn't
indicating anything about whether work was performed.  This will allow
us to find un-annotated pollers easily in the future and fix them
incrementally.

Change-Id: Ifebfa56604a38434fac5c76ba7263267574ff199
Signed-off-by: Roman Sudarikov <roman.sudarikov@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/391042
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-03-13 11:16:20 -04:00
Lance Hartmann
b9be940ae9 lib: Return instead of exit in event
Modifies behavior of spdk_app_start() and spdk_app_parse_args()
such that they return on failure instead of terminating with
exit().

Change-Id: I82566417f04e1ae2e3ca60a00c72e664db26c9e4
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/401243
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-08 11:47:01 -05:00
Jim Harris
66fc591ff7 blob: change spdk_bs_io_xxx_blob functions to spdk_blob_io_xxx
These new names are much more clear and are aligned with other
functions such as spdk_blob_close.

Keep the old names around for now but deprecate them.  We will
remove them in next release.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc60fd0b19fa2a8b0247a1f5835774d342e721f9

Reviewed-on: https://review.gerrithub.io/400884
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-02-23 18:37:17 -05:00
Daniel Verkamp
1023ca7b46 nvmf: allow configuration of ns NGUID and EUI64
Add optional parameters to namespace creation to let the user pick the
namespace globally unique identifier and EUI-64.

Change-Id: Ia3eebaf22f8a64733a00a83f90cafb4977c2d07a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/399531
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-20 12:01:47 -05:00
Daniel Verkamp
c7852cf98d jsonrpc: allow send_buf to grow as needed
Reallocate the send buffer if more data is written by the RPC handler
than currently fits in the buffer.

Change-Id: I590dd173b843aba48c768adfafaf87e4b47bcc19
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/399925
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-02-15 11:03:54 -05:00
Daniel Verkamp
3839639759 nvme: add API to allocate CMB I/O buffers
Change-Id: I2a3c7a272dc08be5a5ecb4339622816482c4cbb0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Signed-off-by: Stephen Bates <sbates@raithlin.com>
Reviewed-on: https://review.gerrithub.io/397036
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-02-12 19:11:44 -05:00
Daniel Verkamp
ff12a9b540 rpc: remove [Rpc] configuration file section
This section was deprecated in the previous release, and its removal
was planned for v18.04.

Users may configure the RPC listen address via the command-line '-r'
parameter.

Change-Id: Ic714417b87d422492b8742852c16983eb61793c2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/398239
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-02-06 11:15:39 -05:00
Daniel Verkamp
f68211cd4b changelog: add thin provisioning and live migration
Change-Id: I939a6f492f4a97994052479b515b63075ff38e0b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/397272
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-30 17:42:38 -05:00
Jim Harris
4d48d87a7f iscsi: remove idle connection handling
Some upcoming changes will effectively render this moot
anyways by adding an epoll/kqueue descriptor to poll
on in all cases (not just connections that have been
idle for 5ms).

The epoll/kqueue code was just ifdef'd out instead
of removed - some of this code will be useful
and reusable with minimal changes in the upcoming
patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0c354390537e6369cb3c32e78a59c300dec6d098

Reviewed-on: https://review.gerrithub.io/395553
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: <shuhei.matsumoto.xt@hitachi.com>
2018-01-22 23:22:17 -05:00
Dariusz Stojaczyk
d6859ab609 CHANGELOG: added Virtio library
Change-Id: Ifed877fdbcbac855c41f94d13f1cf57f637d97ee
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394518
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-01-17 18:02:33 -05:00
Dariusz Stojaczyk
21ec62533d CHANGELOG: add fio 3.3 support
Change-Id: I650d5cdb6237d1e74edbaa5d80a6c9eaeaa65a7f
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/394382
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-11 15:02:54 -05:00
Daniel Verkamp
77311f59cf changelog: make blobstore API changes a list
Also add parentheses to function names so they will get auto-linked
(once these functions have Doxygen comments).

Change-Id: Iba19bd60dc5ea9d2120234e185a7ca3e6604e177
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/393818
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-01-11 11:31:26 -05:00
Lance Hartmann
095f4254f1 lib: Return instead of exit/abort in env_dpdk
Modifies spdk_env_init() and spdk_mem_map_init() such that
they return on failure instead of terminating with exit()
or abort().

Change-Id: I054c1d9b2e46516ff53d845328ab9547f54bdbc4
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/393987
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-01-10 16:20:15 -05:00
Cunyin Chang
df6e317ed9 changelog: modify the log about hotplug of vfio-attached devices.
Change-Id: I2f78a42c2a54273b4c117cab0794901df60b0874
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/391341
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-01-08 17:48:23 -05:00
Xiaodong Liu
c3b3138117 nbd: remove nbd test app
nbd functionality now can be gotten by nbd rpc.

Change-Id: I85ebcb44d9ed263bcee19162dc060f3b268dddcd
Signed-off-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/391371
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-01-05 17:39:48 -05:00
Ben Walker
7b397f6281 nvmf: No longer tie subsystems to CPU cores
The "Core" parameter in the configuration file has been removed. New
connections are handed out to available cores using round-robin.

Change-Id: I24527fa22a0b2738ebbf5fb030e3bb373ead5da2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/388295
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-12-15 16:26:33 -05:00
Jim Harris
ae5a01dd9f blob: change spdk_bs_iter_next parameter to spdk_blob *
Similar to previous change, the ** paradigm is a bit
problematic for asynchronous routines that could fail.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife7748280482356c4c51a796817b71cd7bc7e479

Reviewed-on: https://review.gerrithub.io/391483
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-15 12:28:44 -05:00
Jim Harris
e734bb9f9f blob: change spdk_blob_close parameter to spdk_blob *
Using the ** paradigm is a bit problematic for asynchronous
routines that could fail.  Currently we were inconsistent in
that some error paths would zero the pointer while others
did not.  So make this just a plain pointer, which simplifies
the API and its implementation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I67147931c6e8350896a4505022a6a314655de3d3

Reviewed-on: https://review.gerrithub.io/391482
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-15 12:28:44 -05:00
Ben Walker
5165aee686 channel: Make spdk_for_each_channel support async operations
While iterating, allow the user to perform asynchronous
operations. To continue iteration, the user is expected
to call spdk_for_each_channel_continue.

Change-Id: Ifd7d03d5fbf17cf13843704274b036d49ca0484a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/391309
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-12-14 12:05:07 -05:00
Jim Harris
d52dbda288 blob: rename bs_md functions
As part of clarifying the API and preparing for some
future changes, rename the following functions:

spdk_bs_md_create_blob => spdk_bs_create_blob
spdk_bs_md_open_blob => spdk_bs_open_blob
spdk_bs_md_delete_blob => spdk_bs_delete_blob
spdk_bs_md_iter_first => spdk_bs_iter_first
spdk_bs_md_iter_next => spdk_bs_iter_next

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I86bf792717b68379484a6108396bb891fe1c221e

Reviewed-on: https://review.gerrithub.io/391031
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-12-13 13:06:16 -05:00
Jim Harris
2c3591f183 blob: rename spdk blob metadata functions
spdk_bs_md_resize_blob => spdk_blob_resize
spdk_bs_md_sync_blob => spdk_blob_sync_md
spdk_bs_md_get_xattr_value => spdk_blob_get_xattr_value
spdK_bs_md_get_xattr_names => spdk_blob_get_xattr_names
spdk_blob_md_set_xattr => spdk_blob_set_xattr
spdk_blob_md_remove_xattr => spdk_blob_remove_xattr
spdk_bs_md_close_blob => spdk_blob_close

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99b593c03bc61354ba6aa5a446b0fc82df1e9ce2

Reviewed-on: https://review.gerrithub.io/391006
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-12-12 18:36:18 -05:00
Cunyin Chang
f991ac6d90 Changelog: Add section about hotplug of vfio-attached device.
Change-Id: I0a6cd8a1ac4648b06ad057912cafcf97a54bb377
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Reviewed-on: https://review.gerrithub.io/390700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-12-08 14:07:40 -05:00
Ben Walker
44770c2a11 bdev: Remove poller abstraction
Use the new one from io_channel.h.

Change-Id: I7bf6729caf6eeebcb58450a36119601957ad5da4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/388290
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-11-28 15:29:35 -05:00
Ben Walker
9c35e39c54 event: Move spdk_poller_register to io_channel
Make this generic and not directly dependent on
the event framework. That way our libraries can
register pollers without adding a dependency.

Change-Id: I7ad7a7ddc131596ca1791a7b0f43dabfda050f5f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/387690
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-11-28 12:30:37 -05:00
Daniel Verkamp
2e7fe8e888 build: add 'make install' rule
For now, this only installs libraries and headers; we will need to
consider which binaries should be installed and what they should be
named before we add them to the install rule.

Change-Id: I78dc8631f793d0df88cd884b0ac66406df9e4427
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/387637
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-11-27 17:58:02 -05:00
Jim Harris
6bef902ca5 rpc: add default UNIX domain socket listen address
RPC is a default feature required for almost all usages,
so enable RPC by default, but with a UNIX domain socket
for security reasons.

-r can now be used from the command line to specify
an alternative RPC listen address from the default
/var/tmp/spdk.sock.

Remove the Enable parameter from the Rpc config section
but still allow specifying an alternative listen address
using the Listen parameter as an alternative to the
command line option.  This keeps backward compatibility
for this release for anyone using the configuration file
still.

Remove the Rpc sections from all configuration files
that were using them, except for those that specified
alternate TCP ports for multi-process test cases.  We
can fix these later to use an alternate UNIX domain
socket and to use the command line instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife0d03fcab638c67b659f1eb85348ddc2b55c4c4
Reviewed-on: https://review.gerrithub.io/386561
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-11-13 14:57:49 -05:00
Piotr Pelplinski
d17345183e lvol: updated documentation.
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I566a4d277ac1595fa7b4d630df52301a9abeec49
Reviewed-on: https://review.gerrithub.io/381629
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-10-26 15:04:27 -04:00
Dariusz Stojaczyk
5d0bdadc4e changelog: add virtio-initiator
Change-Id: I7fb7adb2d7ffdd4e99544a9b96c4342dde9b840e
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/383502
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-10-24 13:40:20 -04:00
Daniel Verkamp
c7e322fe02 changelog: clean up and fill out v17.10 details
Change-Id: Iafe021f5d7e38bfea8cefd297439e33efe8a1e4e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/383495
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-10-24 13:40:20 -04:00
Tomasz Zawadzki
2df9f03c47 blobstore: New API call to destroy blobstore [1/3]
Currently exposed API allows to load/unload and to
initialize blobstore on a device.

A spdk_bs_destroy() call is added in order to reach
functional parity with spdk_bs_init(). It was not
possible to remove blobstore from device from within
SPDK previously.

spdk_bs_destroy() takes blobstore pointer as argument
(instead of bs_dev), because blobstore has to be already
loaded to destroy it.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2c493a4407868fcf08fd1766a19fc8463f634ef5
Reviewed-on: https://review.gerrithub.io/382019
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-10-20 16:06:53 -04:00
Paul Luse
c5e63c71ff blobcli: add interactive shell mode
All of the same commands and options are available but by
starting with the -S option the user enters an interactive
shell mode to interact with blobstore significantly
improving response time by only having to init DPDK and NVMe
subsystems one time.

Change-Id: Ib927ba0848166dba1090484cecbbcf011122b714
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/380833
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-10-18 12:46:06 -04:00
Piotr Pelplinski
c0ed81a72b bdev_pmem: documentation
This patch adds documentation adn test plan for PMEM functional tests

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ia2e8bcb9901ab216c3e45d2db6d06a3a02fdd308
Reviewed-on: https://review.gerrithub.io/378618
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-10-17 16:28:16 -04:00
Daniel Verkamp
8acc4ae5da nvme: add controller option to specify host ID
A random host ID is generated per SPDK application startup if the user
doesn't specify a host ID during controller startup.

This also changes the default host NQN for NVMe-oF connections to a
random UUID NQN based on the host ID.

Change-Id: Ib0f70dd63e53087716842b412a1f134a9991d4da
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/380528
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2017-09-29 11:43:25 -04:00
Tomasz Zawadzki
27f44662ac lvol: Logical volume implementation
Change-Id: Ia96ae78ff9530d953181ac5f7255a38f3c8ec430
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Reviewed-on: https://review.gerrithub.io/375392
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-14 19:30:54 -04:00
Daniel Verkamp
0caab4e1da nvmf: add Write Zeroes support
Change-Id: I743f5e4d1c24ad5ef9f1fef4c2678e347b179a9f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/377260
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-14 17:41:52 -04:00
Paul Luse
9b99f2c0f3 log: add new parm to spdk_trace_dump()
Allows called to specify something other than stderr such as
stdout. Existing callers updated to use stderr.

Change-Id: I48a703e1474a45952878121a83c19fef1c43d630
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/377420
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-09-06 17:33:03 -04:00
Daniel Verkamp
e83f976743 bdev: introduce APIs with block units
Add new versions of all of the I/O calls that take parameters in blocks
instead of bytes.  These are intended to replace the old APIs, but
we'll keep them for now to preserve compatibility.

Change-Id: I85ab665c653e8c697016c628837d49aa0c3bfcd0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376255
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2017-09-06 13:01:13 -04:00
Daniel Verkamp
691912ef44 nvmf: allow user to specify NSID via conf and RPC
The nvmf library now supports the ability to assign arbitrary NSIDs,
rather than automatically assigning the next one in line.  Expose this
functionality to the user via the configuration file and RPC interfaces.

Change-Id: Ia85a9a6dfe31a2cd0605c7a6c098eec0c1b7de68
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376463
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-09-05 13:03:09 -04:00
Daniel Verkamp
a2db49a121 nvmf: add AllowAnyHost option to subsystems
The previous behavior with an empty host NQN whitelist was to allow any
host to connect.

Change-Id: I5401e52d96642cf20afe0d50c692613e67262edf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/376432
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-09-05 13:03:09 -04:00
Daniel Verkamp
47182bd7c9 io_channel: allow user to provide a thread name
This moves the thread name setting code into the generic SPDK thread
setup code, so now all spdk_threads can be named, not just ones created
by the event framework.

Change-Id: I6c824cf4bcf12fe64a8e2fc7cdc2d6c949021e40
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/375220
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-29 13:11:08 -04:00
Jim Harris
179ed697b3 blob: add readv/writev support
Most of the work here revolves around having to split
an I/O that spans a cluster boundary.  In this case
we need to allocate a separate iov array, and then
issue each sub-I/O serially, copying the relevant
subset of the original iov array.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d46b3f832245900d109ee6c78cc6d49cf96428b

Reviewed-on: https://review.gerrithub.io/374880
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-22 18:19:00 -04:00
Seth Howell
d21ec1f00d lib/nvme: Add spdk_nvme_ns_get_ctrlr to the public api
This can be used in callbacks when a user has a namespace but needs to
call controller specific functions such as alloc_io_qpair.

Change-Id: I00c931e2846e42f540c17f3254fe97a45e9bd079
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/375022
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-22 16:31:05 -04:00
Ben Walker
ea957ff3a6 Fix up merge conflicts in the CHANGELOG
There were a number of unflagged merge conflicts in the CHANGELOG
that resulted in duplicated sections and notes being out of place.
Clean them up.

Change-Id: I88a590fad9d8812554665102e764bdaeccaea827
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/374714
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-08-17 19:37:23 -04:00
Ben Walker
e5f6a99b61 nvmf: No longer use in capsule data buffers for CONNECT
Previously, the shared buffer pools were allocated on the
nvmf controllers. When a new connection was established,
the CONNECT command needs a 4k buffer, but we didn't know
which nvmf controller it belonged to until after the
CONNECT command completed. So there was a special case
for CONNECT that used in capsule data buffers instead.

Now, the buffer pool is global and always available. We
can just use that always, with no more special cases.

This has the additional nice side effect of allowing
users to run the target with no in capsule data buffers
allocated at all.

Change-Id: I974289f646947651c58d65cf898571d80e9dee9b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/374360
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2017-08-17 16:55:55 -04:00
Ben Walker
053d5733e6 env: Add a default value for mempool cache size
This is just a convenience and replaces the common practice
of passing -1.

Change-Id: Id96734307ebf52ef0ee7dba0e7ac89602b2b5b1a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/374520
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-17 16:55:55 -04:00
Changpeng Liu
e36f83d64e bdev_nvme: disable hotplug feature by default
When users don't enable hotplug option in their configuration
section, SPDK will enable it by default. DPDK will print probing
messages continuously for NVMe devices which don't belong to SPDK.

Change-Id: I8c43335a282ecba206b4b5305bd881d2bd07836e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/374486
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-17 13:38:02 -04:00
Daniel Verkamp
99f838a36a nvme: add support for NS Optimal IO Boundary
An optional field was added in NVMe 1.3 to indicate the optimal I/O
boundary that should not be crossed for best performance.  This is
equivalent to the existing Intel-specific stripe size quirk.

Add support for the new NOIOB field and move the current quirk-based
code so it is updated in nvme_ns_identify_update().

Change-Id: Ifc4974f51dcd59e7f24565d8d5159b036458c6e5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/373132
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-16 15:55:29 -04:00
Seth Howell
670dc20f63 bdev: add write_zeroes function
Add functionality to the bdev layer to handle the nvme write_zeroes
function.

Change-Id: I0dadad273b28c16db5a2275f7d8d57e98253a8d3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/372171
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-08-15 15:27:09 -04:00
Daniel Verkamp
671179409c bdev/aio: allow user to override block size
Add a file-backed AIO bdev to test it out.

Change-Id: Ifdf206bbdf6cae9379fdc02c80755e96a7198bce
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/373673
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-11 19:36:30 -04:00
Daniel Verkamp
8bcb643266 changelog: add a note about VTune integration
This was added in SPDK v17.07, but the changelog wasn't updated.

Change-Id: I897d8c578c1506bbd022b6cde00736f854da6d6f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/373633
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-08-10 13:09:03 -04:00
Ben Walker
60c38d4022 bdev: Change unmap to use offset/len instead of descriptors
This is far simpler, although it does limit the bdev
layer to unmapped just one range per command. In practice,
all of our code reports limits of just one range per command
anyway.

Change-Id: I99247ab349fe85b9925769e965833b06708d0d70
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/370382
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2017-08-04 20:03:37 -04:00
Ben Walker
4b8e7d50c0 bdev: Add an example fio plugin
Change-Id: I5e0ebb8d452a41ad848b319af9bb978546807d5e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/366495
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-08-04 20:03:37 -04:00
Daniel Verkamp
054c533862 changelog: fill out release summary line
Change-Id: Id038a1ba5c050ac4aa3fe5b621fe2f5968bd74c5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/369912
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-20 16:58:27 -04:00
Daniel Verkamp
ce4fcbce71 nvme: add I/O qpair creation options
Add a new struct spdk_nvme_io_qpair_opts to allow the user to override
controller options on a per-I/O qpair basis.

Existing callers with qprio == 0 can be updated to:

  ... = spdk_nvme_ctrlr_alloc_io_qpair(ctrlr, NULL, 0);

Callers that need to specify a non-default qprio should be updated to:

  struct spdk_nvme_io_qpair_opts opts;
  spdk_nvme_ctrlr_get_default_io_qpair_opts(ctrlr, &opts, sizeof(opts));
  opts.qprio = SPDK_NVME_QPRIO_...;
  ... = spdk_nvme_ctrlr_alloc_io_qpair(ctrlr, &opts, sizeof(opts));

Change-Id: I8ac3ea369535cfde759abbe75e1d974b6450a800
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/369676
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-18 18:33:20 -04:00
Daniel Verkamp
15337ade6b changelog: add vhost-blk, GPT
Change-Id: Iedae857cba4d69278013c31395515062bfa2f829
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/367741
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-07-14 13:07:03 -04:00
Daniel Verkamp
49977bc28e changelog: begin filling out v17.07 section
Change-Id: I83813557d33e34be13dab543ab513127b568bf80
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/362044
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2017-06-02 12:04:58 -04:00
Daniel Verkamp
509fedf011 doc: link to vhost docs from readme and changelog
Change-Id: Ia12de190c5b6c42d1a52dafc5fbab9de93913037
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-03-29 21:40:11 -07:00
Daniel Verkamp
06e4124b24 changelog: add blobstore and vhost
Change-Id: Ie07c165e6a3d6c231e3703d25303e67a1ba7fb5e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-03-29 11:25:16 -07:00
Ben Walker
0829424e19 conf: No longer allow wildcard claiming of NVMe devices
All devices must be specified by BDF. Add support for scripts
to use lspci to grab the available NVMe device BDFs for the
current machine.

Change-Id: I4a53b335e3d516629f050ae1b2ab7aff8dd7f568
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-03-14 08:38:04 -07:00
Daniel Verkamp
55dc5a1c1b changelog: begin filling out the next release
Change-Id: Ie8478974fe8cc8acb7b5823a4c75fe68aef7e882
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-02-17 09:53:43 -07:00
Daniel Verkamp
aa61dc505e changelog: use #/##/### headers consistently
Change-Id: Ie85c86a72999cdaa2e7058a3918be06347ea7734
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-02-17 09:53:43 -07:00
Daniel Verkamp
93ab7c31e4 doc: move porting guide into doc/ and add to index
This moves all the documentation under the doc/ tree for consistency.

Replace the link to include/spdk/env.h to raw text - the Doxygen
Markdown parser will automatically turn it into a link since it is a
Doxygen input file.

Change-Id: If6bde1cfc965cada2c741acd5505026545ad4cf7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-17 16:25:04 -07:00
Daniel Verkamp
0f6c389e69 changelog: add bdev changes and NVMe multi-process
Change-Id: Ib68feab08123d6b631a506cd9024c39eb36c5832
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 15:09:01 -07:00
Daniel Verkamp
fbb277ea58 changelog: fill out major 16.12 features
Change-Id: I09f3505049970645aca33191c80d893a68828a76
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-08 16:59:56 -07:00
Daniel Verkamp
720d5fa683 changelog: add note about build/lib move
Change-Id: I75c2359e0ca87d33931ea89e4bd623f07209233b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-18 12:51:47 -07:00
Daniel Verkamp
0aec36be1f nvme: return virtual address in SGL callback
Instead of the next_sge callback returning the physical address
directly, make it return the virtual address and convert to physical
address inside the NVMe library.

This is necessary for NVMe over Fabrics host support, since the RDMA
userspace API requires virtual addresses rather than physical addresses.
It is also more consistent with the normal non-SGL NVMe functions that
already take virtual addresses.

Change-Id: I79a7af64ead987535f6bf3057b2b22aef3171c5b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-11-09 08:57:40 -07:00
Daniel Verkamp
2832b2161f changelog: deallocate to dataset management change
Change-Id: I2417a23e0ce71476035779022a51fbcca3dff838
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-06 08:58:37 -07:00
Ben Walker
6b1e4e732d Drop libpciaccess and switch to DPDK PCI
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.

Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.

Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 15:59:00 -07:00
Daniel Verkamp
6b566ff11d changelog: document nvme request_mempool change
Change-Id: I397a5a7871545e5e401bce0926c3f3a4365aab05
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-04 14:10:43 -07:00
Daniel Verkamp
0fc7daffe5 Replace "NVMf" with "NVMe over Fabrics" in docs
Use the final official spec name rather than a non-standard
abbreviation.

Change-Id: I4d797294be35b2fbf7b39570ea3246eb71c8d8ce
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-08-08 16:35:11 -07:00
Ben Walker
daad9666f3 Update the changelog to prepare for the next release.
Change-Id: I0aab56c075c7d1956781380b45b36f068b0776c3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-08 08:44:17 -07:00
Daniel Verkamp
771d2c1a52 changelog: updates since 16.06
Change-Id: Ide070c4a2a69b98603aefbbe75ecef2d7d7f3f73
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-30 13:50:41 -07:00
Daniel Verkamp
39f552fc2d changelog: update to 16.06 and add NVMf
Change-Id: Iebc8bba692530977d837fb6f3781fdb52a7e1945
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-07 10:16:08 -07:00
Daniel Verkamp
bce8ce481d CHANGELOG: add NVMe E2E and VFIO PCI support
Change-Id: Icc6073bc3ef46dfa3704d410e65f0a1ea248d6bc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-04-28 16:00:33 -07:00
Daniel Verkamp
3272320c73 nvme: make I/O queue allocation explicit
The previous method for registering I/O queues did not allow the user
to specify queue priority for weighted round robin arbitration, and it
limited the application to one queue per controller per thread.

Change the API to require explicit allocation of each queue for each
controller using the new function spdk_nvme_ctrlr_alloc_io_qpair().

Each function that submits a command on an I/O queue now takes an
explicit qpair parameter rather than implicitly using the thread-local
queue.

This also allows the application to allocate different numbers of
threads per controller; previously, the number of queues was capped at
the smallest value supported by any attached controller.

Weighted round robin arbitration is not supported yet; additional
changes to the controller startup process are required to enable
alternate arbitration methods.

Change-Id: Ia33be1050a6953bc5a3cca9284aefcd95b01116e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-14 16:00:54 -07:00
Daniel Verkamp
04cde460cf Add a changelog
Import the release notes from previous releases and fill out the major
features for the next release.

Change-Id: I5050d6cc83a92fac4f1dbed5f7ef039cca208765
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-09 07:25:09 -07:00