Commit Graph

7295 Commits

Author SHA1 Message Date
Piotr Pelplinski
e5e427c94b histograms: add rpc calls
This patch adds RPC calls for histograms in bdev layer.
Following calls are added:
 - enable_bdev_histogram - enable/disable histogram structures for specified bdev and each of its channels.
 - get_bdev_histogram - merges histograms from all channnels and encodes histogram as base64


Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Ib423a919dc1cde7dd7d92247db5482cfb9d66956
Reviewed-on: https://review.gerrithub.io/c/433573
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 14:14:32 +00:00
Arshad Hussain
8f0d7e57fa app: Fix spelling error in iscsi_tgt.c
This patch fixes spelling error while printing
error message using SPDK_ERRLOG under file
app/iscsi_tgt/iscsi_tgt.c

Change-Id: I7e052822358dc72a65c8ca8a8ac908a7f74c8f4b
Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/441246
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 10:52:31 +00:00
Arshad Hussain
8ce4e176c7 bdev: Improve spdk_bdev_open() to print string describing error number
This patch improves spdk_bdev_open() to additionally print string
describing error number in event when spdk_bdev_open() fails.

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I68579b671152a6880347bff9e1aa0943a80196bc
Reviewed-on: https://review.gerrithub.io/c/441262
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 10:50:34 +00:00
JinYu
b8769cdb08 nvmf: Add the Keep Alive feature
The controller shall treat a Keep Alive Timeout in the same manner
as connection loss. If the Keep Alive feature is in use and the
timer expires, then the controller shall:
1, stop processing commands and set the Controller Fatal Status
(CSTS,CFS) bit to '1';
2, terminate the NVMe Transport connection;
3, break the host to controller association;
A timer poller is added to each subsystem to monitor timeout event.

Change-Id: I001afab8a6764f30c39df37fa96384180d117486
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439330
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-24 04:24:11 +00:00
Darek Stojaczyk
058c4c890b vhost/scsi: add per-session SCSI target state
Each Vhost SCSI session will now keep its local
SCSI targets state that can be accessed without the
global vhost mutex.

Hotplug will still add the SCSI target reference to
the device struct, but will also asynchronously tell
each active session to add it's session-local copy.

Hotremove, on the other hand, will now try to
asynchronously remove a SCSI target from all sessions
first and only afterwards it will remove it from the
device struct.

This allows us to safely hotplug and hotremove SCSI
targets into Vhost SCSI devices that have multiple
active sessions.

Each session will still use its management poller to
try to locally remove those targets that were scheduled
for hotremoval and the additional
spdk_vhost_dev_foreach_session() will now also try to
remove each one of them from the entire Vhost SCSI
device after making sure they were already removed from
all sessions.

Change-Id: Idd080b618768c71cd1cd564efeaf930bf79fb578
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439321
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-24 03:55:12 +00:00
Darek Stojaczyk
e8eef29003 vhost: add active sessions counter
Before we implement the support for multiple sessions
per device, we still need to make a few intermediate
changes that will require a counter of currently polled
sessions. So here it is.

Change-Id: I0a1d928eafa75efa1b5c2e6670a5ceb282c87fa4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441734
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-24 01:56:37 +00:00
Pawel Wodkowski
bbcb35f58b jsonrpc: support half closed connections
Fix case when remote is doing SHUT_WR but we still have requests in
progress. In this case we should finish requests, send response and then
close the connection.

Fixes #604

Change-Id: I009029c95e0557c7347a78c3a50d35b30fc8141e
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441718
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-23 22:26:16 +00:00
Maciej Szwed
adb39585ef lvol: add option to change default data erase method
Some users require to do write zeroes operation when
erasing data on lvol. Currently the default method is
unmap. This patch adds flag to spdk_rpc_construct_lvol_bdev
call that changes default erase method. This is also a base
implementation for possible future function for erasing
data on lvol bdev.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8964f170b13c2268fe3c18104f7956c32be96040

Reviewed-on: https://review.gerrithub.io/c/441527
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-23 22:25:37 +00:00
Ziye Yang
0652c0a6c4 test/nvmf: fix the error message in nvmf_lvol test
The error message in the chandler pool is:
bdev.c:1127:_spdk_bdev_finish_unregister_bdevs_iter: *ERROR*: Unregistering claimed bdev 'Malloc2'!
bdev.c:1127:_spdk_bdev_finish_unregister_bdevs_iter: *ERROR*: Unregistering claimed bdev 'raid0'!

And this patch can fix this.

Change-Id: I8a247344aa09b62a7395a5513631e7bef6cf0d32
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441319
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-23 01:43:14 +00:00
Darek Stojaczyk
d171896a40 vhost: move around some struct definitions
The backend struct will get some new dependencies
soon, so move its definition further in a header
file.

Change-Id: I39c25027312777c7e570b12511dc9c5e9b9023d4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2019-01-22 23:24:05 +00:00
Darek Stojaczyk
fb98b0e519 vhost: allow breaking from the external_event_foreach() chain
Returning negative value from a `foreach` callback
will now break the entire chain. This is required
for refactoring spdk_vhost_dev_foreach_session() to
use the same mechanism as external events. Before
we actually do the refactor, we add the only feature
that external events were missing.

Change-Id: I70bda3df99748de51429e329a056c37a3bc7e348
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2019-01-22 23:24:05 +00:00
Wojciech Malikowski
70b86ec995 ftl: Added unit tests for FTL library
This patch implements unit tests for the following modules:
 * band
 * PPA (Physical Page Address) translations
 * write buffer

Change-Id: Ia7292bd3027347e8a3da77dafe71cde2c016bf38
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431328
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 23:22:16 +00:00
GangCao
05b43152b2 QoS/Bdev: add RPC support for Read/Write separate bandwidth limits
This patch adds the RPC support for the Read/Write separate
bandwidth limit controls. The basic usage as following:

usage:
rpc.py set_bdev_qos_limit [-h] [--rw_ios_per_sec RW_IOS_PER_SEC]
                               [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                               [--r_mbytes_per_sec R_MBYTES_PER_SEC]
                               [--w_mbytes_per_sec W_MBYTES_PER_SEC]
                               name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --r_mbytes_per_sec R_MBYTES_PER_SEC
             Read megabytes per second limit (>=10, example: 100).
             0 means unlimited.
  --w_mbytes_per_sec W_MBYTES_PER_SEC
             Write megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I822ec4814d21adff9826ce03a6af3783b1b98f44
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/417650
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
GangCao
316cb3b150 QoS/Bdev: add the Read/Write separate bandwidth rate limits
This patch adds the support of read and write separate
bandwidth rate limits control with the configuration file.

Below is the example (in MiB) for the configuration section:

[QoS]
  Limit_Read_BPS Malloc0 100
  Limit_Write_BPS Nvme0n1 200

Change-Id: I0221516ce70c3fbb07b9e80c1c814ed5ba271c88
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/416672
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 23:21:43 +00:00
Pawel Wodkowski
1da6e2f5c5 lvol: make unique_id an array
We know how big this should be so no need to dynamically allocate it.

Change-Id: I00ae6cb7c7f525d946d213e0f58cc5fe05bcf932
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441128
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 19:45:15 +00:00
Pawel Wodkowski
d263cba089 lvol, blob: dont fail removal if IO fails
Everything need to be removed to get the remove callback called.
Otherwise we will end up with dangling devices and user callbacks
possibly not called.

Fixes #567

Change-Id: I37259f6cd97268060170a6b17a0c0df4d543a224
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-22 19:45:15 +00:00
Pawel Wodkowski
53bb2cc3bc blobstore: don't ignore bserror
If IO fail e.g. during hotremove error shouldn't be ignored as this will
trigger operations (like crc checking) that shouldn't be done. Also
false error messages are printed.

Change-Id: Ie023ddcd9bdba2378e69808302ff9978497c7852
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440889
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-01-22 19:45:15 +00:00
Tomasz Zawadzki
01455bb15b json_config: dont dereference when no next config entry
Change-Id: I83576452d9ece2445a52cb37524eab80bcb56d52
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441570
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 19:21:46 +00:00
Tomasz Zawadzki
a96748a782 json_config: verify jsonrpc client request allocation
Change-Id: I9f2c95881a7ee36630d9b7d3e2347cf5b8bfb847
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441568
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 19:20:51 +00:00
tone.zhang
88ddf0f22e UT: fix the sock_ut failure because of the port conflict
In SPDK sock_ut, the port 3260 is tested. It is conflict with the SCSI
Target Daemon (tgtd). If the service is enabled, it makes sock_ut failure.

Suite: sock
  Test: posix_sock .../home/ubuntu/spdk/lib/sock/posix/posix.c: 238:spdk_posix_sock_create: *ERROR*: bind() failed errno = 98
FAILED

The patch changes port 3260 to UT_PORT, and enhance the error log when
socket bind is failed.

Also enhance error log carrying port value in posix.c.

Change-Id: I31e47af49335cdf7eaffa44237860ebc140d2419
Signed-off-by: tone.zhang <tone.zhang@arm.com>
Reviewed-on: https://review.gerrithub.io/c/439983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 17:28:24 +00:00
Piotr Pelplinski
42dba6047b histograms: add unit tests for public bdev api
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idb725cc9d1477afa458f8d50624996031ddb1b9e
Reviewed-on: https://review.gerrithub.io/c/433644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 17:06:52 +00:00
Shuhei Matsumoto
307631e10a nvme_perf: Remove unnecessary zeroing the buffer by memset
Change-Id: I503eb154a1c4fd8ddb5b37855cd84b65118b70af
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/441269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 16:21:08 +00:00
Shuhei Matsumoto
21b049f939 nvme_perf: Support NVMe formatted with DIX
This patch adds DIF generation and verification for NVMe namespace
formatted with DIX.

Change-Id: Ic67c88c485aacffa40fd0c73ba164f2cb1f88991
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/439969
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 16:21:08 +00:00
Chunyang Hui
35cdee26ad bdevperf: add check functions for input parsing
Change-Id: I71f927bb809f3465a70f0329c2c5609088ec60b4
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-22 14:05:57 +00:00
Tomasz Zawadzki
27290f500d doc/lvol: add json rpc description for set_read_only_lvol_bdev() RPC
Change-Id: I1fc27b0b321275104c62d781e80ae1288d33b7cb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 14:04:40 +00:00
Darek Stojaczyk
0f068a6b65 test/env_dpdk_post_init: use only a single core
Starting DPDK on all cores causes fairly frequent hangs
on one of our CI BSD systems, so we limit DPDK to just
the first CPU core. This will let our CI pass while we
investigate the real issue.

Change-Id: Iaa5b665b1364acd04555063677adbd12230ed992
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441567
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-22 14:01:57 +00:00
Arshad Hussain
4cec12f208 lib: Fix spelling error in vhost_nvme.c
This patch fixes spelling error while printing
error message using SPDK_ERRLOG under file
lib/vhost/vhost_nvme.c

Signed-off-by: Arshad Hussain <arshad.super@gmail.com>
Change-Id: I295c603d5e07935b200669f3c870c6e7d1136936
Reviewed-on: https://review.gerrithub.io/c/441255
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-22 08:54:10 +00:00
Jie Wang
e24da0913c lvol: fix Null-checking after dereferenced
fix Null-checking after dereferenced

Change-Id: I5c9dad9c894e8d7cc9e9847c7c03d204fdbacfcb
Signed-off-by: Jie Wang <wangjie88@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/439094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-22 08:50:14 +00:00
Pawel Wodkowski
1c54ba1c17 vhost_nvme/rpc: free partially decoded request
Partially decoded request need to be free even if
spdk_json_decode_object() fails.

Change-Id: Icd00f835537dbaf197cc4f05930be8c543a534a6
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-22 01:09:59 +00:00
Chunyang Hui
26979c50c7 app/parse: add default return value for app parse function
Change-Id: Icae46bb96502c7edf3a453e0fff1d8560241c5ab
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-22 01:03:39 +00:00
Ben Walker
7800ef7ff2 fio: Correctly set thread after allocating it in bdev fio_plugin
This was broken recently when spdk_allocate_thread was modified
to no longer implicity set the thread.

Change-Id: I436e368f45ba908d1542cf1bbcb6de0b18c595dc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441066
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-21 13:54:46 +00:00
Ben Walker
c1fbefa1bb thread: Simplify two unit tests with nested poll calls
Now that spdk_thread_poll can be called from within
a thread, some of the unit tests can be a bit simpler.

Change-Id: Ib7b92c3f59862e5c128f5e0cc8a08f65e2b89e93
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441160
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-21 13:54:46 +00:00
Ben Walker
13586cd82d thread: Allow nested spdk_thread_poll calls
This wasn't allowed at first to make debugging these
thread changes easier. Now they're mostly solid, so
let's allow this behavior.

Change-Id: Ida689c738ed550ec46c3cb6cab6788a651c6088b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441159
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-21 13:54:46 +00:00
Ziye Yang
c43cb6a706 nvmf/tcp: fix the issues of qpair resource recycling to avoid memory leak.
This patch will solve the following two cases:

1 Free the pdu resources. Add the checkout of c2h_pdu_data_cnt of the qpair.
2 Do not recyecle the req accoriding to the pdu in the send_queue, but directly
recylcing the reqs in TCP_REQUEST_STATE_TRANSFERRING_CONTROLLER_TO_HOST state.

Change-Id: I5856c3421019ec49d576d3dae4c62fefbb3925ca
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/440847
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-21 07:45:09 +00:00
JinYu
a3c9ab66c8 nvmf/ctrlr: free ctrlr->qpair_mask when failure to create ctrlr
Fix potential bug. In _spdk_nvmf_subsystem_add_ctrlr(), befor free(
ctrlr) we should free ctrlr->qpair_mask. Because we set qpair->ctrlr
= NULL, when destroy qpair the qpair_mask is not released. For the same
reason, req->qpair->ctlr = ctrlr is placed at the bottom of the function.

Change-Id: I38e268b532ff3ce87721c02f15ac4f674856d103
Signed-off-by: JinYu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-21 03:52:02 +00:00
Seth Howell
cf73fb2f1f nvmf/rdma: add a pool of request_data structs
This change is related to enabling multi-sgl element support in
the NVMe-oF target.

For single SGL use cases, there is a 1:1  relationship between
rdma_requests and ibv_wrs used to transfer the data associated with
the request. In the ingle SGL case that ibv_wr is embedded inside of
the spdk_nvmf_rdma_request structure as part of an rdma_request_data
structure.
However, with Multi-SGL element support, we require multiple
ibv_wrs per rdma_request. Insted of embedding these
structures inside of the rdma_request and bloating up that object, I
opted to leave the first one embedded in the object and create a pool
that requests can pull from in the Multi-SGL path.
By leaving the first request_data object embedded in the rdma_request
structure, we avoid adding the latency of requesting a mempool object
in the basic cases.

Change-Id: I7282242f1e34a32eb59b55f326a6c331d455625e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/428561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-18 20:43:46 +00:00
Ziye Yang
3c88819bc0 nvmf/tcp: Use the common buffer cache for each polling group
Purpose: To avoid the buffer contention among different
polling groups if there are multiple core configurations
for NVMe-oF tcp transport.

Change-Id: I1c1b0126f3aad28f339ec8bf9355282e08cfa8db
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/440444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-18 19:21:58 +00:00
Tomasz Zawadzki
3bb815ae9f lvol: exposed marking lvol bdev as read only
Added set_read_only_lvol_bdev() RPC that marks
lvol bdev as read only.
This enables to create clones off of a base lvol,
without having to create additional lvol bdev with snapshot.

Change-Id: Ic20bbcd8fbebcef157acce44bfa1da035b0da459
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 19:02:36 +00:00
Piotr Pelplinski
8a3620ffe0 histograms: add histograms to bdev layer
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I677f0b368447fc928cba318c2771e64156ace416
Reviewed-on: https://review.gerrithub.io/c/433619
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 18:58:52 +00:00
Tomasz Zawadzki
8331ebd19c ftl: Added trace support
This patch adds supplementary tracing for OCSSD library to allow
for more efficient debugging and profiling of the user I/O path
as well as the background tasks (maintaining write pointer,
defragmentation, ANM event handling, etc.).

Change-Id: I741f1304f4ee0eba019e31bea7814af475c3296e
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431327
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 18:58:04 +00:00
Wojciech Malikowski
5c3591c72e ftl: Added initialization module
This patch adds OCSSD device initialization consisting of the
following:
 * retrieving SSD's geometry and its state
 * verifying configuration
 * initializing core structures (L2P, bands, write buffer, write
 pointers, bad block tables, etc)
 * initializing threads

Change-Id: Ieb643f78035702e834b9ef7830a3db215e35bfb7
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 18:58:04 +00:00
Darek Stojaczyk
8b9ccc1419 dpdk-sub: update the submodule to DPDK 18.11
Change-Id: Ieb6f4e7d329198f48c7f8a55a9e3eccc3091023b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 17:59:37 +00:00
Ziye Yang
3608464f04 nvme: fix the aer request sent to disabled controller
The purpose this patch is to fix the following issue:

https://github.com/spdk/spdk/issues/568.

The root cause of issue is in nvme_rdma_fail_qpair
since we want to recycle all outstanding rdma_reqs.
There is an aer req, the callback of which is:
nvme_ctrlr_async_event_cb. In this function, we
will call nvme_ctrlr_construct_and_submit_aer again,
however the nvme controller is already in shutdown state.
(The ctrlr->vcprop.cc.bits.en is set to 0).

Change-Id: I422f0fe5faf472e9a1cb6bbd174e806e6405b95c
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-18 17:44:08 +00:00
Jim Harris
da992ebcc4 always ignore submodules with git-status
In a couple of cases, we do specify --ignore-submodules
for the actual check, but when dumping the results
to the console, we omit the --ignore-submodules.  Fix that.

Same for check_format.sh - don't consider submodules for
files changes that suggest a CHANGELOG.md update.

This is in preparation for adding isa-l as an SPDK
submodule.  isa-l doesn't use any .gitignore files, so I
want to make sure we're just ignoring anything related to
isa-l build artifacts.  We could probably remove our
local DPDK submodule patch to its .gitignore after this
patch but will leave that for another time.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id9be9ce87569004c426d02c6cd44d645f3ff859f

Reviewed-on: https://review.gerrithub.io/c/440808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-18 17:21:02 +00:00
Jim Harris
f823b7f5c1 ipsec_mb: use EXTRA_CFLAGS to point DPDK to intel-ipsec-mb submodule
This allows us to remove the requirement to install intel-ipsec-mb to
system directories.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I579655a98b515cf148b7cd17823a9bb541ea6ad7

Reviewed-on: https://review.gerrithub.io/c/440785
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 17:21:02 +00:00
Ziye Yang
972b78802c doc/nvmf: Revise the NVMe-oF target user guide
We currently support both RDMA and tcp transport,
so need to revise the user guide doc to add some TCP
transport info.

Change-Id: Ic110a5c2ed1a678706adb5a80fc93bd6520c036d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/438482
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-18 17:14:29 +00:00
Seth Howell
caa06154bd rdma: fix the poll_group_create error paths.
It was possible to leak pollers if we had multiple devices in the
transport. The new err_exit path fixes this.

Change-Id: Iafd5643c67fae741113f10afe761af1988cb6a9b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439419
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 16:57:37 +00:00
Seth Howell
3709db7383 nvmf: add buf_cache_size to conf and rpc functions
Change-Id: I11514dc5981e9750d693202ff86af22b2064ae72
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 16:57:37 +00:00
Seth Howell
3d0c41383b doc/jsonrpc: add info about nvmf_create_transport
This was somehow missed.

Change-Id: I07363ed4ca53500c8d0c74a44cdaf837530298d0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439432
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-01-18 16:57:37 +00:00
Seth Howell
2740f26841 nvmf_rpc: add control for rdma buffer cache size
Change-Id: I63e433a7e0aa5374b12ca6c12a4855900cf2d2c6
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439431
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-18 16:57:37 +00:00