852 Commits

Author SHA1 Message Date
Pawel Wodkowski
6397801009 jsonrpc: fix received JSON buffer overwrite
The receive buffer and JSON values array are shared across all requests
on one connection. If RPC handlers deferre processing response, do the
lazy decode or capture JSON by pointer instead of copying it then
content of the request array might be overwritten by now. As we don't
have any requirement here we must assert that the received request is
valid till response is finished.

Fix this issue by copying request data and work on the copy. This change
also void the need of having JSON RPC 'id' field releasing over 128
bytes from each spdk_jsonrpc_request.

Change-Id: I665be446cbcd8f625e5a73514582efad3021a4ff
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-10 14:31:37 +00:00
Shuhei Matsumoto
ed45352564 dix: Return DIF error information at verification for separate metadata payload
For separate metadata payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.

This change will make logging of DIF error and comparison between
injection and verification possible for separate metadata payload.

Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.

Change-Id: Ifce2130e902bc090cbe205fa8df12559739ced57
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435097
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
65624bd5e4 dix: Generate DIF for separate metadata payload
This patch adds APIs to generate and verify DIF for SGL
payload with separate metadata as byte alignment and granularity.

Change-Id: I4e553c9b6f8c96e576a69115002963d32379d439
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/434150
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
045e77c235 dif: Return DIF error information at verification for copied extended LBA payload.
For copied extended LBA payload, Introduce a struct to collect DIF error
information and pass the reference to the struct to bit-flip error injection
and verification.

This change will make logging of DIF error and comparison between
injection and verification possible for copied extended LBA payload.

Merging this patch to the previous is possible but add this as a separate patch
to reduce the patch size.

Change-Id: I9f7ac1332fe63cb206c692d0876c8de41ef9286f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/435227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
bc5507e440 dif: Copy data and generate DIF for extended LBA payload
This patch adds APIs to copy data and generate and verify DIF
for internally created extended LBA payload.

This patch is for read strip and write insert operation to make
DIF transparent to the upper layer.

Change-Id: I025f35571490aa197aac5e7572549e31c2479b41
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/432987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00
Shuhei Matsumoto
65e491a8de dif: Return DIF error information by struct at verification
Introduce a struct to collect DIF error information and pass
the reference to the struct to bit-flip error injection and
verification.

This change will make logging of DIF error and comparison between
injection and verification possible.

Change-Id: I43941f1a7f4b0ad1e8ff324f2ac8fad9e195a100
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438023
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-09 19:54:35 +00:00
Seth Howell
1ed1f968ae mock: add MOCK_CLEARED_ASSERT macro
This is useful for proving to scan-build that we are not using a mocked
function. Sometimes it needs to be called repeatedly every time any mock
variables are called.

Change-Id: Id0ef196620c2984252f792a7e40162e10b989748
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-01-09 08:36:40 +00:00
Seth Howell
baea123097 crypto: clear g_var in driver init error paths.
It is probably best to clear the pointers to global memory when those
pointers are no longer valid. It also helps make the unittest a little
bit clearer.

Change-Id: I73f9568f40ba8bff63f4e3346c83a48766f26129
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-09 08:36:40 +00:00
Konrad Sztyber
ea002f5068 build: use linker specified by the LD env variable
Modifed scripts/detect_cc.sh to take additional parameter specifing
the linker to use. Default to LLD on FreeBSD systems.

Change-Id: Idf97e9676a144028c0803d272ae6f0e903b0dd1f
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-07 19:49:11 +00:00
Shuhei Matsumoto
18461cba7c dif: Clean-up APIs by introducing context data
This patch reduces the number of parameters of APIs by setting
necessary information to context struct and pass it to APIs.

This will simplify the code and will clarify the core logic.
Besides, by introducing context data, update of reference tag
is further consolidated.

Change-Id: I1b3eec596847eb4ef1cf43441035df21f746192a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438021
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-07 19:45:00 +00:00
Shuhei Matsumoto
a5e3d61d6b dif: handle the case that metadata is supported but DIF is not enabled as normal
The case that metadata is supported but DIF is not enabled was
handled as error. But the case is described as an normal case
in NVMe specification.

This patch changes the case to normal.

Besides a break was missing and added in this patch.

Change-Id: I784c36fed3b92da9ae5500d945f1dc20a2ca406b
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/438362
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-07 19:45:00 +00:00
Ziye Yang
f31096782c nvmf: Only present subsystem if it is ready
We do not want to present those subsystems which are not
ready.

Change-Id: I7f5c171fbac4c31d839421e37e93e62569c0e87a
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/437222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-07 06:02:26 +00:00
Chunyang Hui
19feb4e181 nvme: add security receive and security send wrapper
Change-Id: Id25040d62f89d4e8f2268bb3383c5665c0508f5a
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-07 05:51:07 +00:00
lorneli
02be32d482 bdev: optimize coversion of bytes to blocks
Since division is more expensive than right shift operator, in
function spdk_bdev_bytes_to_blocks, use right shift instead of
division if the blocklen of bdev is a power of two.

Change-Id: Ib3dbc792e86582bba30b3dc028efbd12c69075ba
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/438318
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-01-02 08:31:59 +00:00
Ben Walker
f7e461942e test: Use ut_multithread framework in blob unit test
Change-Id: Ibc6d3ee1881943a9327a0512ad28de9d254ac48c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-12-31 06:17:02 +00:00
Seth Howell
679bf39e59 test/unit: fix scan-build error in the crypto unittests.
The scan-build error this fixes assumes that we do not reallocate an
iov_base for the g_bdev_io object in between calls to
_crypto_opration_complete. So I explicitly tell it that's what we are
doing.

Change-Id: Ie15c517ea60a2a527a0520005cb044ab2ba4412e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437988
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2018-12-31 05:49:15 +00:00
Ben Walker
270a25df8d test: Use ut_multithread framework in bdev unit test
Change-Id: Ibf638312840ef320b44217861b92cf693af6eff5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437598
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-23 00:56:01 +00:00
Ben Walker
999d6df4ca test: Use ut_multithread framework in bdev/part test
Change-Id: Ia8a27251ee87d0784e0a26dc2dbecf787356da2d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/422471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-23 00:56:01 +00:00
Darek Stojaczyk
edaea9164d vhost: move vid to session struct
Each connection is created with the `new_connection`
rte_vhost callback with a unique vid parameter. Storing
the vid inside the device struct was sufficient until
we wanted to have multiple connections per device.

Change-Id: Ic730d3377e1410499bdc163ce961863c530b880d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/437775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-23 00:42:54 +00:00
Darek Stojaczyk
73844cccf4 vhost: introduce device sessions
Grouped a few spdk_vhost_dev struct fields into a new
struct spdk_vhost_session. A session will represent the
connection between SPDK vhost device (vhost-user slave)
and QEMU (vhost-user master).

This essentially serves two purposes. The first is to
allow multiple simultaneous connections to a single
vhost device. Each connection (session) will have access
to the same storage, but will use separate virtqueues,
separate features and possibly different memory. For
Vhost-SCSI, this could be used together with the upcoming
SCSI reservations feature.

The other purpose is to untie devices from lcores and tie
sessions instead. This will potentially allow us to modify
the device struct from any thread, meaning we'll be able
to get rid of the external events API and simplify a lot
of the code that manages vhost - vhost RPC for instance.
Device backends themselves would be responsible for
propagating all device events to each session, but it could
be completely transparent to the upper layers.

Change-Id: I39984cc0a3ae2e76e0817d48fdaa5f43d3339607
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/437774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-23 00:42:54 +00:00
Ziye Yang
58f1624497 nvmf: add the transport shared buffer num configuration option.
Previously, we allocate the buffer size according
to the MaxQueueDepth info, however this is not exactly
a good way for customers to configure, we should provided
a shared buffer number configuration for the transport.

Change-Id: Ic6ff83076a65e77ec7376688ffb3737fd899057c
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 19:55:57 +00:00
Shuhei Matsumoto
3947bc2492 util/crc16: Add spdk_crc16_t10dif_copy to use in read strip and write insert
For DIF and DIX, read strip and write insert operation will copy
data together with DIF generation and verification.

This patch adds spdk_crc16_t10dif_copy for those cases.

Change-Id: I9a77fa8b367486fe0b6704d58dcdf95d5210c875
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
f0e00ef287 dif: Inject bit-flip error for extended LBA payload
This patch adds APIs to inject bit flip error into any field
of the extended LBA payload.

Change-Id: I3ca601999e55ea6228bb525ac8c0744c7df32398
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/434292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
f9cbc493dc dif: Use simple helper functions to iterate iovec array
Introducing helper functions will simplify a little complex
implementation.

Change-Id: I94911497b8aecc0ffbd148519ce22501ef69c79d
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/435387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-20 17:52:29 +00:00
Shuhei Matsumoto
8b6c22b1a0 dif: Generate DIF for extended LBA payload
DIF and DIX will be supported in SPDK throughout, e.g., NVMe
driver, NVMe-oF initiator and target, NVMe block device, malloc
block device, SCSI, iSCSI target, FIO plugin, and Perf.

Generic and common APIs to generate and verify DIF and inject
bit flip error to any field will be helpful for them.

This patch is the first in the patch series.

This patch adds APIs to generate and verify DIF for SGL extended
LBA payload as byte alignement and granularity.

Change-Id: Ie6588d960113761f10efbf2d2a3cae004af37ce8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/432261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-20 17:52:29 +00:00
Ben Walker
b035e2b848 test: Use ut_multithread framework in blobfs async test
Change-Id: I3c20fc64b6c39286aed8a9f89a5700db5b30a736
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-20 15:49:39 +00:00
Ben Walker
36adf2d68c test: Use ut_multithread framework in lvol unit test
Change-Id: I618c0ee6034cc082a7370e19d91b3f90ac3b589d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/422468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-20 15:49:39 +00:00
Seth Howell
8caf6b6469 test: fix bogus scan-build error in blobfs_async_ut
Scan-build :/

Change-Id: Iada0b716457178b3cba2ba90cf375a4011518557
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437737
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 15:49:39 +00:00
Ben Walker
06a0d9ab33 test: Use ut_multithread framework in nvmf/ctrlr
Change-Id: I497abe0996287203f4b99850202a23f35d360f5e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/422467
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-19 08:47:47 +00:00
Ben Walker
84b5a8c1d6 test: Use ut_multithread framework in nvmf subsystem unit test
Use the default thread implementation instead.

Change-Id: I844b02ba3c45d323e9f16b328a31b9d38dc8f754
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/422466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-18 23:31:08 +00:00
Ben Walker
7b94053827 thread: Rename spdk_thread_msg to spdk_msg_fn
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.

Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Shuhei Matsumoto
e303567bc1 util/crc16: Add init_crc parameter as seed value to spdk_crc16_t10dif
Add init_crc parameter as seed value to spdk_crc16_t10dif API to generate
a CRC value spanning multiple separate buffers.

This will be necessary for upcoming DIF/DIX patches.

Having init_crc parameter is general, and so change the existing API
without adding seed version of the existing API.

Change-Id: I0ac7919b18013967e41829dcedd3e4e73204d5d6
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/437204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-18 17:28:56 +00:00
Ziye Yang
94cd652b18 nvmf/tcp: Add a poller to check the timeout of each qpair
This makes the timeout check for each qpair in the group
efficient. If there are many qpairs in the group, we
can scale.

Change-Id: I75c29a92107dc32377a2ef7edb5ac92868f1c5df
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/435277
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-18 08:34:30 +00:00
Ziye Yang
04d09f9207 nvmf/tcp: Use generic transport options structure
Remove the unnessary fields in spdk_nvmf_tcp_transport

Change-Id: I632608ba654b30f3511f5e1d925c6743c9100365
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-17 04:25:25 +00:00
Ben Walker
605e530a4e thread: Use TLS to accelerate thread look up
Change-Id: I8136db265c9cb8d61de4845ce6eaff2351b5b597
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435939
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-14 18:54:03 +00:00
Wojciech Malikowski
212eb89376 util: added bit array bitmask load, store and clear
Change-Id: I7cd47184d0dfb038297c0ac04d7dfb5d31f96b6b
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2018-12-14 15:34:53 +00:00
Maciej Szwed
54f6083a81 blob: Remove snpashot from the list only on blob deletion
When last clone of a snapshot is being deleted
we remove that snapshot from snapshots list.
We should not do that as it still works as a
snapshot and it is read-only, but it does not list
as a snpashot from get_bdevs. Instead remove snapshot
entry from the list when blob that represents that
snapshot is being removed.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I8d76229567fb0d9f15d29bad3fd94b9813249604

Reviewed-on: https://review.gerrithub.io/436971
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 22:26:56 +00:00
Jim Harris
0b0af8f30b reduce: calculate vol size from backing dev size
This more closely matches the sequence of events
a user will take in creating a compressed volume.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2311aed49e2a623feb9c5fbdbd25fb3201aeeeb5

Reviewed-on: https://review.gerrithub.io/436065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
212770db66 test/reduce: test 512 byte backing blocklen
Parameterize the existing unit tests to maximize
code reuse.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaa6b1bd52ad48ad4c19f6e6419e3e669b2d61e56

Reviewed-on: https://review.gerrithub.io/434117
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
383b117309 reduce: implement read and write with data
For write operations, copy data to req->buf and write
to disk.

If chunk already specified in logical map, read the
chunk first into req->buf, and overwrite with data
specified by the write operation.

If chunk not specified in logical map, fill logical
blocks not specified by the write operation with
zeroes.

For read operations, read chunk into req->buf first,
then copy relevant data into the buffers specified
by the read operations.

These operations are all functional, but have room
for future improvement.  For example, this patch
will issue a separate backing read/write operations
for each backing block in the chunk - this could be
optimized to coalesce operations where the backing
blocks are contiguous.

While here, clean up freeing bufspace in one of
the error paths - this needs to be freed using
spdk_dma_free instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6dbf4fc9a8fdf0f5424b1f1f9178c79891c96d0d

Reviewed-on: https://review.gerrithub.io/434116
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-13 14:55:39 +00:00
Jim Harris
84f8b6339f reduce: start implementation of spdk_reduce_vol_writev
Focus of this patch is adding the foundations of manipulating
the chunk_map and backing_block bit arrays and persisting
chunk maps and the logical map to the pm_file.

No data is writting the backing device yet.  That will come
in later patches.  This also does not rebuild the bit arrays
from the pm_file during spdk_reduce_vol_load - that will also come
in a later patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib5336dbe907c253e545704471de8c4e812bbc157

Reviewed-on: https://review.gerrithub.io/434115
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-13 14:55:39 +00:00
Jim Harris
3213962e91 reduce: replace ziperrno with reduce_errno
ziperrno was left over from earlier versions of this
patch set that used "sbzip" instead of "reduce".

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4159141b92727bb0a24db5f9aefcccaadc360d9e

Reviewed-on: https://review.gerrithub.io/436399
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-12-13 14:55:39 +00:00
Ziye Yang
3032fe53e3 nvmf/tcp unit test: fix the issue caused by forgetting rebase
Change-Id: I19bfc17636e0bf36d5a26e18ab2650ea6026f3a0
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/437043
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: Jim Harris <james.r.harris@intel.com>
2018-12-13 05:04:19 +00:00
Liang Yan
d1b38b92d2 test/nvme: replace PAGE_SIZE with OCSSD_SECTOR_SIZE
Same as 29be88f (test/blob: always use SPDK_BS_PAGE_SIZE instead of
PAGE_SIZE), ARM system could not find defination of PAGE_SIZE when
building nvme_ns_ocssd_cmd_ut.c., we use OCSSD_SECTOR_SIZE instead of
Page_SIZE here, also use OCSSD_SECTOR_SIZE instead of "0x1000" for const
uint32_t sector_size.

Change-Id: Ib3062232e44b0be26ade7c64340918f2f23ada03
Signed-off-by: Liang Yan <lyan@suse.com>
Reviewed-on: https://review.gerrithub.io/430802
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-13 05:00:54 +00:00
Yanbo Zhou
4c30f7fc78 test/nvmf: Add unit tests for NVMe-oF TCP transport
It is the initial patch to add the unit test for
NVMe-oF TCP transport.

Change-Id: Ia8c90a205fc78dd817f91675337ca81021211155
Signed-off-by: Yanbo Zhou <yanbo.zhou@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/426175
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-13 02:42:12 +00:00
Jim Harris
9c2aea2ad5 build: remove duplicate spdk.app.mk includes
spdk.unittest.mk includes spdk.app.mk, but some unit test
Makefiles include both spdk.unittest.mk and spdk.app.mk,
meaning spdk.app.mk gets included twice.  Fix that.

This hasn't been an issue because spdk.app.mk currently
only includes variables - but no rules.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I23d39e8084f79442fb06ae9b5a6a68d6134adff4

Reviewed-on: https://review.gerrithub.io/434281
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 18:07:53 +00:00
Shuhei Matsumoto
37af0cc930 iscsi: Propagate SCSI error status to split SCSI read/write commands
For split SCSI read command, if there is any failure in the sequence of
it, the first error must be propagated to all subtasks of it.

For split SCSI write command, if there is any failure in the sequence
of it, the first error must be propagated to the primary subtask.

Before this patch,
for read task:
- any failure is propagated to already completed subtasks, but
  is not propagated to any subtasks not completed yet, and
- if any failure occurs in non-primary subtasks, it is not propagated
  to the primary subtask.

for write task:
- if the primary subtask completes after any failure of non-primary
  subtasks, the failure will be overwritten by the success of the
  primary task.

This patch fixes these issues.

Change-Id: I2d878798cbb40a8c5bd6a6fe5efb32b8de4a8ecd
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 16:37:55 +00:00
Shuhei Matsumoto
69bef7f728 ut/iscsi: Use STUB in iSCSI unit tests
Change-Id: I815ac69b6468ab910da789c6980c27ebd14846da
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 16:37:55 +00:00
Shuhei Matsumoto
760c8def9c ut/scsi: Use STUB in SCSI unit tests
Change-Id: I74f2de3310341504850bd9034baefce6126f540f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-11 16:37:55 +00:00
Shuhei Matsumoto
308b62c6ba iscsi: Fix uncorrect LUN reset called when exiting connection.
This direct call of LUN reset has caused unexpected behavior
during heavy stress test.

Apply correct LUN reset handling for this case too.
Even if there are multiple LUNs, LUN resets are serialized in
the SCSI layer by previous patches and parallel calls of LUN resets
will not cause any issue.

Change-Id: I920e2864ef0fb41c6bd2b6153ef3caff6876e893
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/436115
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-11 16:37:55 +00:00