6715 Commits

Author SHA1 Message Date
Seth Howell
0e6a32deab nvme_rdma/nvmf: add cb_fns to check mr contiguity
This is necessary to confirm that a buffer that spans a 2_MB boundary is
still in a single MR.

Change-Id: If0d14e514ab2197a0d2e3af4f565f56d50591210
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/435179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-29 15:16:13 +00:00
Jim Harris
9cec99b84b env_dpdk: tell DPDK to not free dynamically allocated memory
This keeps us from having to deal with ALLOC and FREE events
for mismatching regions - which necessitated splitting new
regions into individual pages.  This caused all kinds of
problems with NVMe-oF - for example, buffers that spanned
memory regions, or bumping up against MR limits on RDMA
NICs.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I18dcdae148436b55d4481bb9fb8799f4832c7de1

Reviewed-on: https://review.gerrithub.io/434895
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-29 15:16:13 +00:00
Ziye Yang
7a39a68c4f nvme/tcp: disable the digest by default.
This patch disables the header digest and data digest
by default, which will improve the performance.

In the another patch, we will make it configurable.

Change-Id: Icdf8cda28217ec35a6b87bb932cdb1e4f8492471
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-29 05:02:32 +00:00
Ziye Yang
03370d0544 nvmf/host/perf.sh: Remove the delete nvme controller statement.
This removal operation is unnecessary.

Change-Id: Ifd525b00661adfeeecd12d460ed081575f2be6b2
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/434299
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-29 04:04:17 +00:00
Piotr Pelplinski
676717e4da bdev: calculate tsc_diff in bdev_io_complete
This will be required in following histogram patches.

Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I2eee6629243b7a4838a80dc1de33ae485c58081e

Reviewed-on: https://review.gerrithub.io/433874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-29 03:59:32 +00:00
Changpeng Liu
d2525134e7 nvmf: check block size is 512 aligned for each Namespace
NVMf target can't support extended LBA format for now, so print a
error log for those NVMe backend devices with extended LBA format.

Fix the issue #497.

Change-Id: Idda76ba934dd0eb45f92ae22b0b71398b3ae69dd
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/432799
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
2018-11-28 22:56:50 +00:00
Darek Stojaczyk
d815107bd5 nbd: fixup return codes
Minor cleanup just to make sure they are consistently <= 0.

Change-Id: I8427fd201e60e3f8ebbcf4929eb58ca164910623
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-28 22:55:09 +00:00
Darek Stojaczyk
f40ab9893d bdev/gpt: use per-base split tailq
Currently spdk_bdev_part_base_get_tailq(gpt_base) will
return the global gpt tailq containing all the gpt part
bdevs, which is not what callers of this function expect.

Although the spdk_bdev_part_base_get_tailq() is currently
unused for gpt parts, it's still worth fixing it to make
the behavior consistent with other part bdev modules.

Fix this by having per-gpt-base tailqs which contain only
associated gpt partitions.

Change-Id: Ib3c4286fcc6912f2a252beb5b3dcafc0e5316434
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434836
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-28 22:51:30 +00:00
Darek Stojaczyk
230857be48 bdev/split: use per-base split tailq
Currently spdk_bdev_part_base_get_tailq(split_base)
will return the global split tailq containing all
the split bdevs, which is not what callers of this
function expect. E.g. the construct_split_vbdev RPC
returns all split bdevs rather than the ones just
created.

Fix this by having per-split-base tailqs which
contain only associated splits.

Change-Id: I0fc25b28def0404f6a67152b5c21180e71660667
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434805
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-28 22:51:30 +00:00
Darek Stojaczyk
eb20b890a7 bdev/part: use spdk_bdev_part_base instead of spdk_bdev as an API param
Considering it's the part base object that's now accessible
in its remove callback, we can simplify the part API by making
it accept the part base object directly.

Change-Id: I87c3278929a063c115828d02e0def7fa536e6682
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434835
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2018-11-28 22:51:30 +00:00
Darek Stojaczyk
4d1894b3c4 bdev/part: use part_base as base desc remove ctx
Currently in the base bdev remove callback we don't
have access to anything but the spdk_bdev that's
being removed. Subsequent patches require the access
to more than that - e.g. some local metadata related
to that bdev.

By passing the part base object, we automatically get
access to e.g. spdk_bdev_part_base_get_ctx - a context
tightly associated with the part base, which can be
anything the upper layer (vbdev module) sets up.

Change-Id: Ifb99323978ef71ff6dd3b4ebf84fd21ef2920eb8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-28 22:51:30 +00:00
Jim Harris
5704be8987 build: remove xx_MODULES_LINKER_ARGS and xx_MODULES_FILES
Application Makefiles can now just add xx_MODULES_LIB_LIST
to SPDK_LIB_LIST.  This is possible now since all
SPDK libraries are linked with --whole-archive, so there
is no need to differentiate between "modules" libraries
and other SPDK libraries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iabf81a388b72d3b2a2f48287a8491ddc977722ac

Reviewed-on: https://review.gerrithub.io/434277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
0cc8f76225 build: remove unused LINKER_MODULES
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I944a2cd3fb646ce96c58412c72fdd299651ca148

Reviewed-on: https://review.gerrithub.io/434276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
e7d0bd8c17 build: remove unused SPDK_WHOLE_LIBS
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I11f6391b14de0c0be09d387f9d41768b429f855f

Reviewed-on: https://review.gerrithub.io/434275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
42b6771f40 build: add ALL_MODULES_* variables to spdk.modules.mk
This allows a lot of simplification to SPDK application
makefiles.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5fa463f6369834b84a8d92e79fa7768082209d7a

Reviewed-on: https://review.gerrithub.io/434274
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
6005fdba84 build: add missing xx_MODULES_LINKER_ARGS
There were several applications that were missing
either SOCK_MODULES_LINKER_ARGS or COPY_MODULES_LINKER_ARGS.

For the ones missing SOCK_MODULES_LINKER_ARGS (fuse and rocksdb),
the nvme inititator with TCP transport would not have worked
at all.

Adding COPY_MODULES_LINKER_ARGS to the bdev fio plugin enables
ioat which isn't critical, but adding it makes it consistent
with other apps and will allow its Makefile to be simplified in
some future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0098350d75d27ad2b2d408221b727698f5e902e4

Reviewed-on: https://review.gerrithub.io/434260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
9a47106188 build: use SYS_LIBS for modules system lib dependencies
This is a step towards simplifying the Makefiles for
SPDK applications.  After this change, all of the
xxx_MODULES_LINKER_ARGS variables are just a list
of SPDK libraries wrapped in --whole-archive.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0cf069c2f74e8998d42dea9598db55f333c861b7

Reviewed-on: https://review.gerrithub.io/434146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
3da5847023 build: add ENV_LINKER_ARGS to LINK_C and LINK_CXX
This allows us to remove $(ENV_LINKER_ARGS) from all of the
application Makefiles.

Modify spdk.unittest.mk to explicitly set ENV_LINKER_ARGS
to nothing, to avoid DPDK libraries from getting linked
into unit tests.  This seems better than creating separate
LINK_C rules for unit tests.  Provide a UNIT_TEST_LINK_ENV
flag to allow select unit tests to override this behavior -
it's needed for some of the env unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I647f2c848cf434023cf5a1f3a45869bbc4fc8102
Reviewed-on: https://review.gerrithub.io/434145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
80bfabcfcf test/unit/iscsi: use test_env.c
Unit tests shouldn't link in DPDK env libraries - they
should all use the test_env.c

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib08ae33224ed9a12fa1ff255c4769a005fc10369

Reviewed-on: https://review.gerrithub.io/434259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
2ac1cbbf07 build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53f0
Reviewed-on: https://review.gerrithub.io/434107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
0b5fa082db build: remove sock from SOCK_MODULES_LIST
The various *_MODULES_LIST are intended for the modules
that implement the specific interface - not for the
associated core library itself.

For example, BLOCKDEV_MODULES_LIST doesn't contain bdev,
it contains all of the bdev modules.  Similar for
COPY_MODULES_LIST and copy.

So make SOCK_MODULES_LIST consistent.  This is a real
pain because we have to add sock explicitly to a bunch
of individual Makefiles, but I have plans to clean that
up in some future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d293372cfb38bfa46c8478a8d9ca9ce23924747

Reviewed-on: https://review.gerrithub.io/434125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Changpeng Liu
2bebd09bd7 vhost/nvme: remove VHOST_USER_NVME_IO_CMD socket message
VHOST_USER_NVME_IO_CMD is designed to deliver NVMe IO command
header to slave target via socket, this can be used in BIOS
which will not enable Shadow Doorbell Buffer feature, since
we enabled the shadow BAR feature to support some old Guest
kernel without Shadow Doorbell Buffer feature, so the message
isn't required, just remove it.

Change-Id: I72e55f11176af2405c8cc09da404a9f4e5e71526
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/420821
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 13:24:42 +00:00
Changpeng Liu
fbc53ae3fb vhost/nvme: add shared BAR space to enable old Guest kernel
For some old Linux Guest kernels, the new NVMe 1.3 feature: shadow
doorbell buffer is not enabled, while here, make a dummy BAR region
inside slave target, when Guest submits a new request, the doorbell
value will be write to the shared memory between Guest and vhost
target, so that the existing vhost target can support both new
Linux Guest kernel(newer than 4.12) and old Guest kernel.

Also, the shared BAR space can be used in future which we can move
ADMIN queue processing into SPDK vhost target, with this feature,
the QEMU driver will become very small and easy for upstreaming.

Change-Id: I9463e9f13421368f43bfe4076facddd119f4552e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 13:24:42 +00:00
Maciej Szwed
6569a529d6 nvmf: destroy mutex on controller destruction
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0eb5c7891a8614313607cd006f23e00c75d7d789

Reviewed-on: https://review.gerrithub.io/434818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 11:04:53 +00:00
Tomasz Zawadzki
9e2f1bff81 trace: verify tpoint_id does not execeed SPDK_TRACE_MAX_TPOINT_ID
Change-Id: I8b873ba2ebfd8dc4736572206caac11a51ab985e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434022
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-26 19:37:37 +00:00
Liu Xiaodong
9d2f39ab0b lib/trace: records num-trace-entries for lcores
With the trace_size records for each lcore, spdk_trace
can read trace_file in which each lcore has different
number of trace entries.
Offset of each trace_history from the beginning of this
data structure.

Change-Id: I06afaba129812fe40ed000265fc66b02c5d9e3d9
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433503
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 18:34:18 +00:00
Liu Xiaodong
fd4c75721e lib/trace: set num-trace-entries by app param
Number of trace entries in circular buffer per lcore can be
assigned by the boot parameter of SPDK app with
"--num-trace-entries <NUM>"

Change-Id: I855ce6b4f14a716dcdd9078913da5ea8e577af3a
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433099
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 18:34:18 +00:00
Paul Luse
fa4def5ceb bdev/crypto: code simplification
vbdev registration was happing in a loop in 2 places immediately
following a call to claim(), moved the registration into the
function.

Change-Id: I880dccae02ac0262558119265d2940d0adca33dd
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/433727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-26 18:27:42 +00:00
Paul Luse
3be1436c09 bdev/crypto: optimization: use shared sessions
Initial implmentation had a 1:1 session to crypto op ratio. After
working with a similar concept in CompressDev checked to see if
this was required and apparently it is not.

Saves a decent number of API calls per crypto op and in the poller.
Also saves on mempool usage.  Performance improvement measurement
is WIP.

Change-Id: I73f2355e720a16fd46bc4a02657419f779f07cbb
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/433726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
2018-11-26 18:27:42 +00:00
Tomasz Zawadzki
6177d220fb bdev/qos: assert io channel when acquiring new reference
Change-Id: Ib546f14be158404ec14067e0f4f1ecb60def0f02
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-26 18:19:26 +00:00
Jim Harris
50656e7a07 test: modprobe nbd at the start of autotest
This helps avoid issues with nbd not being ready when
we need it.

Still keep various "modprobe nbd" in different test
files - this ensures these test files can still be
run in isolation.  But remove the various "rmmod nbd"
to ensure that nbd always stays loaded.  There is no
harm in just keeping nbd loaded all the time.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3597ff1f7ac53162c33a0384a33f33776cf6f182

Reviewed-on: https://review.gerrithub.io/434838
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-26 18:17:28 +00:00
WangHaiLiang
2f5feaf5b1 doc: update a payload example about rpc_client
Add 'id' description in jsonrpc_proxy.html
Fix issue #512.

Change-Id: Ifa4ee35ea21f70106e79910bccd63d8ed316248c
Signed-off-by: WangHaiLiang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/434294
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 16:52:13 +00:00
Maciej Szwed
be0eb272d8 tcp: Initialize mutex only if everything else succeeded
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib0bb6b40852ca4b49d46c2cbeb603b7a2ec4c46f

Reviewed-on: https://review.gerrithub.io/434080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-26 07:06:42 +00:00
Maciej Szwed
5ecc5ef110 nvme: free tqpair on error paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I7abcead55f2eda0a7b8308f81ace70d8a1a59f2d

Reviewed-on: https://review.gerrithub.io/434079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-26 07:06:42 +00:00
Maciej Szwed
dfff013c04 nvme: unlock mutex on failure
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id96bd06c61bc49d9ecbdb16f30804a7dda635771

Reviewed-on: https://review.gerrithub.io/434050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2018-11-26 07:06:42 +00:00
Ziye Yang
c1ce8db0df nvmf: Add the TCP/IP transport related host test
This is the first patch to enable the automated perf
test with TCP transport, which is used to guard the
correctness of the TCP transport related code.

Change-Id: Ic3df3aeae6df2024e22e50be70c7c61e36f85243
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-11-22 06:59:19 +00:00
Changpeng Liu
b2fb55192c scsi: fix possible memory leak for unmap command
Althrough it has very small chance to be executed, it's nice
to have it fixed.

Change-Id: I899681ccc13ed59c7fdd343ef7791df4e69e490f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433976
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-21 23:38:55 +00:00
Ben Walker
1f20593b87 nbd: Wait for kernel module to become ready on start up
The ioctl NBD_SET_SOCK can return EBUSY if the kernel module
hasn't loaded entirely yet. Wait for it to become ready.

Change-Id: If3c6d0d8bb678ef8cab0efc1c5e800e95e19133e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433939
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Ben Walker
bb005da50a nbd: Split spdk_nbd_start into two parts
This is going to need to wait for the kernel module
to load in the future, so split the function in
half.

Change-Id: I872d000acd4fc25737d5f50c8e0ae33641a6d7fa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Ben Walker
2fe6d1d5d4 nbd: Make spdk_nbd_start asynchronous
Add a callback to spdk_nbd_start so that it can complete
asynchronously. As of this patch, it always calls the
callback immediately.

Change-Id: I6156fb203145362afa5e4102183b6cf143051c0c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433937
Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Jim Harris
027a8e4b54 test/iscsi: remove reference to auth.conf in calsoft json
Also remove the discovery auth group from the json file,
and use RPC to set it instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I597b60a7c7e4de9d7e1719b0145ab2b11c6f26b0

Reviewed-on: https://review.gerrithub.io/434265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-21 20:09:07 +00:00
Darek Stojaczyk
d808a62db5 pci: introduce a global hotplug lock
Despite the scary commit title, this patch just unifies
per-driver mutexes into a single pci mutex.

On each hotplug we modify some DPDK global resources,
which per-driver locks aren't sufficient for. If
multiple threads try to attach devices at the same time,
then we'll likely have a data race. DPDK hotplug APIs
don't provide any kind of thread safety on their own.

Change-Id: I89cca9fea04ecf576ec5854c662bae1d3712b3fb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433864
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:25 +00:00
Darek Stojaczyk
9cf7d886af pci: do not manually unmap resources if probe fails
We need to do it only for DPDK 16.11, which leaks the
mappings otherwise. DPDK was fixed in version 17.02 with
the following commit:

e84ad157 (pci: unmap resources if probe fails)

Unmapping the resources twice doesn't actually cause
us any trouble, but prints an ambiguous error message.

Change-Id: I8b62e86d5fff8fe924dbf9ae2e37cff29298d412
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433863
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:04 +00:00
Pawel Wodkowski
bf9dbae56b tests: wait for each start and stop nbd disk RPC
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.

Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 07:28:31 +00:00
Maciej Szwed
b86e1be001 vhost: Move mutex initialization after possible failure paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I428027e13f2426684cff536430423d2e4953e29b
Reviewed-on: https://review.gerrithub.io/433824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:54:17 +00:00
Tomasz Zawadzki
65724c3f88 nvme/tcp: destroy controller when failing to create qpair
Change-Id: I5e7d16e377c03165f338709a71d6e4f03beffc0a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:51:08 +00:00
Ben Walker
7f6068f04d test/blobfs: Simplify threading in sync unit test
This will make future changes to the threading model
easier.

Change-Id: I4abfe0850648a0f7449804066d4a02823393e4a6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433680
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-21 00:59:08 +00:00
Changpeng Liu
e8b6523280 iscsi: check the length of initiator/target name
According to the RFC3720, the length of initiator name and
target name should not larger than 223 bytes, while here,
add a check for initiator/target name.

Change-Id: I1517a4c4e1b0a944b239665f38f5dfb46f0075d2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433797
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-21 00:55:55 +00:00
Andrey Kuzmin
af034b6de4 bdev: unregister bdevs top-down during shutdown.
There are some use cases such as multipath and RAID expansion where a
vbdev could have been registered before one of its base bdevs.

Currently we unregister bdevs at shutdown in reverse order of their
registration.  Continue to do that in general, but skip any bdev that
is still claimed.  Any bdevs skipped in this way will eventually be
unregistered once any bdevs that have claimed it have completed
unregistration.

Change-Id: Iafde9558430bc5ce56e8608ef50bcb2b5fbfbf71
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/432136
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 22:49:23 +00:00
Ben Walker
bc7b547368 nbd: Add a function to get the nbd path
Change-Id: I7d121e8c6985a1b355c130248963008da0c63ac7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 22:48:50 +00:00