6704 Commits

Author SHA1 Message Date
Jim Harris
5704be8987 build: remove xx_MODULES_LINKER_ARGS and xx_MODULES_FILES
Application Makefiles can now just add xx_MODULES_LIB_LIST
to SPDK_LIB_LIST.  This is possible now since all
SPDK libraries are linked with --whole-archive, so there
is no need to differentiate between "modules" libraries
and other SPDK libraries.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iabf81a388b72d3b2a2f48287a8491ddc977722ac

Reviewed-on: https://review.gerrithub.io/434277
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
0cc8f76225 build: remove unused LINKER_MODULES
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I944a2cd3fb646ce96c58412c72fdd299651ca148

Reviewed-on: https://review.gerrithub.io/434276
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
e7d0bd8c17 build: remove unused SPDK_WHOLE_LIBS
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I11f6391b14de0c0be09d387f9d41768b429f855f

Reviewed-on: https://review.gerrithub.io/434275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 16:43:01 +00:00
Jim Harris
42b6771f40 build: add ALL_MODULES_* variables to spdk.modules.mk
This allows a lot of simplification to SPDK application
makefiles.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5fa463f6369834b84a8d92e79fa7768082209d7a

Reviewed-on: https://review.gerrithub.io/434274
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
6005fdba84 build: add missing xx_MODULES_LINKER_ARGS
There were several applications that were missing
either SOCK_MODULES_LINKER_ARGS or COPY_MODULES_LINKER_ARGS.

For the ones missing SOCK_MODULES_LINKER_ARGS (fuse and rocksdb),
the nvme inititator with TCP transport would not have worked
at all.

Adding COPY_MODULES_LINKER_ARGS to the bdev fio plugin enables
ioat which isn't critical, but adding it makes it consistent
with other apps and will allow its Makefile to be simplified in
some future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0098350d75d27ad2b2d408221b727698f5e902e4

Reviewed-on: https://review.gerrithub.io/434260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
9a47106188 build: use SYS_LIBS for modules system lib dependencies
This is a step towards simplifying the Makefiles for
SPDK applications.  After this change, all of the
xxx_MODULES_LINKER_ARGS variables are just a list
of SPDK libraries wrapped in --whole-archive.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0cf069c2f74e8998d42dea9598db55f333c861b7

Reviewed-on: https://review.gerrithub.io/434146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
3da5847023 build: add ENV_LINKER_ARGS to LINK_C and LINK_CXX
This allows us to remove $(ENV_LINKER_ARGS) from all of the
application Makefiles.

Modify spdk.unittest.mk to explicitly set ENV_LINKER_ARGS
to nothing, to avoid DPDK libraries from getting linked
into unit tests.  This seems better than creating separate
LINK_C rules for unit tests.  Provide a UNIT_TEST_LINK_ENV
flag to allow select unit tests to override this behavior -
it's needed for some of the env unit tests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I647f2c848cf434023cf5a1f3a45869bbc4fc8102
Reviewed-on: https://review.gerrithub.io/434145
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 10:21:45 +00:00
Jim Harris
80bfabcfcf test/unit/iscsi: use test_env.c
Unit tests shouldn't link in DPDK env libraries - they
should all use the test_env.c

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib08ae33224ed9a12fa1ff255c4769a005fc10369

Reviewed-on: https://review.gerrithub.io/434259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
2ac1cbbf07 build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53f0
Reviewed-on: https://review.gerrithub.io/434107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Jim Harris
0b5fa082db build: remove sock from SOCK_MODULES_LIST
The various *_MODULES_LIST are intended for the modules
that implement the specific interface - not for the
associated core library itself.

For example, BLOCKDEV_MODULES_LIST doesn't contain bdev,
it contains all of the bdev modules.  Similar for
COPY_MODULES_LIST and copy.

So make SOCK_MODULES_LIST consistent.  This is a real
pain because we have to add sock explicitly to a bunch
of individual Makefiles, but I have plans to clean that
up in some future patches.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0d293372cfb38bfa46c8478a8d9ca9ce23924747

Reviewed-on: https://review.gerrithub.io/434125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-28 08:13:46 +00:00
Changpeng Liu
2bebd09bd7 vhost/nvme: remove VHOST_USER_NVME_IO_CMD socket message
VHOST_USER_NVME_IO_CMD is designed to deliver NVMe IO command
header to slave target via socket, this can be used in BIOS
which will not enable Shadow Doorbell Buffer feature, since
we enabled the shadow BAR feature to support some old Guest
kernel without Shadow Doorbell Buffer feature, so the message
isn't required, just remove it.

Change-Id: I72e55f11176af2405c8cc09da404a9f4e5e71526
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/420821
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 13:24:42 +00:00
Changpeng Liu
fbc53ae3fb vhost/nvme: add shared BAR space to enable old Guest kernel
For some old Linux Guest kernels, the new NVMe 1.3 feature: shadow
doorbell buffer is not enabled, while here, make a dummy BAR region
inside slave target, when Guest submits a new request, the doorbell
value will be write to the shared memory between Guest and vhost
target, so that the existing vhost target can support both new
Linux Guest kernel(newer than 4.12) and old Guest kernel.

Also, the shared BAR space can be used in future which we can move
ADMIN queue processing into SPDK vhost target, with this feature,
the QEMU driver will become very small and easy for upstreaming.

Change-Id: I9463e9f13421368f43bfe4076facddd119f4552e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/419157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 13:24:42 +00:00
Maciej Szwed
6569a529d6 nvmf: destroy mutex on controller destruction
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I0eb5c7891a8614313607cd006f23e00c75d7d789

Reviewed-on: https://review.gerrithub.io/434818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-27 11:04:53 +00:00
Tomasz Zawadzki
9e2f1bff81 trace: verify tpoint_id does not execeed SPDK_TRACE_MAX_TPOINT_ID
Change-Id: I8b873ba2ebfd8dc4736572206caac11a51ab985e
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434022
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-26 19:37:37 +00:00
Liu Xiaodong
9d2f39ab0b lib/trace: records num-trace-entries for lcores
With the trace_size records for each lcore, spdk_trace
can read trace_file in which each lcore has different
number of trace entries.
Offset of each trace_history from the beginning of this
data structure.

Change-Id: I06afaba129812fe40ed000265fc66b02c5d9e3d9
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433503
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 18:34:18 +00:00
Liu Xiaodong
fd4c75721e lib/trace: set num-trace-entries by app param
Number of trace entries in circular buffer per lcore can be
assigned by the boot parameter of SPDK app with
"--num-trace-entries <NUM>"

Change-Id: I855ce6b4f14a716dcdd9078913da5ea8e577af3a
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433099
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 18:34:18 +00:00
Paul Luse
fa4def5ceb bdev/crypto: code simplification
vbdev registration was happing in a loop in 2 places immediately
following a call to claim(), moved the registration into the
function.

Change-Id: I880dccae02ac0262558119265d2940d0adca33dd
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/433727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-26 18:27:42 +00:00
Paul Luse
3be1436c09 bdev/crypto: optimization: use shared sessions
Initial implmentation had a 1:1 session to crypto op ratio. After
working with a similar concept in CompressDev checked to see if
this was required and apparently it is not.

Saves a decent number of API calls per crypto op and in the poller.
Also saves on mempool usage.  Performance improvement measurement
is WIP.

Change-Id: I73f2355e720a16fd46bc4a02657419f779f07cbb
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/433726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Fiona Trahe <fiona.trahe@intel.com>
2018-11-26 18:27:42 +00:00
Tomasz Zawadzki
6177d220fb bdev/qos: assert io channel when acquiring new reference
Change-Id: Ib546f14be158404ec14067e0f4f1ecb60def0f02
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434321
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-26 18:19:26 +00:00
Jim Harris
50656e7a07 test: modprobe nbd at the start of autotest
This helps avoid issues with nbd not being ready when
we need it.

Still keep various "modprobe nbd" in different test
files - this ensures these test files can still be
run in isolation.  But remove the various "rmmod nbd"
to ensure that nbd always stays loaded.  There is no
harm in just keeping nbd loaded all the time.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3597ff1f7ac53162c33a0384a33f33776cf6f182

Reviewed-on: https://review.gerrithub.io/434838
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-26 18:17:28 +00:00
WangHaiLiang
2f5feaf5b1 doc: update a payload example about rpc_client
Add 'id' description in jsonrpc_proxy.html
Fix issue #512.

Change-Id: Ifa4ee35ea21f70106e79910bccd63d8ed316248c
Signed-off-by: WangHaiLiang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/434294
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-26 16:52:13 +00:00
Maciej Szwed
be0eb272d8 tcp: Initialize mutex only if everything else succeeded
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib0bb6b40852ca4b49d46c2cbeb603b7a2ec4c46f

Reviewed-on: https://review.gerrithub.io/434080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-26 07:06:42 +00:00
Maciej Szwed
5ecc5ef110 nvme: free tqpair on error paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I7abcead55f2eda0a7b8308f81ace70d8a1a59f2d

Reviewed-on: https://review.gerrithub.io/434079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-26 07:06:42 +00:00
Maciej Szwed
dfff013c04 nvme: unlock mutex on failure
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id96bd06c61bc49d9ecbdb16f30804a7dda635771

Reviewed-on: https://review.gerrithub.io/434050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2018-11-26 07:06:42 +00:00
Ziye Yang
c1ce8db0df nvmf: Add the TCP/IP transport related host test
This is the first patch to enable the automated perf
test with TCP transport, which is used to guard the
correctness of the TCP transport related code.

Change-Id: Ic3df3aeae6df2024e22e50be70c7c61e36f85243
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425699
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-11-22 06:59:19 +00:00
Changpeng Liu
b2fb55192c scsi: fix possible memory leak for unmap command
Althrough it has very small chance to be executed, it's nice
to have it fixed.

Change-Id: I899681ccc13ed59c7fdd343ef7791df4e69e490f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433976
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-21 23:38:55 +00:00
Ben Walker
1f20593b87 nbd: Wait for kernel module to become ready on start up
The ioctl NBD_SET_SOCK can return EBUSY if the kernel module
hasn't loaded entirely yet. Wait for it to become ready.

Change-Id: If3c6d0d8bb678ef8cab0efc1c5e800e95e19133e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433939
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Ben Walker
bb005da50a nbd: Split spdk_nbd_start into two parts
This is going to need to wait for the kernel module
to load in the future, so split the function in
half.

Change-Id: I872d000acd4fc25737d5f50c8e0ae33641a6d7fa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433938
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Ben Walker
2fe6d1d5d4 nbd: Make spdk_nbd_start asynchronous
Add a callback to spdk_nbd_start so that it can complete
asynchronously. As of this patch, it always calls the
callback immediately.

Change-Id: I6156fb203145362afa5e4102183b6cf143051c0c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433937
Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Jim Harris
027a8e4b54 test/iscsi: remove reference to auth.conf in calsoft json
Also remove the discovery auth group from the json file,
and use RPC to set it instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I597b60a7c7e4de9d7e1719b0145ab2b11c6f26b0

Reviewed-on: https://review.gerrithub.io/434265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-21 20:09:07 +00:00
Darek Stojaczyk
d808a62db5 pci: introduce a global hotplug lock
Despite the scary commit title, this patch just unifies
per-driver mutexes into a single pci mutex.

On each hotplug we modify some DPDK global resources,
which per-driver locks aren't sufficient for. If
multiple threads try to attach devices at the same time,
then we'll likely have a data race. DPDK hotplug APIs
don't provide any kind of thread safety on their own.

Change-Id: I89cca9fea04ecf576ec5854c662bae1d3712b3fb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433864
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:25 +00:00
Darek Stojaczyk
9cf7d886af pci: do not manually unmap resources if probe fails
We need to do it only for DPDK 16.11, which leaks the
mappings otherwise. DPDK was fixed in version 17.02 with
the following commit:

e84ad157 (pci: unmap resources if probe fails)

Unmapping the resources twice doesn't actually cause
us any trouble, but prints an ambiguous error message.

Change-Id: I8b62e86d5fff8fe924dbf9ae2e37cff29298d412
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433863
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:04 +00:00
Pawel Wodkowski
bf9dbae56b tests: wait for each start and stop nbd disk RPC
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.

Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 07:28:31 +00:00
Maciej Szwed
b86e1be001 vhost: Move mutex initialization after possible failure paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I428027e13f2426684cff536430423d2e4953e29b
Reviewed-on: https://review.gerrithub.io/433824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:54:17 +00:00
Tomasz Zawadzki
65724c3f88 nvme/tcp: destroy controller when failing to create qpair
Change-Id: I5e7d16e377c03165f338709a71d6e4f03beffc0a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:51:08 +00:00
Ben Walker
7f6068f04d test/blobfs: Simplify threading in sync unit test
This will make future changes to the threading model
easier.

Change-Id: I4abfe0850648a0f7449804066d4a02823393e4a6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433680
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-21 00:59:08 +00:00
Changpeng Liu
e8b6523280 iscsi: check the length of initiator/target name
According to the RFC3720, the length of initiator name and
target name should not larger than 223 bytes, while here,
add a check for initiator/target name.

Change-Id: I1517a4c4e1b0a944b239665f38f5dfb46f0075d2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433797
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-21 00:55:55 +00:00
Andrey Kuzmin
af034b6de4 bdev: unregister bdevs top-down during shutdown.
There are some use cases such as multipath and RAID expansion where a
vbdev could have been registered before one of its base bdevs.

Currently we unregister bdevs at shutdown in reverse order of their
registration.  Continue to do that in general, but skip any bdev that
is still claimed.  Any bdevs skipped in this way will eventually be
unregistered once any bdevs that have claimed it have completed
unregistration.

Change-Id: Iafde9558430bc5ce56e8608ef50bcb2b5fbfbf71
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/432136
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 22:49:23 +00:00
Ben Walker
bc7b547368 nbd: Add a function to get the nbd path
Change-Id: I7d121e8c6985a1b355c130248963008da0c63ac7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 22:48:50 +00:00
Ziye Yang
c42bee0cb6 nvmf/host/perf.sh: refactor the code into a common function
Purpose: prepare to add the TCP transport test.

Change-Id: I5dd2acabb9259b7a85919ab2dca41eacf9dee913
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/433964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-20 22:41:53 +00:00
Tomasz Zawadzki
a7e049e722 app/trace: do not print shm_name when using file_name
Change-Id: Ibaeac2d85f3e29df00f64d3e5c1d17d663e32278
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-20 22:40:39 +00:00
Jim Harris
880a1426f2 Revert "build: link all static libs with --whole-archive"
There was an implicit merge conflict between this patch
and e956be96e which added NVMe/TCP support including adding
"sock" as an nvme.libtest.mk depdendency.

This reverts commit 7962ce312d5afeac1fee903bed4708cd150646b7.

Change-Id: Ie7e58e89128f28080a986e12cd014de7df4752e5
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/434103
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 20:33:27 +00:00
Ben Walker
66e620d8c5 test/blobfs: Simplify threading in unit test
This will make the conversion to the new threading library
easier later on.

Change-Id: Ie7358e89128f28080a986e12cd014de7df4752e4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 15:13:54 +00:00
Ben Walker
b49efdf4ae thread: Add unit test for channel destruction behavior
Change-Id: I8a78a4ad242d1c7fc1af2706d318852315d938ef
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-20 15:13:54 +00:00
Darek Stojaczyk
8ee5ca50e7 nvme_pcie: re-probe the pci bus on hotplug event
Explicitly attaching a PCI device with spdk_pci_device_attach()
bypasses any kind of blacklists and should be only used
on a user request. Hotplug uevent is certainly not a user
request and should respect the blacklist, hence it's now
changed to call spdk_pci_enumerate() to probe new devices.
The enumeration callback will reject devices other that the
one we got hotplug request for, so no behavior is changed
in that matter.

This patch also fixes undefined behavior caused by reading
unitialized struct nvme_pcie_enum_cb;

Change-Id: I1399fbdd426152a13ed75c85a52bc7f0491ce287
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:05:01 +00:00
Tomasz Zawadzki
63bafd40a7 reactor: handle failure when getting reactor in spdk_reactor_get_tsc_stats()
Change-Id: I465bb4a20ccc6c67fb2bb68b6c3600094eafff7d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:04:38 +00:00
Tomasz Zawadzki
c49786722a lib/thread: check for NULL return on _get_thread()
In addition UT were changed to set_thread() before registering
io device.

Change-Id: I959dbc800db9c1f50564274a73d71e05e843d8c9
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:03:17 +00:00
Tomasz Zawadzki
6bba6fe9b5 lvol: check spdk_lvol for NULL before dereferencing it
Change-Id: I0f01e2ccde06fa0a5a4ff31f106381bbf3ae6020
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:01:57 +00:00
Jim Harris
f425b98513 ut_mock: rename library from spdk_mock to ut_mock
This matches the lib/ut_mock directory name, and also
avoids the final library being named spdk_spdk_mock.a.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I80ec09568d0455d1f131e45694b6e2d536e6ca91

Reviewed-on: https://review.gerrithub.io/433909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-20 14:57:57 +00:00
Jim Harris
7962ce312d build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53fe

Reviewed-on: https://review.gerrithub.io/432917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 14:57:57 +00:00