Commit Graph

6876 Commits

Author SHA1 Message Date
Ben Walker
2fe6d1d5d4 nbd: Make spdk_nbd_start asynchronous
Add a callback to spdk_nbd_start so that it can complete
asynchronously. As of this patch, it always calls the
callback immediately.

Change-Id: I6156fb203145362afa5e4102183b6cf143051c0c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433937
Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 21:12:06 +00:00
Jim Harris
027a8e4b54 test/iscsi: remove reference to auth.conf in calsoft json
Also remove the discovery auth group from the json file,
and use RPC to set it instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I597b60a7c7e4de9d7e1719b0145ab2b11c6f26b0

Reviewed-on: https://review.gerrithub.io/434265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-21 20:09:07 +00:00
Darek Stojaczyk
d808a62db5 pci: introduce a global hotplug lock
Despite the scary commit title, this patch just unifies
per-driver mutexes into a single pci mutex.

On each hotplug we modify some DPDK global resources,
which per-driver locks aren't sufficient for. If
multiple threads try to attach devices at the same time,
then we'll likely have a data race. DPDK hotplug APIs
don't provide any kind of thread safety on their own.

Change-Id: I89cca9fea04ecf576ec5854c662bae1d3712b3fb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433864
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:25 +00:00
Darek Stojaczyk
9cf7d886af pci: do not manually unmap resources if probe fails
We need to do it only for DPDK 16.11, which leaks the
mappings otherwise. DPDK was fixed in version 17.02 with
the following commit:

e84ad157 (pci: unmap resources if probe fails)

Unmapping the resources twice doesn't actually cause
us any trouble, but prints an ambiguous error message.

Change-Id: I8b62e86d5fff8fe924dbf9ae2e37cff29298d412
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433863
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-21 14:17:04 +00:00
Pawel Wodkowski
bf9dbae56b tests: wait for each start and stop nbd disk RPC
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.

Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 07:28:31 +00:00
Maciej Szwed
b86e1be001 vhost: Move mutex initialization after possible failure paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I428027e13f2426684cff536430423d2e4953e29b
Reviewed-on: https://review.gerrithub.io/433824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:54:17 +00:00
Tomasz Zawadzki
65724c3f88 nvme/tcp: destroy controller when failing to create qpair
Change-Id: I5e7d16e377c03165f338709a71d6e4f03beffc0a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:51:08 +00:00
Ben Walker
7f6068f04d test/blobfs: Simplify threading in sync unit test
This will make future changes to the threading model
easier.

Change-Id: I4abfe0850648a0f7449804066d4a02823393e4a6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433680
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-21 00:59:08 +00:00
Changpeng Liu
e8b6523280 iscsi: check the length of initiator/target name
According to the RFC3720, the length of initiator name and
target name should not larger than 223 bytes, while here,
add a check for initiator/target name.

Change-Id: I1517a4c4e1b0a944b239665f38f5dfb46f0075d2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433797
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-21 00:55:55 +00:00
Andrey Kuzmin
af034b6de4 bdev: unregister bdevs top-down during shutdown.
There are some use cases such as multipath and RAID expansion where a
vbdev could have been registered before one of its base bdevs.

Currently we unregister bdevs at shutdown in reverse order of their
registration.  Continue to do that in general, but skip any bdev that
is still claimed.  Any bdevs skipped in this way will eventually be
unregistered once any bdevs that have claimed it have completed
unregistration.

Change-Id: Iafde9558430bc5ce56e8608ef50bcb2b5fbfbf71
Signed-off-by: Andrey Kuzmin <akuzmin@jetstreamsoft.com>
Reviewed-on: https://review.gerrithub.io/432136
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 22:49:23 +00:00
Ben Walker
bc7b547368 nbd: Add a function to get the nbd path
Change-Id: I7d121e8c6985a1b355c130248963008da0c63ac7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 22:48:50 +00:00
Ziye Yang
c42bee0cb6 nvmf/host/perf.sh: refactor the code into a common function
Purpose: prepare to add the TCP transport test.

Change-Id: I5dd2acabb9259b7a85919ab2dca41eacf9dee913
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/433964
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-20 22:41:53 +00:00
Tomasz Zawadzki
a7e049e722 app/trace: do not print shm_name when using file_name
Change-Id: Ibaeac2d85f3e29df00f64d3e5c1d17d663e32278
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-20 22:40:39 +00:00
Jim Harris
880a1426f2 Revert "build: link all static libs with --whole-archive"
There was an implicit merge conflict between this patch
and e956be96e which added NVMe/TCP support including adding
"sock" as an nvme.libtest.mk depdendency.

This reverts commit 7962ce312d.

Change-Id: Ie7e58e89128f28080a986e12cd014de7df4752e5
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/434103
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-20 20:33:27 +00:00
Ben Walker
66e620d8c5 test/blobfs: Simplify threading in unit test
This will make the conversion to the new threading library
easier later on.

Change-Id: Ie7358e89128f28080a986e12cd014de7df4752e4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 15:13:54 +00:00
Ben Walker
b49efdf4ae thread: Add unit test for channel destruction behavior
Change-Id: I8a78a4ad242d1c7fc1af2706d318852315d938ef
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-20 15:13:54 +00:00
Darek Stojaczyk
8ee5ca50e7 nvme_pcie: re-probe the pci bus on hotplug event
Explicitly attaching a PCI device with spdk_pci_device_attach()
bypasses any kind of blacklists and should be only used
on a user request. Hotplug uevent is certainly not a user
request and should respect the blacklist, hence it's now
changed to call spdk_pci_enumerate() to probe new devices.
The enumeration callback will reject devices other that the
one we got hotplug request for, so no behavior is changed
in that matter.

This patch also fixes undefined behavior caused by reading
unitialized struct nvme_pcie_enum_cb;

Change-Id: I1399fbdd426152a13ed75c85a52bc7f0491ce287
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:05:01 +00:00
Tomasz Zawadzki
63bafd40a7 reactor: handle failure when getting reactor in spdk_reactor_get_tsc_stats()
Change-Id: I465bb4a20ccc6c67fb2bb68b6c3600094eafff7d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:04:38 +00:00
Tomasz Zawadzki
c49786722a lib/thread: check for NULL return on _get_thread()
In addition UT were changed to set_thread() before registering
io device.

Change-Id: I959dbc800db9c1f50564274a73d71e05e843d8c9
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433611
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:03:17 +00:00
Tomasz Zawadzki
6bba6fe9b5 lvol: check spdk_lvol for NULL before dereferencing it
Change-Id: I0f01e2ccde06fa0a5a4ff31f106381bbf3ae6020
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:01:57 +00:00
Jim Harris
f425b98513 ut_mock: rename library from spdk_mock to ut_mock
This matches the lib/ut_mock directory name, and also
avoids the final library being named spdk_spdk_mock.a.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I80ec09568d0455d1f131e45694b6e2d536e6ca91

Reviewed-on: https://review.gerrithub.io/433909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-20 14:57:57 +00:00
Jim Harris
7962ce312d build: link all static libs with --whole-archive
SPDK uses constructor functions for registration functionality
in several different areas - RPCs, bdev modules, event tracing,
etc.  Since these functions are not explicitly referenced, the
linker will remove these constructor functions from the linked
application unless their library is included using the
--whole-archive linker option.

SPDK has tried to minimize the number of libraries linked with
--whole-archive to only those that are necessary.  But there
are still a lot of exceptions that we need to work around.  The
benefit to trying to restrict it is minimal - for example,
an nvmf_tgt binary built before this patch is 4746320 bytes
on my system, but linking all of the libraries with
--whole-archive only increases it to 4784080 bytes - a difference
of less than 1%.

Note that DPDK also links all libraries with --whole-archive by
default.

This will also simplify some upcoming changes around shared
library dependencies - we will be able to blindly use
--whole-archive when linking the shared library as well.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iab0dbbace06a3d15c835491e55683ab4cf9e53fe

Reviewed-on: https://review.gerrithub.io/432917
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 14:57:57 +00:00
Jim Harris
b0281115b5 reduce: clean up a few items found during code review
1) Add #define with comment for the 2-element iov array
2) Account for user passing trailing slash in pm file path

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id5b620ae9d17dd3534e4cc78ddf1ba176f9c7fcd

Reviewed-on: https://review.gerrithub.io/433903
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 14:50:27 +00:00
Jim Harris
202bb34fcc reduce: add struct spdk_reduce_vol_request
These structures will be used as contexts for every
I/O request.  Allocate a bunch of these requests
when a volume is initialized or loaded.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibf4da3081b03dd3950550e4ca1c5a3a50f72fd52

Reviewed-on: https://review.gerrithub.io/433516
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
8489937b2e reduce: add bit_arrays for free chunk maps and backing blocks
This patch only creates and frees the bit arrays.  Later
patches will initialize the bit arrays during load based
on information in the pm file.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id3ba2b27fa7988778df440b3221ce12c80306c66

Reviewed-on: https://review.gerrithub.io/433515
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
ee26b22b4c reduce: simplify init/load cleanup code
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7a724a3c7c78485c292ebcc65c32e84a5a852b86

Reviewed-on: https://review.gerrithub.io/433514
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
922258a6a7 reduce: initialize vol pm pointer members at load
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id3865c873be1ae80c3e27473ef369b1eeeac7a18

Reviewed-on: https://review.gerrithub.io/433513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
80ba4fa879 reduce: store a params struct in spdk_reduce_vol
These parameters will be needed for a number of reasons
after init/load.

The params struct contains the uuid, so the explicit uuid
member in spdk_reduce_vol is no longer needed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ide12a7816e4a236be5c73a5733e76919ad9af103

Reviewed-on: https://review.gerrithub.io/433512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
a03515de80 reduce: define and initialize pm file regions
A pm file consists of:
* volume metadata
* logical map
* chunk maps

Define all three of these in struct spdk_reduce_vol.
Also define -1ULL to denote an empty entry in the logical
map or a chunk map - and initialize the logical map
and chunk maps entirely with this value when initializing
a new compressed volume.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I9aae23c73d2fbbdc72050ab103fe9e686907eb40

Reviewed-on: https://review.gerrithub.io/433490
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
eebdab6137 reduce: add spdk_reduce_vol_load()
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7923d461af8ec8835fc258fc537230eff52f787e

Reviewed-on: https://review.gerrithub.io/433088
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
b8745a8403 reduce: allocate 2 iovs for reduce_init_load_ctx
We will need the extra iov in the upcoming load path,
so we can load the superblock and pmem file path into
separate buffers.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I99cce0172bd772ca93813a002dcce6943e661f35

Reviewed-on: https://review.gerrithub.io/433087
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Jim Harris
d31f2be55f reduce: have library create pmem file
Currently the API requires the caller to open the
pmem file and pass the mapped buffer pointer, length
and pmem flag to libreduce using the spdk_reduce_pm_file
structure.

Let's have the library just do the pmem_map_file() instead.
Users then just pass the path to create the pmem file.  The
library will still use the spdk_reduce_pm_file structure
internally - so move its definition out of the public header
and into reduce.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I81fcbfdfbb3211dca016d6aa422cf2e1ab16d84d

Reviewed-on: https://review.gerrithub.io/432593
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 14:50:27 +00:00
Ben Walker
e1ec5c60ca event: Use a single event pool instead of one per socket
Previously, a pool of events per socket was created for NUMA
locality. However, when passing a message between threads,
the event would be taken from one pool and put back into another,
resulting in imbalances.

At this time, I do not see an efficient way to allocate the events
so that they remain NUMA local. We'll work on that over time.
However, for correctness right now, go down to a single global
pool of events.

Change-Id: I09f8b0c5c928777e8274c53c6dce21b9c346e2a5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/433519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-20 10:39:38 +00:00
Ziye Yang
e956be96eb nvme: Add the NVMe over fabrics TCP/IP transport support
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:

Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).

Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 20:36:05 +00:00
Darek Stojaczyk
04babbea35 vtophys: remove internal device refcount
This was dead code. Registering the same device more than
once has never been possible within a single process.

Change-Id: I04fa2a62cd9f3d7f99ff799ea4504c49a2232da4
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433866
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-19 20:31:16 +00:00
Pawel Wodkowski
4734811dc6 jsonrpc client: move around functions so no forwad declaration is needed
Change-Id: Ie3cdf0fa51c9a373cb7d98e26dfab6a5d45c6005
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/429911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-19 20:30:03 +00:00
Pawel Wodkowski
edf2305dee jsonrpc client: rework connect functions
Rework connect functions for upcoming non-blocking mode. As we are here
make some variables names shorter and more descriptive.


Change-Id: Ifcba24fc16f0931261067f6c2bf64eef450d0ec9
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/429912
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-19 20:30:03 +00:00
Pawel Wodkowski
e6c30818e3 test: don't use bdev_svc application in this test
bdev_svc require system setup (like bdevs, nvme etc.) otherwise it
writes tons of  errors. Instead just start RPC server on separate thread
and connect to it.

Change-Id: I99ee58cbccbf982d70155c8ecf0fd7f860b3b6ab
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/430316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-19 20:30:03 +00:00
Pawel Wodkowski
2f557958d0 jsonrpc: add internal poll handler
As preparation for non blocking mode make the
spdk_jsonrpc_client_send_request functon just queue the request. Then in
spdk_jsonrpc_client_recv_response we actually send it. Without this
change send function will stay blocking.


Change-Id: Ida2290696d78afcb06d84a4538b74e2311043911
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/429910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-19 20:30:03 +00:00
Shuhei Matsumoto
cd209dbbaf test/iscsi_tgt: Port CHAP to JSON RPC in Calsoft test
Use JSON RPC instead of config file to configure CHAP secret.

Change-Id: Iecaa3eed00fd1b0c25d46693c4ebc04973970f6f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 18:17:22 +00:00
Shuhei Matsumoto
78a45b3f45 test/iscsi_tgt: Port running_config test to JSON config and re-enable it.
Change-Id: I7619ea8dc94e76934c8ebb37f4248376feaa23ef
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433200
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-19 18:17:22 +00:00
Shuhei Matsumoto
a66c9ffd50 test/iscsi_tgt: Port config file from .INI to JSON in the FIO test
Port the config file used in the FIO test from .INI to JSON.

Besides, set_bdev_nvme_hotplug RPC cannot be used easily because
the iSCSI target is not the primary process in the FIO test.

The FIO test doesn't use the hotplug feature and hence remove
enabling hotplug simply.

Change-Id: Idf8809aa829b67a7028f6fa4778a7aae1b431bb8
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433223
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-19 18:17:22 +00:00
Tomasz Zawadzki
33c517c57a ioat/perf: handle spdk_mempool_get() failure
Change-Id: I3b3ca9ce1db609bda59e991f84f511747f49eb87
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433825
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 17:59:25 +00:00
Shuhei Matsumoto
ac7821dc82 config_converter: Fix conversion error to etc/spdk/vhost.conf.in
Just return if the [AIO] section is empty.

Change-Id: I90c42afdca3af7969a1e9d500f92820892857127
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-19 17:58:05 +00:00
Shuhei Matsumoto
603993b274 config_converter: Skip the [Ioat] section to avoid conversion error
IOAT copy engine supports JSON RPC but doesn't support JSON config
dump yet. Hence config_converter fails conversion if the source
.INI config file includes the [Ioat] section.

Skip the [Ioat] section for now.

Change-Id: I7331db582917bf01e92f6251c75f2f5a36ab9382
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/433198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-11-19 17:58:05 +00:00
Tomasz Zawadzki
c3c6922941 app/trace: fix app_name dereference on init
In case when only file_name was provided,
app_name was still dereferenced.

Change-Id: Ica948e072ef02a8daadf303b3e2a004640d19000
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/433609
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-19 15:34:26 +00:00
Changpeng Liu
3126157a21 bdev_nvme: set to deactive state when deleting NS bdev
Since the number of NVMe controller's Namespace is a const
variable, so the Namespace block device is a static array,
and is assigned with NSID after initialization, so when
users want to delete a specified block device, we can
just change it into inactive state.

Fix issue #495.

Change-Id: Ie6e33ad59f2375cc2b5099624bfe1ef6918a6264
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/433543
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-16 16:51:07 +00:00
Changpeng Liu
2706cd4238 nvme: add timeout for Admin commands when initialization
Currently there are no timeout mechanism for Admin commands
when initialization, the NVMe driver may enter infinite loop.
While here, add a new parameter to the controller initialization
options, NVMe controller will report an error when timeout
happens during initialization.

Change-Id: Id0c6b6fa15abe5227b486bee95c8e02914b0d358
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-16 15:29:33 +00:00
Seth Howell
e688d1ccf1 nvme_rdma: properly configure and store max_sges
The max_send_sge and max_recv_sge values can be set to any value from
0...dev_attr->max_sge. WHen we actually set the attributes, we will
receive a qpair with values for max_sge greater than or equal to what we
initially set. We need to store the maximum number of SGEs for later use
when constructing work requests.
Previously we have not relied on these values since we assumed that we
would always be able to have more sges than we asked for initially. This
may change as we try to allocate more SGEs to handle splitting buffers
across memory regions.

Change-Id: Ibbeae1908b86baa3a96d9c6cd2051401aaa2197b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/433307
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-16 15:18:41 +00:00
Seth Howell
1180bf8343 rdma: clean up SGE definitions and properly set values
We have historically conflated SPDK_NVMF_MAX_SGL_ENTRIES and the maximum
number of SGEs associated with a wr data object. For now these are the
same thing, but there should be nothing tying the number of NVMe request
SGL elements to the number of rdma request wr sgl elements.
Also, clarify the rx_sge and tx_sge enums to reflect the actual maximum
number of SGEs associated with either the send and receive queues.
This change doesn't actually modify these values, but sets us up to do
things like split the data in an NVMe SGE into multiple WR SGEs in case
the buffer associated with the NVMe SGE is not contained in a single
RDMA mr.
We also need to store these values in the qpair for later usage.

Change-Id: Iff3756fc72787a4b72a99b2bdf90bf486a8010fa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/433196
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-16 15:18:41 +00:00