Commit Graph

16627 Commits

Author SHA1 Message Date
Ben Walker
cd8c36f2fe util: Add spdk_ioviter for iterating iovecs
spdk_ioviter_next will walk through two iovecs and yield pointers
to common length segments. For example, given a source iovec (siov) with
4 1KiB elements and a destination iovec (diov) with 1 4KiB element, the
following will happen:

first spdk_ioviter_next:

src = siov[0].iov_base
dst = diov[0].iov_base
len = 1KiB

second spdk_ioviter_next:

src = siov[1].iov_base
dst = diov[0].iov_base + 1KiB
len = 1KiB

third spdk_ioviter_next:

src = siov[2].iov_base
dst = diov[0].iov_base + 2KiB
len = 1KiB

fourth spdk_ioviter_next:

src = siov[3].iov_base
dst = diov[0].iov_base + 3KiB
len = 1KiB

fifth spdk_ioviter_next:

len = 0

This is a useful utility for performing operations where both the source
and destination are scattered memory. As an example and a test vehicle,
spdk_iovcpy has been updated to use this internally.

Change-Id: I7e35e76d38e78d07ea1caf6282d0dfc02182aa83
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10284
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-17 09:45:44 +00:00
Jim Harris
b33e68a789 nvme: call probe_cb when directly connecting to discovery ctrlr
The host may have specified a hostnqn to use to connect to
a discovery ctrlr, so we can't just use the default ctrlr
opts to connect - we need to call the probe_cb (if there is
one) to get any options that the host may have specified.

Tested by using discovery_aer tool, creating a subsystem and
listener, and then adding a host on the target side that matches
the hostnqn specified to the discovery_aer tool.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I07266e984e0094d3a768e6a0d5ea3a3bd71e32ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10547
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-17 09:45:17 +00:00
Peng Lian
4c1757ffb9 nvmf: update discovery log when removing hostnqn
In NVMF Revision spec 1.1a, discovery log should be updated
when removing hostnqn of subsystem.

Update unit test to check the discovery log when removing
hostnqn and destroying subsystem.

Signed-off-by: Peng Lian <peng.lian@smartx.com>
Change-Id: I51c597a2493295a677a7aa68e4f13a887f7e1140
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10668
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-16 08:52:20 +00:00
Michal Berger
17a662f9fe test/json_config: Simplify events check
Consider multiple nvme devices loaded to subsystem prior running
the checks.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie4ce8be6ec22c26c1d46ded42ded468b92d6cc57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-12-16 08:49:53 +00:00
Michal Berger
5247e10a06 scripts: Get soft core ulimit from the template
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6794886832ec641ec18b72621f21264970d805d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7118
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-12-16 08:46:14 +00:00
Changpeng Liu
0a3a9f4285 nvmf/vfio-user: fix the compilation issue on kernel 4.9.x
Macro PCI_CFG_SPACE_EXP_SIZE isn't defined in kernel 4.9.x, so
we use a fixed value here instead.

Fix issue #2282.

Change-Id: Ife1444e919e4c1dc9328437002c15befe2f393e7
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10691
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-16 08:45:58 +00:00
Anil Veerabhadrappa
68f0c6160a ut/fc : fix fc_ls_ut compilation failure
This regression was introduced when 'accept' was removed from
spdk_nvmf_transport_ops structure.

Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I5d880791db258a97a1861dbd841e97a7c068ce12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10676
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-16 08:43:39 +00:00
Anil Veerabhadrappa
2344dacab4 nvmf/fc: remove unused variable
Signed-off-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Change-Id: I671afd0d18a74df46acb3c617f5a380c66efef29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10675
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-16 08:43:39 +00:00
Jim Harris
d07dc4d361 nvme/fio: free io_qpairs in close callback
If user uses the 'loops' parameter, then it will
call the open+close callbacks for each loop, but only
cleanup at the end of the entire run. Since we alloc
the io_qpair in open, but don't free them until
cleanup, we end up running out of io qpairs at some
point if we run too many loops.

So solution is to free the io qpairs in the close
callback.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibac864836f94994cdd24a7886894778268b14f73
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10674
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-15 16:45:56 +00:00
Jim Harris
050565e5ab test/nvmf: fuzz nvmf target using LLVM's libFuzzer
LLVM provides libFuzzer which does coverage-guided
fuzzing of a library or application under test.  For
SPDK, we can use this as a new and better way to
generate random commands to the SPDK nvmf target.

By default, libFuzzer provides the main() and your
source file just provides the function called by
LLVM for each iteration of random data.  But this
doesn't really work for SPDK since we need to start
the app framework and the nvmf target.  So we
specify -fsanitizer=fuzzer-no-link, explicitly
specify the location of the fuzzer_no_main library
and then call LLVMFuzzerRunDriver to start the
fuzzing process once we are ready.

Since this is all coverage-guided, we invoke the
fuzzer inside the nvmf target application.  So this
patch creates a new target application called
'llvm_nvme_fuzz'. One core is needed to run the
nvmf target, then we spawn a pthread to run the
fuzzer against it.

Currently there are two fuzzers defined.  Fuzzer 0
does random testing of admin commands.  Fuzzer 1
is focused solely on GET_LOG_PAGE and fuzzes a
smaller subset of the bytes in the spdk_nvme_cmd.

Additional fuzzers can be added in the future for
other commands, testing I/O queues, data payloads,
etc.

You do need to specify CC and CXX when running
configure, as well as specify the location of the
special clang_rt.fuzz_no_main library. The path of
that library is dependent on your clang version and
architecture. If using clang-12 on x86_64 platform,
it will look like:

CC=clang-12 CXX=clang++-12 ./configure --with-fuzzer= \
  /usr/lib/llvm-12/lib/clang/12.0.0/lib/linux/libclang_rt.fuzzer_no_main-x86_64.a

Then just do the following to demonstrate the fuzzer
tool.

make
test/nvmf/target/llvm_nvme_fuzz.sh --time=60 --fuzzer=0

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iee0997501893ac284a3947a1db7a155c5ceb7849
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10038
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-15 04:32:05 +00:00
Changpeng Liu
723adbaf32 UT/vfio-user: fix clang-12 compilation error
Add missed STUBs.

Change-Id: I20989bf4ea66720d62f8ecc9668bb8f74e459666
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-12-15 04:32:05 +00:00
Karol Latecki
654aa4b975 test/blobfs: update rocksdb version
Use one of the recent branches for rocksdb. This allows
to build with less -Wno-error options. Adjust C flags
for more recent GCC versions.

Change-Id: Icf176d909a92195da2ae744a53e0cb53530b4573
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-14 13:23:30 +00:00
Krzysztof Karas
f076b51205 trace: allow to specify multiple tpoint group masks
This change implements mechanism to allow user to
define multiple tpoint masks separeted with a comma
(e.g. 0x400, 0x8).
This is going to be used in the next patch to implement
enabling of individual tracepoints inside a tracepoint group.

Change-Id: I963f89684aa62b6e1dde57e22ddf835aa2c89f05
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10536
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-14 13:18:55 +00:00
Jacek Kalwas
6237da13c0 nvmf: allow to override aer limit by specific transport
it is possible that some specific transport doesn't support
NVMF_MAX_ASYNC_EVENTS (although it is a recommended value by spec)

with that change it is possible to reduce aerl on transport specific
layer so it can be advertised correctly during identify controller

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: Ife6465b5324fb39f9b343c6f42b860e9dd1164b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10422
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-14 13:18:45 +00:00
Jacek Kalwas
43022da379 nvmf: remove accept poller from generic layer
Not every transport requires accept poller - transport specific
layer can have its own policy and way of handling new connection.
APIs to notify generic layer are already in place
 - spdk_nvmf_poll_group_add
 - spdk_nvmf_tgt_new_qpair

Having accept poller removed should simplify interrupt mode impl
in transport specific layer.

Fixes issue #1876

Change-Id: Ia6cac0c2da67a298e88956734c50fb6e6b7521f1
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7268
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-14 13:18:33 +00:00
Krzysztof Karas
bace05499a logs: create a log level flag in abort and hotplug applications
Bloated log files appear during some tests
(nvmf_phy_autotest, during nvmf_abort test fot example),
what makes parsing them inconvenient. This change aims
to disable logs that cause this issue by adding a new flag
for SPDK target and disabling notice level logging completely
(using --silence-noticelog proved insufficient).

Fixes: #2149

Change-Id: Ibbad92d87d90fe73c23d6027e0ff8ec49b0393c2
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10311
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:12:01 +00:00
Krzysztof Karas
eb48f76d61 hotplug.c: add a second switch to parse_args()
Add another level of option parsing to allow usage
of string values.
Currently all values are converted to long
(or have to be checked before the switch),
which results in errors upon adding a string as a value.

This is going to be used in the next patch in the series.

Change-Id: Ib874d74b015eee825b5135ef3d49b9cb1c72029b
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10471
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-14 09:12:01 +00:00
Karol Latecki
1b81d438bb dpdk: update spdk/dpdk 21.08 submodule
Update submodule after recent fixes related to Clang-13.
6af54b79a4
a3c2b3951b
bda789a357

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iee036ed0890d462bf028508d1dd0890b9f9ea9a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10637
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:09:53 +00:00
Michal Berger
ce2a2aed0b pkgdep/git: Encode whitespace in the ice driver's URL
Latest curl versions started to treat such URLs as invalid. See:
https://github.com/curl/curl/pull/7073

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic5ab34a566f89f411ec40cbcb8de57a8d2f3ea88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10607
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:09:38 +00:00
wanghailiangx
df5ad409c2 examples/blobcli: remove a redundant error prompt
The details are as follows:
./build/examples/blobcli -h
Error: -b option is required.
(When we input '-h', this is a redundant error prompt)
......
Commands include:
        -b bdev - name of the block device to use (example: Nvme0n1)
        -d <blobid> filename - dump contents of a blob to a file
        -D - dump metadata contents of an existing blobstore
        -f <blobid> value - fill a blob with a decimal value
        -h - this help screen
        -i - initialize a blobstore
......

Change-Id: Ieccbcc56b02b10504170fb2e60090c25dc0d4b53
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10581
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-12-14 09:09:31 +00:00
Michal Berger
a2815831cc scripts/bash-completion: Use default filename completion for ./configure
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib95e9561299ceed3a37665da1decbfb9fce300dc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10356
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-14 09:09:24 +00:00
Michal Berger
2bc71094b5 configure: Adjust help for bash-completion
bash-completion provides its own _configure() to complete options for
most of the ./configure scripts out there. To make full use of it, our
./configure should provide all --with{,out}-* options as part of the
help output.

Also, add some missing --without-* options to cmdline lookup.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I3498ce906d68422f54e5c6d992acf927bbce787a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-14 09:09:24 +00:00
Mao Jiang
6fbaa11534 nvme/deallocated_value: add vfio-user transport support
Change-Id: I1a87d9245ba8a4f4e01d510cae4a318fa3323ca2
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10257
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-14 09:09:18 +00:00
Rui Chang
5d926c48ea nvmf/vfio-user: Fix doorbell polling not working on some ARM platform
On aarch64 platforms, doorbells update from guest VM may not be seen
on SPDK target side. This is because there is memory type mismatch
situation here. That is on guest VM side, the doorbells are treated as
device memory while on SPDK target side, it is treated as normal
memory. And this situation cause problem on ARM platform.
Refer to "https://developer.arm.com/documentation/102376/0100/
Memory-aliasing-and-mismatched-memory-types". Only using spdk_mb()
cannot fix this. Use "dc civac" to invalidate cache may solve this.

Profiling data did not show big performance degradataion.

Signed-off-by: Rui Chang <rui.chang@arm.com>
Change-Id: I9a18718f8c4307b3007b18c32ab02e6796548958
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10222
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-14 09:09:09 +00:00
Konrad Sztyber
3911bc344b scripts/rpc: option to format framework_get_pci_devices for use with lspci
Added an option, `--format-lspci` that produces output that can be
consumed by `lspci -F`.  Additionally, added a simple convenience script
that executes lspci with the the output of the RPC.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I41a0f846f32506c28cf6ca3a299ed264f64db1a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-14 09:08:59 +00:00
Konrad Sztyber
9647fd4e8d rpc: add method for listing PCI devices
This RPC lists all PCI devices attached to an SPDK application.  Each
device is identified by a BDF and contains a buffer with a copy of its
config space.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I852f421fde105d975458f8e63b8da4f92ed2c69b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10652
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Konrad Sztyber
2c9895dee6 json: add spdk_json_write_bytearray
This function serializes a buffer as a hex string.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I09ab93bc626f6f6543b7c1ef033bcf807050862a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10651
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
2b65309b6c env: remove spdk_pci_get_[first|next]_device
These APIs are not safe, since they do not hold the
pci device lock across calls, which can cause problems
if a device is inserted or removed while handles
returned by these APIs are being used.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I01a80f26d0a0ca4cdfc7181359932b38da8dd43a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10659
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
e123181d71 lsvmd: use spdk_pci_for_each_device API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I1ff5bea00f4b08bf879a3c726689b74f616c1c70
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10658
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
63b41c7237 vmd_led: use spdk_pci_for_each_device API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8046242f05e4ffb676570e65e2217cf593add2e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10657
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
a70c03cc78 spdk_lspci: use spdk_pci_for_each_device
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I68bad0ad9868be7e06da217e028d4db8f37712e8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10656
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
13fbf8851e env: add spdk_pci_for_each_device
This is a safer alternative to spdk_pci_get_first/next_device,
since those APIs do not hold the lock between calls.

Future patches will remove those APIs, and change callers to
use this new API instead.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I71c7e8c1feb9112da8be32a8056b30e105e30463
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10655
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-12-14 09:08:59 +00:00
Jim Harris
59f3cdacb1 nvmf: don't always update discovery log when adding hosts
If a subsystem has no listeners, then there is no need
to update the discovery log when adding a host, or setting
a subsystem to allow all hosts.

This eliminates some unnecessary discovery log update
notifications, especially when setting 'allow any hosts'
on a subsystem immediately after it is created (and before
it has any listeners).

Update unit test to check the adding a host to a
subsystem without listeners does not rev the genctr.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I63dab5df564269e574bb925890088f52063aa378
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10546
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Jim Harris
3867f83dea test/nvmf: add local var for hostnqn string
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia967512bfcc5d7b1df15b6f6b5c132f21d601dce
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Jim Harris
9ac2cf7ff0 nvmf: don't update discovery log on subsystem create/delete
The discovery log isn't updated when a subsystem is created
or deleted, it's only updated when a listener for a
subsystem is added or removed.

So remove the nvmf_update_discovery_log() in the subsystem
create and delete paths. They just generate extra AER
completions that potentially cause the host to do unneeded
work.

Note that if a subsystem is deleted with active listeners,
the subsystem delete path will remove each of the listeners
before deleting the subsystem itself.  So the discovery log
will still get updated when those listeners are removed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id01bbfa3b24d3e1279a614a2fd60be41387a03b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10545
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-10 17:32:18 +00:00
Karol Latecki
17cd6c3488 test/nvmf: increase malloc size in filesystem test
btrfs.tools v5.15.1 in Fedora 35 complains about target
device being too small to create a filesystem.
Let's increase the bdev size to this test.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iedf59328b91516693c88af8c5d47dde849394337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10614
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-10 09:18:26 +00:00
Karol Latecki
62b501dd48 scripts/vagrant: update Fedora vesions.
Update scripts with Fedora 35 and remove older
versions which we no longer use.

Change-Id: I54d9f3bcf76065f3e4e3094411d77153c351afc9
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10612
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-10 09:18:26 +00:00
Michal Berger
91b8cd3791 rpmbuild: Fix cmdline lookup
Fixes issue #2272

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I8845e4cb194c00eaaaa5e2a6c799ca32ba4b9335
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10591
Reviewed-by: <yex.chen@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-09 10:31:25 +00:00
Changpeng Liu
97277e1459 nvme: use transport internal queue state when deleting unfinished IO queue pair
The NVMe bdev module enables asynchronous IO QP creation by default, after
calling `spdk_nvme_ctrlr_alloc_io_qpair` and `spdk_nvme_ctrlr_connect_io_qpair`,
the queue pair is in connecting state at the beginning, then users may call
`spdk_nvme_ctrlr_free_io_qpair` immediately, and the common layer will
change queue state to NVME_QPAIR_DISCONNECTING and NVME_QPAIR_DESTROYING,
so in function `nvme_pcie_ctrlr_delete_io_qpair` the workaround to wait
for create cq/sq callbacks will not be called, instead of using the common
layer queue state here, we should use the internal `pcie_state`.

Fix #2245.

Change-Id: I801caf26563464b135035bf7fa2f63def13de9f4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10445
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-12-09 06:06:02 +00:00
paul luse
fbb24d0ebe lib/accel: remove batching from the framework and plug-in modules
Batching will be made available for DSA specifically through the new
idxd_perf tool.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic51d9ad3692074805b1ffa705cea8be35737c778
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9846
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-08 16:35:40 +00:00
paul luse
998b5d664e examples/accel_perf: remove batching
It's being added to the low level library perf tool which makes
more sense as the accel_fw provides queueing thus allowing any
queue depth.  When using the low level lib you are limited to
a small number reported by HW unless you use batching then you
can get quite a bit more.

Future patches will remove batch from the accel_fw, this is just
the accel_perf tool.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I3775b9ecdf353f730c7f3e2c06bb45a8873e61f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9845
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-08 16:35:40 +00:00
yidong0635
aaff670713 scripts/rpc : Add required notice for TRSVCID.
As we known, there's a bug in argparser, which puts
required descriptions in optional list.
There are many such descriptions, if not to break a
lot of existing scripts, just adding a notice description
behind required item.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I3d8d2db96dc947d8342cc4e04ce3914da64be146
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10580
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-08 08:32:02 +00:00
Karol Latecki
85222826d2 doc: add SPDK NVMe Bdev 21.10 performance report link
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3c65bd5c65885573945301dde1ef1d982e14675e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10551
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-12-08 08:31:49 +00:00
Shuhei Matsumoto
215518069a bdev/nvme: nvme_ctrlr_create() gets prchk_flags from nvme_async_probe_ctx
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3deca8e0aba23299347a6aee6f0f44ee683556e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
619acff501 bdev/nvme: Delete unused nvme_probe_ctx
We set cb_ctx to NULL when calling spdk_nvme_probe_async(). It looks
that nvme_probe_ctx has not been used anywhere for a long time.
nvme_probe_ctx is not public data structure.

Remove nvme_probe_ctx to simplify the code and make the following
patches easier.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7dd5f970a7fde1c9c189fae3c8f28f84d7aed991
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
bf88d1d4a6 bdev/nvme: Factor out the failover trid operation into a helper function
This refactoring will be helpful for the following patches to unify ctrlr
reset and failover and failover trid also when reconnecting ctrlr.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4623a5dd310ac7516c270ccd3b0541c27cc880d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10443
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
ffabc8ac29 bdev/nvme: Inline bdev_nvme_failover_start() into bdev_nvme_failover()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I70593de284f5623db9e30d94b03b6576bd6ca29b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
696ad465d7 bdev/nvme: Remove the failover_in_progress flag from struct nvme_ctrlr
The failover_in_progress flag is used to decide the return value of
bdev_nvme_failover().

bdev_nvme_delete() calls bdev_nvme_failover() with remove=true to remove
nvme_ctrlr->active_path_id. However bdev_nvme_failover() returns zero
if nvme_ctrlr->failover_in_progress is true. bdev_nvme_failover() may
return zero even if it does not remove nvme_ctrlr->active_path_id.

The following will be better.

bdev_nvme_failover() returns -EBUSY if nvme_ctrlr->resetting is true,
and the caller repeats calling bdev_nvme_failover() until the target trid
becomes alternative path or bdev_nvme_failover() returns zero.

To do that, the failover_in_progress flag is not necessary any more.

Removing the failover_in_progress will also simplify the following
patches to unify ctrlr reset and failover.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I57ab944beb1d06ea4def144c81c69705860de35f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10441
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
74f18d6a07 bdev/nvme: Factor out checking if nvme_ctrlr can be unregistered
Checking if nvme_ctrlr can be unregistered is not so simple and
a few changes will be added. So factoring out the check into a
helper function will be valuable.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I310c7e3ad2dae9583df4db575d342c2cb111f3f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10461
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00
Shuhei Matsumoto
7329c1e683 bdev/nvme: Refine and factor out checking if nvme_ctrlr is available or failed
When a I/O or admin passthrough failed, if the corresponding nvme_ctrlr
is not available, we should failover to another path.

When no path was found, if there is at least one nvme_ctrlr which is
not failed, we should wait until it is recovered.

We should improve error recovery not only for multipath (multipath is
"multipath") but also for failover (multipath is omitted or "failover").

To do this easily, clarify the conditions of availability and failure of
nvme_ctrlr and realize them by helper functions.

Use new helper functions for other cases to improve readability too.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I716731f72811d2ec4dfc91f9eadb191d75739af6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10381
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-12-08 08:31:24 +00:00