Commit Graph

16949 Commits

Author SHA1 Message Date
Konrad Sztyber
dcbba659a6 bdev: move the locks inside bdev_name_del
Additionally, added an unsafe version, bedv_name_del_unsafe, which can
be used while already holding the mutex.  It'll make it easier to remove
an alias without taking a lock.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If986909cc52c9b9bdb6f429654b01b83b08b1ea3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11392
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-02-07 21:11:10 +00:00
Karol Latecki
8175ef2944 pkgdep/git: bump up nvme-cli version to 1.16
spdk_nvme_cli_cuse.sh fails with:

```
NVMe status: DATA_XFER_ERROR:
Error while trying to transfer the data or metadata(0x2004)
```

when running test with v1.15 version. v1.16 works just fine.

Fixes #2361

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic058d652d1a8f7adc05fefbad58a5e24d31f2cc8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11408
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-07 17:44:18 +00:00
Jim Harris
c5d80a8bc3 blob: avoid recursion when split IO immmediately complete
In some scenarios, a split IO can immediately complete.  For
example, a very large unmap operation to a newly thin-provisioned
blob has no operations to perform, so the batch for its operation
immediately completes.

But if it immediately completes, we can't recursively submit
the next split IO.  So use variables in the context structure
to detect when an operation immediately completes, to allow
it to unwind and submit the next operation without recursing.

Fixes issue #2347.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8e4c121190c7d08152aa8de20cf6abc55b5edc46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11388
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-02-07 17:40:07 +00:00
Jim Harris
b6992a90d3 blob: add do/while (false) to blob_request_submit_op_split_next
No functional change here, this only prepares this function for
some functional changes in the next patch.  By adding the
do/while loop here we reduce the amount of whitespace changes
in the next patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I09d64fd1fb69ee232af1d298619c762e562fdc79
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11387
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-02-07 17:40:07 +00:00
Karol Latecki
d4b8944881 test/nvmf: enable fio verify_backlog option
Using time_based tasks with do_verify=1 and rw=write
results in verification not being run. Basically
writes consume all run time and there's no time
left for verify:

fio: verification read phase will never start
because write phase uses all of runtime

Using verify_backlog we ask fio to verify after every
N blocks written.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4c1571f4fad893e47cc5fea2ad6b4e2a7d8cc5ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11376
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-02-04 21:58:18 +00:00
Tomasz Zawadzki
9dda3d60b9 lib/vhost: separate out rte_vhost code from vhost
This patch separates out rte_vhost code responsible for
vhost init/fini and vdev register/unregister from vhost.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie69ecd3b2659c805c9c0b0a0076996ef85c8fe71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9535
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-02-04 21:57:42 +00:00
Karol Latecki
19d1b5f965 pkgdep/git: reduce irdma flush delay before building
Reducing this value causes "nvme disconnect" operations
to return much quicker.
Without this change disconnecting from Kernel Target takes
about 40-50 seconds, from SPDK Target about 2 minutes.

Fixes #1748

Change-Id: Ib74d89735b4345cbeaf172190356e2c63f168fd0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11309
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-04 21:06:34 +00:00
Michal Berger
639f6c3a60 pkgdep/git: Update available version of the IRDMA and ICE driver
Change-Id: Id7e4b41dd09426dc46f0b58eea6ee7309c1fa917
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10934
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2022-02-04 21:06:34 +00:00
Mao Jiang
aa221ca1f8 test/nvmf/rdma: cases for creating rdma resources
Change-Id: I2e1d464c7bd76fdd49f673c0c5863ac17372c768
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8460
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-02-04 20:58:56 +00:00
Shuhei Matsumoto
2a99dbc792 nvmf: Support concurrent execution of multiple abort commands
We did not have any practical limitation to support concurrent
execution of multiple abort commands.

NVMe specification recommends that implementations support a minimum
of four abort commands.

Let's follow the NVMe specification.

As stated in the head, we do not have any limitation, and we do not
have to check if abort commands exceeds ACL or not.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I31e066fadcb5d619d0c50c895c4cb64520b33513
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11232
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-04 20:58:29 +00:00
Tomasz Zawadzki
6b9a980795 autobuild: skip doxygen 1.8.20 false positives
Doxygen 1.8.20 contains false positives that are fixed
in later version. Right now if that version of doxygen is used
in tests, any error is ignored. Please see:
(63330f6)autobuild: Ignore warnings comming from doxygen 1.8.20

The false positive is realted to:
https://github.com/doxygen/doxygen/issues/7948

This patch ignores only warnings releated to "\ilinebr"
processing in doxygen, reporting any other warnings.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I19595f35a2e6f66b9917133a19cf45394a240dbd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10388
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-04 20:58:00 +00:00
Richael Zhuang
f36c033c71 uring: fix bug when inserting sock into pending_recv list
There is io error when running NVMe over TCP fio test with uring
socket. It's easy to reproduce the bug with the following
configuration:
target 1 core, 16NVMe SSD, 2 initiators each connects to 8 NVMe
namespaces, each runs fio with numjobs=3.

For if in each round, we inset the sock to the head of the
pending_recv list, and then get max_events socks from head of the
list to process, there is possibility that some socks are always
not processed.
Although there was a strategy to cycle the pending_recv list to make
sure we poll things not in the same order. Such as a list: A B C D E F,
if max_events is 3, then this strategy makes the list is rearranged to
D E F A B C. But it will make this strategy not effective if using
TAILQ_INSERT_HEAD(&group->pending_recv, sock...).

Using TAILQ_INSERT_TAIL(&group->pending_recv, sock...) can fix it.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I8429b8eee29a9f9f820ad291d1b65ce2c2be22ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-02-04 20:57:53 +00:00
Mike Gerdts
5eb363cf8c bdev_ut: UB due to small buffer
bdev_multi_allocation tries to write four characters, an integer between
0 and INT_MAX, and a nul byte into 10 characters. That requires at least
15 characters.

This leads to build failures with "make CONFIG_DEBUG=n CONFIG_UBSAN=y".

Change-Id: I8cb9fd4ede31ae24809e4a04fd60a67dae3a0ac4
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-03 20:08:35 +00:00
Mike Gerdts
7caa514f3c blob: blobcli should dump XATTR_INTERNAL
Refactor the code that dumps XATTR into a function.  Call this function
for XATTR and XATTR_INTERNAL.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic0cb32b14f7a34e030a48e1ea468ec63172e2bf1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11256
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
ab2aad8553 blobcli: include bdev name in error message
When multiple bdevs are accessible by blobcli, it may be helpful to know
which bdev is giving the "Unsupported bdev event" error message.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic55eacad13336366de3d6eac8bde932decbb2732
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
3453391baf blobcli: support for deleting blobs
Add a new option, -w for "whack", to delete a blob.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic7650d9cb6d2aa1cdfa020ad6389c497a481409c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11254
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
ff4fd58b95 blobcli: add forced recovery via blobcli
In the event that the blobstore superblock is marked clean but there are
indications that it wasn't, `blobcli -R` may be used to force blobstore
recovery.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ibc631b6dc0576f81fa1e0b71bd0c18244823e4c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11353
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-02-03 20:07:58 +00:00
Mike Gerdts
a6c5feb0a2 blob: add forced recovery
Add the ability to open a blobstore in such a way that recovery happens
even if the superblock says it is clean.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I475e51beff24428d387446f7785e025294d2f014
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11253
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-03 20:07:58 +00:00
Jim Harris
893ad78814 blob: clarify that the blob is open in the iter_cb
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I543c3c2d54967884d704fc80214f0a1d6aa2db60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11355
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-02-03 19:43:35 +00:00
Jim Harris
af1c4f7ffb lvol: finish initial iteration even if blob has no name
If we find a blob during the initial iteration during load
that doesn't have a name, we cannot just immediately
unload the blobstore, since the 'bad' blob is still
open.  Instead finish the iteration, and unload the
blobstore (with failure status) after the iteration
is complete.

This is somewhat related to issue #1831. By ensuring
we can unload the blobstore, it closes the open
descriptor on the underlying bdev, which allows
the bdev subsystem to exit on application shutdown.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7ecd189842704bb809f25c60efa8f81dcf8ca79c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11352
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-02-03 19:43:35 +00:00
Mike Gerdts
878bec9d73 doc/blob: replace htmlonly graphic with ASCII art
The htmlonly element that uses javascript to draw a diagram hinders
readability of the document in typical terminal-based editors, on
github/gitlab, etc. Not even the official documentation at
https://spdk.io/doc/blob.html renders it, at least in part because
Two.js is not present.

ASCII art can easily convey the same information in a much more
accessible way.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5779d4639b1bee8a976d915d6853f11b05a35b71
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11269
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-02 08:25:02 +00:00
Mike Gerdts
b66f8df748 blob_ut: bs_opts initialized with wrong size
An spdk_bs_opts structure is sometimes partially initialized due to
using sizeof(opts) (struct spdk_blob_opts, 64 bytes) rather than
sizeof(bs_opts) (struct spdk_bs_opts, 72 bytes).

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Iaaa89bb419f66969d0888f49f8991c35b3dc5ea4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11268
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-02-02 08:25:02 +00:00
Mike Gerdts
9f9c7161c9 bdev_ut: test read-only bdev claim
While not documented as such, spdk_bdev_module_claim_bdev() has always
allowed a bdev that is opened read-only to remain read-only when
claimed. This occurs when NULL is passed in place of an spdk_bdev_desc.

This change updates the function's documentation to match the
implementation and adds a unit test to ensure the current behavior
remains.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ief26de60e4408bfe1aa60b7a4e1d8adf273470b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11267
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-02-02 08:25:02 +00:00
Mike Gerdts
79ba049a6f thread: wrong_thread SEGV when NULL thread passed
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I7c121531eda9473b78c46332d6fdb1c8f71c88b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11266
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-02-02 08:25:02 +00:00
Mike Gerdts
9b72cda8b2 blob: fix spelling, white space, grammar
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I236c8a1c7f1ae4b0afd0d20175a1a2a647dba758
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2022-02-02 08:25:02 +00:00
Mike Gerdts
fae72b3464 blob: add logging for blobstore recovery
When a blobstore is not clean, a message is logged at the notice
level.  As other progress is made, messages are logged at the info
level.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Icfbe375faaa95d5be53864f7eb8a73e1ae7c5d01
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11251
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
d14c3bfb8b blobcli: list help options in consistent order
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I38375670125129d0b728077eb4f7a11b23a9f3bb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11250
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
643385ea0a blobcli: uuid xattrs should be printed as strings
At the time of commit d1d22046df, xattr UUIDs were stored as
strings, not as `struct spdk_uuid`. That continues to be true today.
As such, bs_dump_xattr should be treating UUID XATTR values as strings
rather than as `struct spdk_uuid`.

This fix does verify that the UUID string is a well-formed UUID before
printing it. If it is not well-formed, "? Invalid UUID" is printed
along with the raw bytes.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5b574c1c2c4b4802aae3ba23d32ef58fb6fa7586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
d715c82cc6 blob: print sequence and next while dumping pages
Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I2873633e435560ed1199b141851ba43fffcfe2c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11248
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
148bcefa85 blob: report bit arrays that reference each page
While dumping the blobstore with blobcli, read the super block and bit
arrays.  As each metadata page is dumped, indicate which bit arrays
reference the page.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ie023594343861d0fbf065c270424649ec715d8b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11247
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
8389b65ad4 blobcli: do not read from an zero-length blob
Not all blobs have data: an lvstore's super blob is a common
example. Perform a size check prior to trying to dump the blob's
contents.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic5bc1b11cdda72a36c6a3f4b4de0fd6f1a2948c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11246
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-02-01 20:05:13 +00:00
Mike Gerdts
76a577b082 blob: blobcli should use hex for blob IDs
Blob IDs are sequentially assigned starting at 0x100000000.
When debugging with a small number of blob IDs, it is much
more intuitive to see blob ID 0x100000000 rather than blob
ID 4294967296. If blob IDs are displayed in hex, the things
that parse commands should also accept hex to facilitate
copy and paste.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: Ic71eaaf1987609b4f705d372ced4240650b12684
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11245
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2022-02-01 20:05:13 +00:00
Tomasz Zawadzki
047c067c05 so_ver: increase all major versions
To allow SO_MINOR updates on LTS for the whole year it is supported,
the major version for all components needs to be increased.
This is to prevent scenario where two versions exists with matching
versions, but conflicting ABI.
Ex. Next SPDK release adds an API call increasing the minor version,
then LTS needs just a subset of those additions.

Increasing major so version after LTS, allows the future releases
to update versions as needed. Yet allowing LTS to increase minor
version separately.

Disabled test for increasing SO version without ABI change, as
that is goal of this patch. This check shall be removed with SPDK 22.05
release.

This patch:
- increases SO_VER by 1 for all components
- resets SO_MINOR to 0 for all components
- removes suppressions for ABI tests

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id1a5358882dc496faa5b0b5c9a63b326c378c551
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11361
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-31 15:29:56 +00:00
Michael Haeuptle
7a0c901a4c bdev/nvme: RPCs for adding/removing error injections
Provides RPCs for the qpair error injection APIs to bdev_nvme.
These RPCs are useful in testing NVMeoF/NVMe behavior for various
error scenarios in production.

Signed-off-by: Michael Haeuptle <michael.haeuptle@hpe.com>
Change-Id: I0db7995d7a712d4f8a60e643d564faa6908c3a55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10992
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-31 09:51:07 +00:00
Alexey Marchuk
2ccaf2acfa bdev/nvme: Add transport_ack_timeout to bdev_nvme_set_options RPC
It may take a long time to detect network transport error
when e.g. port is removed on remote target. This timeout
depends on 2 parameters - retry_count and ack_timeout.
bdev_nvme_set_options supports configuration of retry_count
but transport_ack_timeout is missed. Note: this parameter
is used by RDMA transport only.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I7c3090dc8e4078f64d444e2392a9e0a6ecdc31c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11175
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: <tanl12@chinatelecom.cn>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-31 09:44:28 +00:00
Shuhei Matsumoto
cc797456f4 ut: Use unit/lib/json_mock.c for stubs
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I2cd488c17dbc92c381cd956ae0d6f5ca709a24dc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11263
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-31 09:44:28 +00:00
Shuhei Matsumoto
def45b4c07 ut/json_mock: Add stubs for json_write_uint8 and _uint16
Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I260b958e0640f737ab77654fedc8007f92eec325
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11262
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2022-01-31 09:44:28 +00:00
Mike Gerdts
96212d45cc lvol: lvol_get_xattr_value failure undetectable
When an unexpected xattr name is passed to lvol_get_xattr_value(), no
error is returned to the caller. The one caller, blob_set_xattrs() via
the xattrs->get_value callback, makes the reasonable assumption that a
lookup that fails to find a value returns a NULL value.  This updates
lvol_get_xattr_value() to match that expectation.

Signed-off-by: Mike Gerdts <mgerdts@nvidia.com>
Change-Id: I5c7a740f2757e6d8265ba2637afecb729acfcdd4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11326
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-31 09:44:16 +00:00
Jim Harris
120a6c1c2a nvmf: do not encode core number into thread name
The nvmf subsystem cannot control which core its
threads get scheduled on.  Even in the normal, default
case, the app thread has already been scheduled on the
first core, so the first nvmf thread will get
scheduled on the second core, etc.

So instead, always use a 0-based index for the names
of the nvmf threads.

Reported-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8a0f161860b985f36920845de28b39dbae9fdca5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11351
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
2022-01-31 09:43:16 +00:00
Tomasz Zawadzki
236cc3a188 CHANGELOG: add entry for NVMe-oF TCP zero copy
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Idc59faec5d9f1f1ab1a769ad57a07ccd5b5d162d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11342
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-28 12:16:49 +00:00
paul luse
21c84e363d idxd: fix bug with cancelling a batch
The new batching code needs to call the cb_fn for each of the
elements of the batch when a batch that hasn't been submitted
yet needs to be cancelled (due to an error in building it).

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6f94b27dd7c64f756193ec3532de98b644b41d7e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11212
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-28 08:08:55 +00:00
paul luse
d5e63730ae idxd: fix busy handling
In several functions. Busy handling also maans paying attention
to the rc when submitting a batch and not clearing chan->batch
unless the call was a success.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic45b10ade2ebdcd845dc33e54dd9c93068ceb98c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11221
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-28 08:08:55 +00:00
Tomasz Zawadzki
234a9ad9e2 CHANGELOG: add missing entries
This patch add entries that should be here,
based on changes in public headers and RPC.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I144ea9a5b66ba136abe29dc0f6c003b93fd059bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11316
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-28 08:08:08 +00:00
Tomasz Zawadzki
405306358f CHANGELOG: fix typos and order of entries
Fix typos, formatting and order of entries.
No new entry is added in this patch.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I69a5ed01eb0d80449b4dae770aeaa70ff86a3be7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11315
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-28 08:08:08 +00:00
Changpeng Liu
82a95325ce nvmf/vfio-user: break bar0 access into 2 functions
With the new added property access API, we can send a internal
property access request to NVMf library, and we can use
it to reset controller.

Change-Id: Iee8b1146d9eb31bc98a9b297e5c635e43e6fdb12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-27 20:55:16 +00:00
Changpeng Liu
d1c2027d01 nvmf/vfio-user: add NVMe live migration support finally
VFIO in QEMU uses region 9 as the PCI passthrough devices' migration channel.

The format of the region 9 migration region is as follows:
 ------------------------------------------------------------------
|vfio_device_migration_info|    data section                      |
 ------------------------------------------------------------------
QEMU will access vfio_device_migration_info to controll the migration
process.

For SPDK vfio-user target, we also implement the BAR9 via libvfio-user,
and we also define the NVMe device specific migration data stored in
data section of BAR9.  QEMU doesn't care about the format in data section,
it will help us to gather the NVMe specific migration data in source VM and
then restore the migration date to data section of BAR9 in destination VM.

The core idea to implement live migration will following the device state
change which is controlled by QEMU.  First QEMU will try to STOP the device
in the source VM, and set the destination VM to RESUME state, SPDK will save
NVMe devic state data structure to BAR9 in the source VM once the subsystem
is paused, then QEMU will read BAR9 in source VM and restore the content of
BAR9 in destination VM, finally in the destination VM, we will restore the
NVMe device state include BARs/PCI CFG/queue pairs in the destination VM.

Change-Id: I42e38f28c3ff59831be63290038b50d199d06658
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-27 20:55:16 +00:00
Tomasz Zawadzki
7d97822f17 vagrant: preallocate nvme images by default
Added preallocation option for raw NVMe disk images.
It matches qemu-img option for raw images.

By default raw images are created with "off" option,
resulting in increased size of the disk image as it is used.

Using "falloc" option preallocates the full size of the disk,
but does not write zeroes to it, compared to "full" option.

This will make sure that in CI, a device used during test
is never exhaused due to lack of host space.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie1c30909dc87552a6d6c385d0ad80905ce68ff9d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11243
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-27 19:36:38 +00:00
paul luse
057fdedbff idxd: zero out descriptor before use
To avoid re-use of descriptors that may have fields set that are
reserved by the one being used now.  For example:

If a batch desc is being built and was previously used by a copy
we need to clear out the dst_addr field or things will explode
as this is a reserved field for a batch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6ba50b76589e38a276683291f5ec2970c80e8aa8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11308
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-27 19:00:22 +00:00
Evgeniy Kochetov
08f9b40113 bdev/nvme: Fix namespace comparison
This patch aligns namespace comparison with Linux kernel
implementation:
- UUID is optional and may be NULL
- command set (CSI) should be the same

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I8f889989f24cd51b104057217f87eb303b30fa68
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11312
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-27 18:53:41 +00:00
Alexey Marchuk
29d94b7f01 nvmf: Stop all listeners when destroying transport
When subsystem is destroyed, it removes its listeners,
however transport level listeners remain active.
This patch removes all transport listerners when
the transport is being destroyed.

Fixes issue 2353/

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ica7bcb0052b626aa62d0da9049bb8f216027dc49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11307
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2022-01-27 15:11:58 +00:00