Commit Graph

13421 Commits

Author SHA1 Message Date
Darek Stojaczyk
494dc66ddb mk: don't link env if not needed
Some SPDK apps were linked with env even though they
didn't use it.

Top-level makefiles can now specify SPDK_NO_LINK_ENV=1.

Change-Id: I057baa5b620f20d829185025dc2e8efdcfa03fac
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3417
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
2020-09-07 09:28:39 +00:00
Michal Berger
3fdfea7bac config/pkgdep: Allow for sourcing OS-based routines
vm_setup.sh currently depends on package managers to determine which
SPDK dependencies must be installed, however, this may cause issues on
systems which provide more than one package manager, e.g., dnf and
yum. Due to that fact, some packages were missing since they were
bound to a specific package manager instead of the distribution, case
and point, nbd. Also, some OS-dependent steps had to be duplicated,
like repo refresh on Centos8, since either of the managers could be
in use.

To address the above, allow for souring of OS-based routines to
overwrite defaults set based on given package manager.

Change-Id: I0d8d88dea4521e9c76b187c69743e4e09aa724e6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4017
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-09-07 09:28:25 +00:00
Michal Berger
f2d958973a pkgdep/yum: Attempt to install wget prior installing nbd rpm
Change-Id: I907ae092d0d46ec6fea89ad24660f6ac6b555c2a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-09-07 09:28:25 +00:00
Michal Berger
8a0d613e6f pkgdep/dnf: Refresh repo on Centos8
The d09acccf changed the order in which dnf, yum managers are picked
up, however, the refresh step was not moved to dnf which actually is
by default shipped with Centos8. Simply add the missing step.

Change-Id: I6f845d794b79ff01d4abfff098b0b29a9c489189
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4015
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-09-07 09:28:25 +00:00
Michal Berger
f1896c3d1c pkgdep/yum: Add missing call to sudo
Change-Id: I7d8b0a457f51ca1214009c758e25c44ab2d6755c
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-09-07 09:28:25 +00:00
Alexey Marchuk
f7b4dd3492 pkgdep: Add a workaround for Centos7.x aarch64
There are 2 issues:
1. building of ninja fails with missed skbuild dependency.
This dependency can be installed with scikit-build module
2. The build of ninja wheel fails due to cmake version. E.g. centos 7.6
uses cmake 2.8 and cmake 3.x can be installed as another package.
It is needed to create a soft link to cmake3 with `cmake` name so
that cmake3 is used instead of cmake 2.8

Change-Id: I53910d8d73b233d3f3a9059ab10938477c5db57b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:28:09 +00:00
Michal Berger
342c08d0b0 scripts/setup: Define fallback driver for each device type
Most distros ship these drivers as modules, however, some, like
clearlinux, have them compiled directly in the kernel (e.g. nvme).

In case modalias lookup fails, have a fallback prepared just in case.

Change-Id: Ib9da8ff69edbb0c4ce427aa089fa7d04b399a20b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4047
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:55 +00:00
Michal Berger
d6c499f3ee scripts/setup: Fix check for mounted devices
The previous approach didn't take into the account systems where root
partition is representend in a virtual form of /dev/root (see
clearlinux). In case the device was bound to virtio this would fool
setup.sh into thinking nothing is mounted and proceed with unbinding
the entire rootfs.

Fix this by checking the maj:min numbers of the device instead of its
name against the mountinfo list.

Change-Id: I0feb2584869f6bb72df3e9a4e619620240cfce3b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4046
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:55 +00:00
Michal Berger
44775a8062 scripts/setup: Add env switch for block dev sync
Make sure that setup.sh waits for block devices during the tests.
This is to make sure that the underlying controllers are ready to
be talked to and minimize the amount of their flakiness.

Change-Id: Id09445de7ac7ccf4c9f679ee2b6c4bdd14c89733
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
0897e4dbfe scripts/setup: Keep track of the driver given device is bound to
Change-Id: I4c3a54d017f2ed26ce1fa5afb539063f73fadbc1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
4f1701f60f scripts: Add tool for keeping uevents in sync
This script tries to be smart by mainly looking up all the kernel
events caught over the netlink. This is done in order to mitigate the
amount of flakiness test scripts may suffer from while waiting for
given device to pop up. This mainly concerns nvme devices when they
are moved back from user space - the time from when the ctrl is bound
to the driver until the block subsystem finally gets the device can be
substantial (even > 2s). Consider:

  $ setup.sh reset; sleep 10s

  vs

  $ setup.sh reset; sync_dev_uevents.sh block/disk nvme0n1

Where the latter could recover some of the seconds back:

  $ time { ./scripts/setup.sh reset; ./scripts/sync_dev_uevents.sh block/disk nvme0n1 ; }
0000:82:00.0 (8086 0953): uio_pci_generic -> nvme
0000:00:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:80:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:00:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:80:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:00:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:80:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:00:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:80:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:00:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:80:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:00:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:80:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:00:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:80:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:00:04.7 (8086 0e27): uio_pci_generic -> ioatdma
0000:80:04.7 (8086 0e27): uio_pci_generic -> ioatdma
* Found nvme0n1

real    0m2.427s
user    0m0.465s
sys     0m0.408s

Change-Id: Ib91efc73742fb00b058d4f54976d27390489ddbf
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3655
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
1c8a890bff scripts/setup: Reorder the status's header
Change-Id: Ifeadc3fac3f5d09143dd13ef3d510de9cecffb8a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4032
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:42 +00:00
Shuhei Matsumoto
aeb693ca07 bdev/gpt: Change ERRLOG to DEBUGLOG to show interleaved metadata is not supported
It is a good enhancement for the GPT vbdev module to support
interleaved metadata but interleaved metadata is not critical and
the priority is low.

However the following error log is always collected when there is
any bdev with interleaved metadata.

  vbdev_gpt.c: 552:vbdev_gpt_examine: *ERROR*: GPT module does not
    support block size 4224 for bdev Null0

This does not cause any failure but confuses user. Hence,  change
the corresponding print from ERRLOG to DEBUGLOG.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id58507cbdc830d2b139fb34faa29e08d73c16141
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4058
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
2020-09-04 15:12:48 +00:00
Shuhei Matsumoto
102ab66926 doc/nvmf: Fix the section "Using RPCs" not to use deprecated --max-qpairs-per-ctrlr
The section "Using RPCs" uses the parameter --max-qpairs-per-ctrlr
but it has been deprecated.

Use max_io_qpairs_per_ctrlr instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iee0646656e5201e0c7b2ee220ea93b674e07fce1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 15:12:48 +00:00
Ben Walker
4aa0bb676b nvme/pci: Check for command completions after deleting submission queue
After a submission queue is deleted, the device is supposed
to post completions for every command to the completion queue.
Previously, we never looked and completed all commands with
an ABORTED status. Instead, complete any commands in the
completion queue with the status the drive gave them.

Change-Id: If851a365d4f305cf4390454b6b26dd0f7c5b82ac
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3875
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-09-04 08:14:44 +00:00
Shuhei Matsumoto
68f1681771 lib/nvmf: Control I/O and some admin commands according to ANA state
For I/O commands, block them if ANA state is inaccessible, persistent
loss, or change.

For Identify command, clear capacity field (nuse) to 0 if ANA state
is inaccessible or persistent loss.

For Get Features command, block features, error recovery, write
atomicity normal, reservation notification mask, and reservation
persistence if ANA state is inaccessible, persistent loss, or change.

For Get Log Page command, error information page does not return
any data yet, and hence there is no change.

For Set Features command, if ANA state is inaccessible or change,
block the command if NSID is 0xFFFFFFFF or if feature is error recovery,
write atomicity normal, reservation notification mask, or reservation
persistence, or if ANA state is persistent loss, block the command.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I15dd593227e451aa2247c53da42b6acad1757907
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4043
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
4688f0819f lib/nvmf: Add ANA state to struct spdk_nvmf_subsystem_listener
Add ANA state to struct spdk_nvmf_subsystem_listener and initialize
it to optimized.

Then ctrlr->listener->ana_state is referred when creating ANA log page.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I978424e51d3f23ca72dee30192bc2693abfe203d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4012
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
5b27db6268 lib/nvmf: Check if I/O qpair is on the same listener as admin qpair
We will have ANA state per listener and per subsystem. On the other hand,
NVMe specification defines ANA state per controller.

However, it is possible that I/O qpair and admin qpair are different
listeners on a single controller.

Let's check if I/O qpair is on the same listener as admin qpair if
ANA reporting is enabled.

The case that I/O qpair is on a different listener from admin qpair
is not usual and so the purpose of this check is just to guard SPDK
from any unexpected behavior.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idb8d255de7f998e45a59a120c2ed5803258873f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4026
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
7bcff376ab lib/nvmf: Add subsystem_listener to struct spdk_nvmf_ctrlr
Find the subsystem listener whose trid matches req->port->trid when
creating a controller, and store it in the controller.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iea343b8d8ae827b554df2245b67aed113469c592
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
e21de9a9cc lib/nvmf: Add trid to struct spdk_nvmf_qpair
Add trid to struct spdk_nvmf_qpair and initialize it at initialization.

admin_qpair->trid will be used to get the corresponding
subsystem_listener via nvmf_subsystem_find_listener() and add it to
struct spdk_nvmf_ctrlr in the next patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0d1a41aede60de88747eff16c7e04f63d0702596
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4009
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Shuhei Matsumoto
58da62239e lib/nvmf: Add nvmf_ctrlr_async_event_ana_change_notice() to notice ANA change
The new function () will be used in the following patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I788cfb38d75c3f1f64e1754912b776a80f0f1be8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4007
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-04 08:14:18 +00:00
Tomasz Zawadzki
ccdbd2ed51 nvme/cuse: fix nblocks and block_size in rw path
nblocks is zero based, so read path was missing the increment.

NVMe device that cuse represents can be of any block_size,
so rather than hardcoding 512 - actually verify it.
Both paths didn't request enough of a buffer from cuse.

Reported-by: Niklas Cassel
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I228dc2572bc94ecbcb913e950d912a7ab5be9434
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4037
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:56 +00:00
Tomasz Zawadzki
ef3f0d97ab nvme/cuse: refactor read/write to pass cuse_device and block_size
This patch does not alter functionality, just moves
around where cuse_device and block_size is determined.

Next patch will fix both paths.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5a827b5b4ab080b2aa0f76f5cdcbcb177b38b474
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4036
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:56 +00:00
Tomasz Zawadzki
3a8c976aef dpdk: update submodule to DPDK 20.08
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6e0de8a0b2ece373aa3338c8e9d48389aa4c5a46
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3865
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:48 +00:00
Tomasz Zawadzki
cff7c8f1f7 test/vhost: workaround issue #1583 by specifying --vq-count
DPDK patch d0fcc38f5
"vhost: improve device readiness notifications"
introduced changes that need to be workedaround when
updating to DPDK 20.08.

This shall be removed after appropriate fix is created
for DPDK.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I2f0b4ef4692ba8d9fbb7d1966332def593d1dc34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4001
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:48 +00:00
Changpeng Liu
d653a65eaf vhost: restart session after KICK socket message
Change-Id: I13fbfcde617a963503a479769374cf8dba5fa278
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4024
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:48 +00:00
Changpeng Liu
7b16ba29fa virtio: use correct number of entries field
Socket message VHOST_USER_SET_VRING_ENABLE will use number of
vring entries as input parameter to indicate the vring is
enabled or not, previously the flag in vhost-user library
wasn't checked before commit d0fcc38f5
"vhost: improve device readiness notifications", so here
we also use correct filed set in SPDK.

Fix issue #1583.

Change-Id: If5ac8a4ba31bdecbb5a64b736346c99e4be0f4b6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3989
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-04 08:13:48 +00:00
Alexey Marchuk
fe87dcadb7 mk: Remove the content of build/lib in "clean" target
Leaving old libraries leads to incorrectly generated libspdk.so -
e.g. when we configure spdk using --with-uring and after that
using --without-uring then libspdk_bdev_uring.so will remain in
build/lib and will become part of spdk.so built using --without-uring

Change-Id: If482e9cf323b58f3f588b7c3bdb629425ae0ceeb
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-03 07:43:38 +00:00
Karol Latecki
103147241b test/nvme_perf: add p90 latency to gathered metrics
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic642b7cc7a26a086dbd6da783caf5ca19a5cd95d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4011
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-03 07:43:35 +00:00
Michal Berger
2dcb0569de pkgdep/git: [QAT] Replace linux/cryptohash.h with crypto/sha.h
The former was dropped and merged with crypto/sha.h in kernel 5.8.

Change-Id: I9005592feacc71c9a0e80f06b5de2d65a6bc06d9
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4030
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-03 07:43:30 +00:00
Michal Berger
d2cbb15c40 scripts/common: Don't cache pci vendor,device ids separately
On its own these values don't uniquely identify a particular pci
device, hence there's no much point in storing them like so.

Change-Id: Icc5cfec28fcccae6cd935dff03fe3ef01826edc4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3950
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-02 14:43:02 +00:00
Michal Berger
904ac49f4c scripts/setup: Configure binding of the controllers in parallel
There are some devices for which nvme driver takes a long time to
finalize the unbind stage. With that in mind, each device would
add up a significant amount of time needed for setup.sh to complete.

To mitigate such a scenario, make sure the controllers are unbound
in a parallel fashion.

Examples taken from the system with 19 nvmes on board:

[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null

real    0m36.250s
user    0m1.024s
sys     0m1.990s

[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null

real    0m4.848s
user    0m0.867s
sys     0m17.605s

Also, take note that this is currently done only for the nvme
devices since other, i.e., ioatdma, seem to trigger a BUG in the
kernel when unbound in parallel. Some details here:

  https://bugzilla.kernel.org/show_bug.cgi?id=209041

Change-Id: Icaeb2b2ecb306f149587bc5da73743b1519bc5d6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 14:43:02 +00:00
Michal Berger
5ea549466e scripts/setup: Commonize the driver binding and BDF verification|status
Change-Id: I8c8f4297891a027c3f4c650bba915d0786f46a47
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3892
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
85501619b4 scripts/setup: Use modalias to verify the bdf's driver
Change-Id: I52afb29b69f758b4cbebf4866099e430953ed681
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3891
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
69ae10f785 scripts/setup: Unify block name lookup
Change-Id: Icde3b528d37f6c9a2d8a2fd9f3548f3ee75ceff0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
51b5fa8579 scripts/setup: Group all devices into respective arrays
This is a first patch from a series attempting to merge similar
routines used throughout setup.sh.

Change-Id: I4c71b88f7556b0e9e2f65a37b8175914022486a4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3889
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-02 07:38:48 +00:00
Seth Howell
58216dd07e lib/nvme: fix mem leak in req submit.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: If64c06177605a8f57d87ba22b86fe58ddebd6f7a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3921
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-09-02 07:38:38 +00:00
GangCao
4803dc36fc Blobfs: add an assert for the memory allocation
Change-Id: Ie9c75206b61242c71ce6f4f408bbdd44f9633077
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3998
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-02 07:38:23 +00:00
Alexey Marchuk
3aaf8e0a0b nvmf: Stop pg poller when all qpairs are destroyed
We stopped the poller to early, so we were not able to
reap all completions on ibv CQ, so RDMA qpair was not freed.
This patch stops the poller when all references to poll group
are released (all qpairs are destroyed)

Fixes #1578

Change-Id: I15c1697db13aef9da7567c7312476306c3ee1d62
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3962
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-01 07:51:45 +00:00
JimboLuCN
dd3d5800b0 script/rpc: fix bdev_set_qos_limit's usage error
Signed-off-by: JimboLuCN <jimbolucn@gmail.com>
Change-Id: I67db5c7ef81ae77edefcc5ecf058826d3be8d2e8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3981
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-01 07:51:38 +00:00
Ziye Yang
03aa8995e9 lib/sock: Fix the coredump issue in sock_map_realese
When tested on Linux 5.8 kernel and configure spdk
with debug mode (--enable-debug), and test SPDK NVMe-oF
tcp transport, and we see the coredump in sock_map_release
with the following statements:
	assert(entry->ref > 0);

After debug, I can confirm that the placement_id value got
from the following function (sock->net_impl->get_placement_id)
changes.
It means that: When the sock is added into the poll group
(spdk_sock_group_add_sock), we get the placement_id (named as
Value(begin)); and when the sock is removed from the poll group
(spdk_sock_group_remove_sock), we get the plaemednt_id on
the same sock (named as Vaule(end)). I found that
Value(begin) ! = Value(end).

So our solution is for a socket, we will get placement_id once,
then we can solve this issue.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ia1d0cf39247b53410260561aca5af38130cc0abb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3983
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-01 07:51:27 +00:00
Tomasz Zawadzki
49601c040f test/check_so_dep: skip rdma dependency if not configured with RDMA
Similar to internal vhost library, rdma lib should be ignored
if SPDK was not compiled using --with-rdma.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic747f96f96182c04592bec44ff64ceafd11634f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3963
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-09-01 07:51:19 +00:00
Michal Berger
e989faa936 scripts/pkgdep: [rhel] Add warning about working under RHEL system
Additionally, fall through potential yum failures to try and install
what's available in repos given system is using.

Change-Id: Icf7f5f3db1d72b67414c7d7e4badb8416a9d25b1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3873
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
1338e698d1 scripts/pkgdep: [rhel] Configure EPEL repo prior installing any packages
For some RHEL releases, especially 8.x, EPEL repo is crucial in order
to install most of the package dependencies, hence we need to make
sure it's in use.

Change-Id: Iaaa1f901e6fffddb65b98fa98b21b9e77e59760f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3872
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
f724f6b9c4 scripts/pgkdep: [rhel] Use python36
On rhel 8.x standard appstream and baseos repos don't provide python
package. Use python36 instead as it's available there.

Change-Id: I125aa3370bdb11f67f2774e00e70196df9680f13
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3871
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
d70ecbe935 scripts/pkgdep: [rhel] Don't use hard link for a python binary
This limitation may simply fail if /usr is part of a separate
mount|fs so softlink it instead.

Change-Id: I1e0387c2792c0bbd20d83e9739b3882a7cbbbf50
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3870
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-09-01 07:50:41 +00:00
Jim Harris
3fe748f728 blob: move bs_release_cluster
We will use it earlier in this file in a future patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I554f2073185d466bd0b4e98bdeec721f763c1b44
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3969
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:33 +00:00
Jim Harris
8ba7450e8c blob: simplify check for available cluster count
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I56a9ca114b84ded4ac23a9eaacadaa6df8f031cc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3968
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:33 +00:00
Jim Harris
f5b949d5fd blob: simplify free cluster counting in inflate path
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5716290476d4abdf8646a36206f07acb5b2dd4ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3967
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:33 +00:00
Jim Harris
2d87587fe5 blob: claim clusters inline during initialization
When claiming clusters as part of blobstore initialization
or recovery, just call spdk_bit_array_set directly rather
than going through the bs_claim_cluster function. We will
be modifying how runtime cluster allocation works so need
to separate the two use cases.  This code is very small so
inlining it has minimal code impact.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iaaa1c817e57b4a2eea62eb4683407364bac1fcc0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3966
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:33 +00:00