5826 Commits

Author SHA1 Message Date
Shuhei Matsumoto
c19ec84378 bdev/nvme: Add multiple namespaces to a single nvme_bdev
This patch removes the critical limitation that ctrlrs which are
aggregated need to have no namespace. After this patch, we can
add multiple namespaces into a single nvme_bdev.

The conditions that such namespaces satisfy are,
- they are in the same NVM subsystem,
- they are in different ctrlrs,
- they are identical.

Additionally, if we add one or more namespaces to an existing
nvme_bdev and there are active nvme_bdev_channels, the corresponding
I/O paths are added to these nvme_bdev_channels.

Even after this patch, ANA state is not utilized in I/O paths yet.
It will be done in the following patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I15db35451e640d4beb99b138a4762243bee0d0f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-14 08:16:32 +00:00
Michal Berger
0403d3736a test/nvmf: Nuke all SPDK net namespaces before starting tests
This is to make sure we won't lose any net interfaces needed for the
phy tests in case they got stuck behind lingering namespace.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic83bc5cca8b536bb3e72fbceb31d2560d840d05e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9795
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-13 07:40:18 +00:00
Monica Kenguva
9221a0d81c test/accel: add UT for test_spdk_accel_submit_copy_crc32c()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I1d70d0fccc7e251777a5568ce869bb47fbaefaca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9504
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-13 07:37:36 +00:00
Monica Kenguva
55ee29ec09 test/accel: add UT for test_spdk_accel_submit_crc32cv()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Iefe925a61accea896e820cae5c401f1ceb8856e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9503
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-13 07:37:36 +00:00
Monica Kenguva
24128a9703 test/accel: add UT for test_spdk_accel_submit_crc32c()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I38d0e301399e0ba1e2196a1a109f36c3c4cd1cd9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9487
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-13 07:37:36 +00:00
Monica Kenguva
fb639b53b0 test/accel: add UT for test_spdk_accel_submit_fill()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I6ecccb015ddc72b3d676d9607bf2ca79aa7435ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9486
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-13 07:37:36 +00:00
Monica Kenguva
b22cd665c9 test/accel: add UT for test_spdk_accel_submit_compare()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: I3287353d76de226f950620f1a4702887d33b1686
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9297
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-13 07:37:36 +00:00
Monica Kenguva
2e736160e1 test/accel: add UT for test_spdk_accel_submit_dualcast()
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: If20750c3671e794b5079972c612743c5ab189862
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9285
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-10-13 07:37:36 +00:00
Jim Harris
355604bfb0 nvme_fuzz: simplify register_ns()
We can just always do the TAILQ_INITs for admin
related TAILQs.  Also no need to set anything to
NULL since the buffer was allocated using calloc.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ifbbda0436e7f33e225cf14dbd103779080b106da
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9729
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-10-13 07:27:14 +00:00
Jim Harris
3ca8b72c9e nvme_compliance: test admin commands with fuse bits set
These commands should always fail.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0abfbe4a4a107957a3271692468fe69eeb823137
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9728
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-13 07:27:14 +00:00
Jim Harris
eb3f756ce4 nvme_compliance: add test for IDENTIFY CNS=1
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I030e64fa8d679d2beebacf657f0c417c422533e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9727
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-13 07:27:14 +00:00
Michal Berger
bec627fcb0 test/common: Supress mem leak triggered during nvme_fio test
This is needed since fio, by default, links to libtcmalloc if found
during the build. See:

01fe773df4

Fixes #2192

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1de8fe9d489f92acb79ef0f169229a6c09ce1179
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9822
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-12 16:22:07 +00:00
Shuhei Matsumoto
5e87727596 scsi: SCSI device supports 256 LUNs at the maximum by default
Most SCSI hosts, Linux, Windows, VMware, supports 256 LUNs per
device now, and it is not easy to test even if any other non-free
OS or driver supports more than 256 LUNs.

Hence increase the macro constant SPDK_SCSI_DEV_MAX_LUN from 64 to
256. Then we do not need to expose it publicly now. So move it to
lib/scsi/scsi_internal.h.

Update the CHANGELOG together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iacde46c3854f326eebfb8befb47d41fce383b027
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9631
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-12 09:10:18 +00:00
Shuhei Matsumoto
4265fc50d9 scsi: Manage LUNs per device by not fixed size array but linked list
Change a fixed size array to a linked list to manage LUNs per SCSI
device.

Keep the linked list sorted by LUN ID because this is necessary to
efficiently find the lowest free LUN ID or check the specified LUN is free.

To avoid traversing the linked list twice, change scsi_dev_find_free_lun()
to return the LUN which comes just before where we want to insert an new LUN.

Additionally, previously spdk_scsi_dev_add_lun_ext() had not checked if
the specified LUN ID was duplicated. Fix the bug in this patch.

Add unit test cases for the function scsi_dev_find_free_lun().

These changes will enable the following patches to increase
SPDK_SCSI_DEV_MAX_LUN from 64 to 256 without consuming additional memory.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I7f6f070ddc680127cf86ae255055da2d1d29e4ec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9630
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-10-12 09:10:18 +00:00
Ben Walker
be6a2feff2 bdev/nvme: bdev_nvme_delete now takes a path_id
Specifying only a transport id is not enough. We need to be able to
describe the host parameters too.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Iadbea553aee4b38e7cacab0b486e7e5746d0d1ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9825
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-12 08:52:01 +00:00
Ben Walker
7d28aae7fb bdev/nvme: Rename connected_trid to active_path_id
This is the currently active path identifier in a failover scenario. The
path is defined by more than just the transport identifier, so fix the
name.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I682c6f4c54f75307e2615bf80e70358180d99fe2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9576
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-12 08:52:01 +00:00
Ben Walker
0262859f3c bdev/nvme: Rename nvme_ctrlr_trid to nvme_path_id
This defines a unique path between a host and a target.

Change-Id: Ia3d24c1b34199a8b596aaf17900ca9694a9da77d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9505
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-10-12 08:52:01 +00:00
Ben Walker
e3c398aa1c bdev/nvme: Don't allow a second identical path for failover
The new path must differ in some way.

Change-Id: I98fd2b2cb3220b482efe0a19bfce94ec4e72bec2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9418
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-10-12 08:52:01 +00:00
Ben Walker
a24d549038 bdev/nvme: bdev_nvme_attach_controller now has a multipath parameter
This parameter may have the values "disable" or "failover". The default
is failover to match existing behavior. In the future we expect to
change the default behavior to disable.

Further, we expect to add an "enable" option soon to do full multipath.

Change-Id: Iebbdc9b95f23101f18d64e085933463498e627be
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9343
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-12 08:52:01 +00:00
Michal Berger
ed763cee62 pkgdep/git: Add missing disable option to refspdk's configure
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I832cc8e2972b081a69a9704886e3c6b37d75847f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9796
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-11 10:10:50 +00:00
Shuhei Matsumoto
845db70ccc scsi: Report LUNs use spdk_scsi_dev_get_first/next_lun() to iterate LUNs
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib812c1f4c180f4173c3b3d668a98e3b23ed32899
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9629
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-11 10:08:22 +00:00
Shuhei Matsumoto
153373a5e7 iscsi: Pass iscsi_lun directly to the callback argument of spdk_scsi_lun_open()
By this change, we will not need to traverse LUN list or tree in the
callback to hot remove.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibe72fba824553d0189b9120884aa2113599a568d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9627
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-11 10:08:22 +00:00
Shuhei Matsumoto
594f46d7a9 iscsi: iSCSI target uses spdk_scsi_dev_get_first/next_lun() to iterate LUNs
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I99cc770c3637d79689d61f996ae40bb4be25cb00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9626
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-11 10:08:22 +00:00
Shuhei Matsumoto
f61d81e47c scsi: Add spdk_scsi_dev_get_first/next_lun() to traverse all LUNs
Add two public APIs spdk_scsi_dev_get_first_lun() and
spdk_scsi_dev_get_next_lun() to remove the dependency on the macro
constant SPDK_SCSI_DEV_MAX_LUN from lib/iscsi and lib/vhost.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6546697f823fe9f4fa34e1161f5c7fa912dd2d59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9608
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-10-11 10:08:22 +00:00
Changpeng Liu
266ef92284 test/vfio-user: enable delete_io_sq_twice test case
Change-Id: I9ba43f2c961b5590b472671b400163cd20199f28
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9678
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-11 01:42:33 +00:00
Changpeng Liu
2dc55b19e8 test/vfio-user: enable NVMe compliance tests with vfio-user
Also add "-g" parameter to use single segment file.

Change-Id: If12c75ccdf1e6074951819fb94cc85a982f9d843
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9677
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-11 01:42:33 +00:00
Michal Berger
19ad1783e4 pkgdep/git: cherry-pick instead of curl'ing the SPDK patches
We clone entire repo and then checkout into a proper tag so there's no
need to extra fetch the patches since they are already part of the
repo.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I38155a33506e373c129a02442b77bc48def23121
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9755
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-08 07:26:39 +00:00
Michal Berger
86f428c922 pkgdep/git: Disable fio in the refspdk builds
This allows us to remove the fio patch needed for the current LTS.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If6a73669842179ced94f857557e6bf594aaddfb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9784
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-08 07:26:39 +00:00
Michal Berger
4f9ef50850 test/nvme: Simplify hotplug test
Instead of relaying on vhost VM image, which may be incompatible with
the host system, perform the hotplug test locally against detecte
nvme device.

Also, limit this test to Linux since it never was intended to run
on other platforms in the CI due to qemu dependency.

Fixes #2150

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie048343ceb52c1a45193a6c857bc3a09fa1ec3fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9461
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-07 09:36:00 +00:00
paul luse
78ef7cfc23 test/accel: add UT for test_spdk_accel_submit_copy()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2fb94a20bda23e9da07db5405977603dfaa319f9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6473
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-07 09:25:20 +00:00
paul luse
bcd3ed39e7 modules/crypto: remove dependency on rte_cryptodev_pmd.h
Call rte_cryptodev_close() to free qpair memory instead of using
an internal function.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I1bd7f0dd86de83f278f6be3263cdf3fbd8e1c77f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9720
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-07 09:23:55 +00:00
Jim Harris
99fb67803c test: allow user to override autotest_common.sh HUGEMEM
By default on Linux, we hardcode HUGEMEM=4096.  But some
environments (like VMs) may not have enough memory to
allocate that much hugepage memory, and the user may
want to run some test scripts that don't require that
much memory.

So allow user to specify HUGEMEM in their environment
before calling a script that uses autotest_common.sh.
If it's set, it will use the specified value,
otherwise default to the values used prior to this
patch.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6e441ed2d703a04430305ad712903a7fa41190c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9726
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-06 11:17:21 +00:00
Jim Harris
93351b0e01 test: use single core when testing nvme fuzz with nvmf target
There's really no need to specify 4 cores for both the fuzz app
and the nvmf target, since only one is ever used.  Reducing the
mask to a single core each makes the fuzz.sh tests easier to run
in VMs with a limited number of cores.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id956ac8d0d7f75cb0e76b78c9175b4aa520218bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9725
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-06 11:17:21 +00:00
Jim Harris
a29c5a164f nvme_fuzz: add -U option to skip IOMMU check
There are many cases where we can safely run the nvme_fuzz
app without having to worry about DMA corruptions - for
example, any test using the TCP/RDMA/vfio-user transports
against a target using an emulated backend like null or
malloc.  So add a -U option to skip the IOMMU check if
the user so desires.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia123b7fb49056f49e2d805c9c3d5b3169c0d589e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9724
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-06 11:17:21 +00:00
Jim Harris
c42d8a16df nvme_fuzz: use spdk_nvme_connect() API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7e2bd33ba4a70000493a2f8f1c67ede784dd29cd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9723
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-06 11:17:21 +00:00
Michal Berger
65f01122b2 pkgdep/git: Bump fio to 3.28
Older versions of fio depend on raw driver which recently got removed
from the linux kernel:

git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=603e4922f1c

Due to that, these fio versions won't build anymore under latest
kernels available, e.g., for fedora (due to missing raw.h from the
kernel-headers package). Details here:

https://github.com/axboe/fio/commit/382975557

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie63769395c482f1965c6664e8f5143c71ef80253
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9718
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-06 07:21:43 +00:00
Denis Nagorny
d42045b2af bdevperf: test time wrong time fix
spdk_bdevperf_shutdown_cb computed total time after the
bdevperf_test_done call. It's wrong. Let's get
g_shutdown_tsc delta when it is really needed.

Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Change-Id: Ic4421305b16277af874008e293da8cb121c0244a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9598
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-04 15:02:46 +00:00
Konrad Sztyber
00d587140f bdev/nvme: remove CSTS from driver-specific JSON info
It's not providing a lot of value, while being pretty problematic, as
the read is blocking and cannot be easily changed to be non-blocking, as
dump_info_json is a synchronous interface.

Now that dump_info_json isn't using any synchronous interfaces from the
NVMe driver, we can send a bdev_get_bdevs call in the async_init.sh
test to verify that.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ida31c8d1000a52b0782f698afe46b071ed4e41df
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9488
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-04 15:00:35 +00:00
Konrad Sztyber
c5ebb7ff99 bdev/nvme: use asynchronous ctrlr detach functions
This patch replaces the synchronous `spdk_nvme_detach()` calls with its
asynchronous counterparts in the controller unregister path.

An additional poller is introduced to periodically poll the NVMe driver
for detach completion.  Once the detach is completed, the poller is
unregistered and the nvme_ctrlr is destroyed.  The poller uses the same
period (1ms) as the async probe poller.

Since reset and detach cannot happen at the same time, reset_poller was
renamed to reset_detach_poller and it can now store the pointer either
to the reset or detach poller, depending on the circumstances.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5eb2dd6383d98d25d1f9748af08c1a13d18acb0e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8729
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-04 15:00:35 +00:00
Konrad Sztyber
51b018da0f test/nvmf: add asynchronous NVMe initialization test
The test verifies that the NVMe driver initialization is asynchronous.
There are some error logs during shutdown, as the detach path isn't
asynchronous yet, but as long as the detach is performed implicitly,
the test completes successfully.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I97483d8a8ba2487e279f7674514ad2cc2c54d504
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8625
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-04 15:00:35 +00:00
Changpeng Liu
742ae4ec72 nvmf/vfio-user: check SQ doorbell is valid or not before use
According to the specification, we should also post an AER
error event for this error case.

Fix #2171.

Change-Id: Ifb2343453ea5e36ce244938a939537ee6ed1c4e1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9584
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-09-30 08:08:05 +00:00
Denis Nagorny
7098e2ed31 bdevperf: prevent multiple performance statistics poller registering
Current implementation allows to run bdevperf with --wait-for-rpc and
without -z option. As the result several statistics pollers will be
registered. Let's add guardian to prevent this.
In the case of multiple test running guardian also will be helpful.

Signed-off-by: Denis Nagorny <denisn@nvidia.com>
Change-Id: I885f9dba6fa324fe4fbaba104d3058bf9f686c19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9613
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-09-29 08:58:12 +00:00
Jim Harris
1227342cd4 test/nvme: add nvme_compliance test app
This is a rough first cut at an nvme_compliance test
application.  Initial focus will be on testing
the SPDK vfio-user target, but over time many of the
tests here will apply to TCP and RDMA as well (or
even running the test against a virtual nvme device
backed by vfio-user in a VM).

Usage:

test/nvme/compliance/nvme_compliance -r <trid>

There is also a test/nvme/compliance/compliance.sh
script that starts the vfio-user target with a single
malloc namespace, and runs the nvme_compliance
app against that target.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2c19df07c9ad93a69420545ab825015f49957fc2

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: John Levon <levon@movementarian.org>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-09-28 07:30:28 +00:00
Ben Walker
4fb3fea387 bdev/nvme: In bdev_nvme_attach_controllers, if adding a path the subnqn
must match.

Change-Id: I4e19ab8668268b273580d2b23ec30eafe21001ef
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9342
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-09-28 07:29:50 +00:00
Ben Walker
8dfefec509 bdev/nvme: Improve error reporting when adding additional paths to a
controller

Change-Id: I53ac0c6f8879bf80bc1345ef620a215d434f536f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9340
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-09-28 07:29:50 +00:00
Ben Walker
bf9e3156a9 test/nvmf: Add detach_controller calls to multicontroller.sh
This test will expand with more ways to add second paths, so we need to
remove the ones we add right after they're added.

Change-Id: Ie95ba01176cdedd858a98306f876d2d253a1c8cc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9454
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2021-09-28 07:29:50 +00:00
Ben Walker
14739d6e13 bdev/nvme: bdev_nvme_detach_controller is now much more flexible
It can match by any provided parameter to remove paths.

Change-Id: I5e7a87342bbb90943dc97fb52f142814fcf0acfa
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9453
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-09-28 07:29:50 +00:00
Ben Walker
a91079fd2d bdev/nvme: connected_trid is now an nvme_ctrlr_trid
Instead of storing an spdk_nvme_transport_id, store the object that
contains it. This will make a few later patches easier.

Change-Id: I36b74889fe39af3b7ab2b900fb3ea4b3f39e1f83
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9484
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-09-28 07:29:50 +00:00
Jim Harris
bcff088852 scheduler/dynamic: don't adjust tsc too much for very busy cores
If a core has a very high busy percentage, we should
not assume that moving a thread will gain that
thread's busy tsc as newly idle cycles for the
current core.

So if the current core's percentage is above
SCHEDULER_CORE_BUSY (95%), do not adjust the
current core's busy/idle tsc when moving a thread
off of it.  If moving the thread does actually
result in some newly idle tsc, it will get adjusted
next scheduling period.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I26a0282cd8f8e821809289b80c979cf94335353d

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9581
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-09-28 07:29:03 +00:00
Jim Harris
f1acee8f83 scheduler_dynamic: fix busy tsc accounting
For the src thread, add the busy_tsc of the thread
we are moving to the idle_tsc of the current core.
This is consistent with how are accounting for the
cycles in the target core too.

We will disable the load_balancing.sh script for now.
We will reenable it later in this patch set once
a few other changes are made, along with some updates
to the load_balancing.sh script based on the changes
made in this patch set.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8af82610804e97dabf62ccd90f75a0e6e37d276f

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9550
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-09-28 07:29:03 +00:00