Commit Graph

16898 Commits

Author SHA1 Message Date
Jim Harris
f2bf7e9727 bdev/nvme: connect to discovered controllers
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3b05ab3d22851d433e3d0573e65943c4a30b9aa4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10695
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
2022-01-12 08:20:23 +00:00
Konrad Sztyber
2bb8a83e0a bdev/malloc: complete requests through poller
Requests that are completed immediately (i.e. those not using the accel
engine) are now queued and their completion is delayed to the completion
poller.  It ensures that they're not completed from the context of a
submission, which gets rid of an spdk_thread_send_msg() call.

It significantly improves performance on some workloads.  For instance,
4k zcopy reads (queue depth 128) on an malloc bdev exposed through
NVMe/TCP went from 204k IOPS to 485k IOPS.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I196f55fc07d167f1ed117d2430e9c37f9d05f70d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10805
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
0f0c16a76a bdev/malloc: remove bdev_malloc_(reset|flush)
The only thing these functions were doing was completing the IO, so it
could just be inlined.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5fbd9df763dd68953b1bda9c7752c57ef9ee5dd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10804
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
0a49fbd241 bdev/malloc: completion poller
This poller is registered on each IO channel and can be used to schedule
asynchronous completion of a request.  This can be especially useful for
requests that can be completed immediately.  For now, nothing enqueues
the requests to be completed through this poller - this will be changed
in the following patch.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If6b26541907bb46402fc0904216bff74dad57b88
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10803
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
fcd5f60144 bdev/malloc: malloc IO channel
It'll allow the malloc bdev to store per-thread data.  For now, it's
only used to keep the pointer to the accel library's IO channel, more
fields will be added in subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I604a38877ae8d6075b911f5a484d1793d4bc2ddb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10802
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
45ded6b8de nvmf/tcp: tracepoints for zero-copy request states
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I173ccb4febf88f56da6e28e59072619755bfd130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10801
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
43f23e3d15 nvmf/tcp: make request state values explicit
It makes it easier to read the logs, as the state values are printed as
integers.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I70a9e8860401c18e9305a5fc5771df0bc564d337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10800
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
b542979620 test/nvmf: zero-copy abort test
Extended the zcopy test with a step checking aborts.  It's similar to
`test/nvmf/target/abort.sh`, with the difference being that here the
aborted requests are executed using zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia25936b7097a94d278c467ce07c009091ea3a55a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
d1024c4bc6 bdev/delay: zero-copy support
This patch adds support for zero-copy operations in the delay bdev.
They use the same delay values as regular IO operations:
 - (avg|p99)_read_latency for zcopy_start with populate=true,
 - (avg|p99)_write_latency for zcopy_end with commit=true.

All other zcopy operations (e.g. zcopy_start with populate=false) are
not delayed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8b32c1d99f9f2b36b16617122881ea95d02ecc87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10798
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
da2fb51627 test/nvmf: zero-copy test
The test has two phases: first it runs a bdevperf with rw=verify to
check data consistency and then runs bdevperf in the background while
sending RPC requests causing the subsystem to be constanty
paused/resumed.

In-capsule data is set to 0 to make sure all IO requests are using
zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1c1d7eb04714c8506307cb95b6cbc5988c8946a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10797
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
794d47d4c4 nvmf/tcp: zero-copy support
This patch adds support for using zero-copy operations to execute IO
requests in the TCP transport.  Of course, they're only used if the
underlying bdev supports them.  Additionally, only requests with no
in-capsule-data can be executed using this mechanism.

Added several new states to accommodate for the difference in a way
zero-copy is handled.  Also, these flows very depending on the type of a
request (read or write).  It stems from zero-copy semantics: to perform
a write we need to wait for zcopy_end completion, while for reads
zcopy_end can only be submitted once we send all of the requested data
to the host.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie02b494c24bc1acc98557cb4b02e867abf9064e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10796
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
6631c2a8aa nvmf/tcp: initialize zcopy phase in nvmf_tcp_req_get
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ia74148fb36733deaf7b2f833ac0247859311a805
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10794
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
a50a70ecdf nvmf: abort outstanding zcopy reqs in qpair disconnect
Zero-copy requests are kept on the outstanding queue for the whole
duration of the request - from the initial zcopy_start submission to the
completion of zcopy_end.  This means, that there's a period in which a
request doesn't wait for a completion from the bdev layer, but is still
on the oustanding queue (after zcopy_start callback, before zcopy_end
submit).  If a qpair gets disconnected while a request is in this state,
we need to manually force its completion, as otherwise it might hang
indefinitely (e.g. waiting for host data).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I53731b8e363b725efa564ca3c7d89b46f5fb2a24
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10793
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-12 08:20:11 +00:00
Konrad Sztyber
974a32b72e nvmf: resume queued zcopy requests
The zero-copy requests can also be queued when a subsystem is paused, so
we need to properly resume and submit them by using zcopy_start.

Since only requests that haven't received the zero-copy buffer (i.e.
before zcopy_start was called) can be queued, we don't need to bother
with checking zcopy_phase.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ie629688f6961eb2ae05741df496720b91be4d80d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10792
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-12 08:20:11 +00:00
Changpeng Liu
58701cc4c5 app: add missed spdk_env_fini() call when exiting
Change-Id: I5cedf3754b512960808168dc50e36ab55f0c7c7e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10910
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-11 14:19:05 +00:00
Changpeng Liu
9ec9c8b33d examples/nvme: add missed spdk_env_fini() when exiting
Althrough `rte_eal_cleanup` will be called in the destructor
function when exiting the application, it's OK not to call
`spdk_env_fini`, we still add this function call in case
there are cleanup function need to call in future.

Change-Id: I0a3eca44c5ed1d62059796ea8f1977fb19356939
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-11 14:19:05 +00:00
Shuhei Matsumoto
6ac23b3e60 bdev/nvme: Clear I/O path cache if a path whose ns is optimized is restored
If a path whose namespace is optimized is restored, the corresponding
I/O path cache should be cleared and the path should be chosen as the
optimal path.

This bug was found by a system test.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: Ibc3983dbff3418adb090a09df32c2a92a8910d05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/11004
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-10 22:18:46 +00:00
Shuhei Matsumoto
3308bdf1b9 bdev/nvme: Rename functions for a full ctrlr reset sequence
Rename a few functions for a full ctrlr reset sequence to
clarify what we do and make the following patches easier.

Signed-off-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Change-Id: I051e3ab68c3cd77fd6040a2d069d50a700123ae6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10920
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-10 22:18:46 +00:00
Shuhei Matsumoto
521a9bb22c bdev/nvme: Fix race between failover and add secondary trid
We sort secondary trids to avoid using disconnected trids for failover.
However the sort had a bug.

This bug was found by running test/nvmf/host/multipath.sh in a loop.

Verify the fix by adding unit test.

Fixes #2300

Signed-off-by: Shuhei Matsumoto <shuheimatsumoto@gmail.com>
Change-Id: I22b0ede4d2ef98b786c3e0d1f5337a2d568ba56d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10921
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-10 22:18:46 +00:00
Changpeng Liu
e57252f98a test/app: add missed spdk_env_fini() or spdk_app_fini() when exiting
Change-Id: I9d511713277ea85d34e0a0e59b570fcbfa014fb1
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10911
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
f2aa5435e9 examples/vmd: add missed spdk_env_fini() call when exiting
Change-Id: If80008cac2a291786984ba3ea06196ef7310271e
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10909
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
abcc84acd8 examples/ioat: add missed spdk_env_fini() call when exiting
Change-Id: I993d41dbec264fef140c83dcd585231c8d958a97
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10908
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
569e2eeda1 examples/idxd: add missed spdk_env_fini() call when exiting
Change-Id: I8993fa796722d65e899218dc9bc4da95c1538e81
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10907
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
50ced20e1c examples/bdev: add missed spdk_env_fini() call when exiting
Change-Id: Icee84d574a9519f6746f899741b553100f357ab0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10906
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Changpeng Liu
974e069878 examples/nvmf: add missed spdk_env_fini() when exiting
Change-Id: Ia6d1b05602541e2884394a7cb242480fcc33eea9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10905
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2022-01-10 18:36:14 +00:00
Jim Harris
b68f2eeb0b bdev_nvme: add bdev_nvme_start_discovery RPC
This patch adds the framework for a discovery
service in the bdev/nvme module.

Users can specify an IP/port of a discovery service.
The bdev/nvme module will connect to a discovery
controller, get the discovery log page, and then
register for AERs.  It will connect to each
subsystem specified in the initial log page.
AER completions will trigger fetching the log
page again, at which point new subsystems will
be connected to, or removed subsystems will be
detached.

This patch does the following:
* Adds the new start_discovery RPC
* Connects to the discovery controller
* Gets the discovery log page
* Registers for AERs
* Detach from discovery controllers at shutdown

Subsequent patches in this series will:
* Connect to subsystems listed in discovery log page
* Detach from subsystems that were listed in earlier
  discovery log pages but subsequently removed
* Add a stop_discovery RPC

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I54bfa896a48c5619676f156b5ea9f2d1f886c72f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10694
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
2022-01-10 15:23:39 +00:00
Jim Harris
9035986e35 test: add iptables entry to forward packets across veth bridge
It seems like some distros or kernels do not automatically
forward packets across the bridge we set up in nvmf_veth_init.
So add an iptables entry to explicitly add a forwarding rule.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I50bc203afcfee888c27c0e1d77609e554dedc61e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10810
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2022-01-10 15:23:39 +00:00
Liu Xiaodong
ba32ee2b29 vhost: set timeout for session's stop_poller
If there is still some inflight IO which prevents
vhost_session_stop_done(), stop_poller can try
within 4 seconds, and then call vhost_session_stop_done
with -ETIMEDOUT.

This can avoid endless blocking in ctrl pthread if there
is no response from vhost session or its backend bdev.
Then spdk vhost target can still serve all other vhost
devices and operations besides the error one.

Change-Id: I2fc78b4da926c936a2e42dc0e66ce1c60001330d
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10393
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-07 15:24:12 +00:00
Konrad Sztyber
d39038e1b6 nvmf: check for zcopy_start failure in request_complete
It ensures that we decrement io_outstanding counter for requests for
which zcopy_start failed.  Also, removed a note stating that such
requests are reverted to regular IO path, as this is not the case.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I8eaee88abfd94b73614b367fef9bb938c9962617
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10791
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
7a374fbc0b nvmf: make zcopy_end void
Since spdk_bdev_zcopy_end() cannot really fail (it only fails if we pass
a bad bdev_io), we can simplify the nvmf zcopy_end functions by making
them void and always expect asynchronous completion.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I6e88ac28aba13acadea88489ac0dd20d1f52f999
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10790
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
92d7df1f47 nvmf: use spdk_nvmf_request_exec to submit zcopy_start
Since this path now supports sending zero-copy, use it for zcopy_start.
Additionally, it makes it possible make zcopy_start void, as it reports all errors
asynchronously via request_complete(), and remove some of the duplicated
error checks.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I41f43ce1651432d9a7d74e3680d4a3f780128a1d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
d88fa8c11e nvmf: complete resubmitted requests
If a request gets resubmitted and is completed immediately (i.e. the
processing function returns SPDK_NVMF_REQUEST_EXEC_STATUS_COMPLETE), the
upper layer needs to be notified via spdk_nvmf_request_complete().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I236d6e68d1d7d83afdaa30d8bb07e2b133f43155
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10788
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
686b9984b3 nvmf: return async/complete status in bdev zcopy operations
Additionally, the NVMe completion status is now updated and the IOs are
queued if the bdev layer doesn't have enough IO descriptors.  It makes
the zcopy operations behave similarly to the other IO operations.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I455ae781e32aa6e60d144d2c91f109bd8be46664
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10787
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
0e09df57dd nvmf: rename zcopy operations to zcopy_(start|end)
It makes their names consistent with the bdev API.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I314051f0980b46959d6560aa25885f13b4c28f2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10786
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
71cbc22960 nvmf: execute zcopy requests through process_io_cmd
The zcopy_start requests are now executed through
nvmf_ctrlr_process_io_cmd.  It makes the zero-copy share checks with the
regular IO path.

Note, that zcopy_end doesn't utilize this path and is directly submitted
to the bdev layer, as it doesn't need to perform these checks (they were
already verified in the accompanying zcopy_start).

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ic9ffecb15e7c351a9d60e731cc711d6500b845db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10785
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
272231686f nvmf: don't check req for zcopy when adding to outstanding queue
This will allow the zero-copy requests to share more code with the
regular IO path.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Idb57447694903a56d984f95aa2f1a3bddc3e4e82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10784
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
f65099d378 nvmf: remove zcopy check in spdk_nvmf_request_exec
It will make it possible to submit zero-copy requests through
spdk_nvmf_request_exec().

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Ibc14fe77cd477b11ed55d1350a7486caaad81add
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10783
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
7d23ac8657 nvmf: remove zcopy phase checks from IO functions
The code should never reach these functions for requests using
zero-copy.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If9f30e05a43b340a982604d5b985242d63ce252b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10782
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
594ccf5a0f nvmf: rename spdk_nvmf_using_zcopy to spdk_nvmf_request_using_zcopy
It's more descriptive that way, as it's clear the function works on a
single request. Also, passing a request instead of zcopy_phase makes it
more convient to use.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: If4d7b087511e128f3590ac7b3b5adcb8ace12003
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10781
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:53:42 +00:00
Konrad Sztyber
aa1d039836 nvmf: zero-copy enable flag in transport opts
It makes it possible for the user to specify whether a transport should
try to use zero-copy to execute requests when possible.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I40a92b0d7a6707f4c9292795f380846acb227200
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:53:42 +00:00
Krzysztof Karas
39b7e10fd4 hw_hotplug.sh: fix the incorrect path to log.txt file
This test fails due to incorrect path to log.txt file needed
to detect start of hotplug application.

The change that introduced the bug:
Change-Id: Ia14507a282796ad28f067c086b9112ae6463b654

This change corrects the file path.

Change-Id: I5d43c47044f46100fa4ca7b845973c427cae237c
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10976
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 18:52:31 +00:00
Yifan Bian
89f253dd20 test/llvm_nvme_fuzz: add DELETE I/O COMPLETION QUEUE and DELETE I/O SUBMISSION QUEUE command test cases
Change-Id: Idf20d03077ea24d347170ada955dda6d92efce49
Signed-off-by: Yifan Bian <yifan.bian@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10984
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 18:52:05 +00:00
wanghailiangx
8e9bf1815d lib/log: move some log level function to log.c
These set_level/get_level functions and REGISTER(log),
have nothing to do with log_flag.c, why we put them there.
I think we might as well put them to log.c.
And "define MAX_TMPBUF 1024", repeated.

Change-Id: I5ade71b923d61446a5f81f0d2f26fdc4a3057f02
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10923
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 16:05:53 +00:00
Jim Harris
63ff27f910 check_format.sh: handle spdk_ functions that are moved
If a function prefixed with spdk_ was moved between
files, the check_format.sh naming convention checks
would fail.  This is because it thinks the function
was added, but doesn't see it getting added to the
header file, since the header file wasn't touched
by the commit.

So resolve this by doing the defined/removed checks
on a per-library basis, rather than per-file.  The
checks already handled the case where functions
were moved within a file, and that will all work
the same now that we check on a per-lib basis.

Fixes issue #2307.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If85a1e9c3cd349b701a10531726e814b60fba26d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10967
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 16:05:53 +00:00
Jim Harris
8b81801e2d env_dpdk: do rte_eal_cleanup() at process exit
We already use destructor functions in env_dpdk to do
some cleanup at process exit, so let's also add one
to call rte_eal_cleanup.  This ensures all hugepage
files are freed before the process exits.

Fixes issue #2267.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8c93f503d77f35717b3d18a63ea49b31789dbc00
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10983
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-06 01:28:11 +00:00
Changpeng Liu
1e46373984 nvmf/vfio-user: use static CNTLID
Change-Id: I5c52f296bfea21968604d31f8e8ee3490317b440
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10150
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2022-01-06 01:20:32 +00:00
Changpeng Liu
2a6c2c289c nvmf: support static CNTLID
SPDK NVMf subsystem supports dynamic controller model, for
transports other fabrics, users should use static controller
model.

Change-Id: I364ea61a71b04d51932fd9e0e16f401a383ff67c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10149
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2022-01-06 01:20:32 +00:00
Jim Harris
76159db955 bpf: use map in nvmf_path.bt
Modify nvmf_path.bt to collect the (ip, port)
tuples in a map, and then print the map every
second. This generates a lot less output than
printing the path for every single I/O.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8f655ed7a4558735f51ddf1169134fbf6950cc1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10616
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <smatsumoto@nvidia.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2022-01-05 21:44:24 +00:00
Maciej Wawryk
1b3d4cecb3 scripts: update local.conf for openstack tests
Default configuration need to be changed due to Openstack network
service change from OVS to OVN. Previous configuration is no longer
valid with new network service

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I39f77a3ac75c35f1551c0097b8344e654b2029dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10959
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2022-01-05 12:10:07 +00:00
Michal Berger
e0482c503e autopackage: Move extra RPM tests to nightly
This is done to save some time in per-patch testing

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I777e30ed90a9cd02ec12c0fd0c19a5616c491f6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8637
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2022-01-05 09:24:59 +00:00