551 Commits

Author SHA1 Message Date
Evgeniy Kochetov
29f31a90e1 sock: Add sock_impl option to disable zero copy on send
Zero copy send can cause performance degradation with small
payloads. This patch adds an option to disable it if required. By
default zero copy is enabled.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I14f2b21ad375e770cb08f850360898bac675b351
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 00:30:45 +00:00
Evgeniy Kochetov
63c5e51ebc sock: Add sock_impl option to disable receive pipe
Receive pipe reduces number of system calls and gives significant
performance improvement with kernel TCP stack and relatively small IO
sizes. With user space TCP/IP implementations there are no system
calls and double buffering introduced by pipe has negative impact on
performance. Receive pipe remains enabled by default.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ic5ddee42293df2c233ba7ffbe6662de7917ac586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 00:30:45 +00:00
Evgeniy Kochetov
59b5ba5ca9 sock/posix: Add helper macros to get/set fields in sock_impl_opts structure
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I2fe650556edf22e253976dcd4ddf07d649789d11
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-24 00:30:45 +00:00
paul luse
f295f5b31a module/accel/ioat: add support for batching IOAT accelerated ops
Also remove the batching check in accel_perf as all 3 engines
now support it.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ib4cf4b148b50df50a4fc7be9e861cc83f355623a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-23 22:26:39 +00:00
paul luse
be34c31e16 module/accel/ioat: add batching suport for sw operations
This patch has the basic infrastructure to support the accel
framework batching API but only for commands not HW accelerated
by IOAT, that will come in the next patch...

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I6168831ac5698a9e58a81ef35ce919d75a72d0f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3153
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2020-07-23 22:26:39 +00:00
paul luse
8cee297c3f lib/idxd: add batching support for compare command
Also one small bug fix w/compare in accel_perf as a result
of changes made in accel_perf sicne base compare was added.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Id8e67bd9de9cbd006ac148f4a77807cc3e8e662b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2958
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-22 23:01:45 +00:00
paul luse
80da74964a lib/idxd: add support for batching crc32c requests
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I38e8b26272e86e84e579262496b367375976638d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2955
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-22 23:01:45 +00:00
paul luse
396b05a3e4 lib/idxd: add support for batching the fill command
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I0996db55e78b7b74bf0a603ac0e9825a4d80d2e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2953
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-22 23:01:45 +00:00
Sudheer Mogilappagari
2d877401fd nvmf/tcp: Add recv to busy poll hw queue for data
Call recv to trigger busy polling even when no socket is active. when
epoll_wait returns zero,  the first socket in poll group is used to
trigger busy polling in kernel stack and potentially reap incoming data

Change-Id: I15f04cb4a2c7b382dd07391eda69678fd7919790
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-22 12:20:47 +00:00
paul luse
22058d70fc lib/idxd: add support for batching dualcast commands
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia5b5a346ae5c8470d9792766dba2bcf68c6fe2fd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2915
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-22 12:20:27 +00:00
paul luse
93a5e08fc8 module/accel/idxd: remove developer debug mode code
No longer needed.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: If29fd918958aaf9b36f9c749512119cecbd8f92e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2880
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-22 12:20:27 +00:00
Tomasz Zawadzki
78f44434a6 lvol: report io_unit_size as block_size instead of page size
Bdev underneath lvs has a certain value set as block size,
then lvolstore inherits that as `io_unit_size`,
which ends up as block size for the lvol bdev presents.

Before this patch lvs on device with block size of 512,
would create lvol bdev with block size of 512.
Meanwhile reporting in bdev_lvol_get_lvstores(),
that block_size is 4k instead.

This was result of addition of io_unit_size and
forgetting to update the value here. Previously
only I/O of page_size was allowed.

This patch adjusts the value reported to
io_unit_size.

Alternatively it could have been removed,
or name changed to "io_unit_size".
Yet for API compatibility reason this name
remains.

Info on actual page size which is always 4k,
is not useful to bdev users. No need to add
another field for that.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I19701c408c389f64e25c027c7bba789294bbed94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3289
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-17 07:20:14 +00:00
Sven Breuner
0190e71eb6 module/bdev_compress: Fix use logical block size in vbdev_compress_claim
Use logical block size instead of base bdev block size in
vbdev_compress_claim. This is relevant to enable the compress bdev to
present a logical block size that is different from the block size of
the base bdev (e.g. 512B logical on 4KB base bdev).
However, this is just an internal fix/preparation and there is
currently not yet a way for users to specify a different
logical block size, e.g. through an rpc.py parameter.

Signed-off-by: Sven Breuner <sven@excelero.com>
Change-Id: Id7562f0da234ce11242b7ee78837a317ee51c211
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-15 07:09:40 +00:00
Shuhei Matsumoto
47b0d4275c bdev/nvme: Support abort IO by using spdk_nvme_ctrlr_cmd_abort_ext()
Change NVMe bdev module to enable abort as IO type.

Change _bdev_nvme_submit_request() to process abort request when the
IO type is abort.

The current thread tries aborting I/O command in the I/O qpair first.
If no I/O command to abort was found, send message to the thread which
is registered when creating controller. The controller thread tries
aborting admin command in the admin qpair next. If no admin command
to abort was found, complete the abort request with failure.

spdk_nvme_ctrlr_cmd_abort_ext() is used to try aborting command whose
cb_arg matches. qpair is set to NULL when trying to abort admin command.

Before calling spdk_nvme_ctrlr_cmd_abort_ext(), save the current
thread to process admin command completion correctly.

spdk_bdev_abort() supports any bdev module other than NVMe bdev
module and does not check CDW0 but checks only if the completion
status is success or failure. So add bdev_nvme_abort_done() and
converts the NVMe completion status to the bdev completion status.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If6aebae0ba2f6c5834ee926e161af9c4d825f341
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
613c9386ff bdev/nvme: Add thread pointer to struct nvme_bdev_ctrlr
Add thread pointer to struct nvme_bdev_ctrlr. The thread which
created nvme_bdev_ctrlr is set to the pointer.

The thread pointer will be used to limit only one thread to submit
admin abort.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia39d5cbc7a13b0e0022c0d5591069ea8776ef774
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3244
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Shuhei Matsumoto
9d6bb6decc bdev/nvme: Use SPDK_POLLER_REGISTER to use function name as poller name
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I30da42413501d03433ef4550327675471185df3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3243
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-14 07:54:42 +00:00
Evgeniy Kochetov
35429c9b5d sock: Save socket subsystem configuration in JSON format
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I32c25e6410c418ffa00a76559aa7b6999e2269ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/617
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:15 +00:00
paul luse
fc250841ca idxd: add batch capability to accel framework and IDXD back-end
This patch only includes the basic framework for batching and the
ability to batch one type of command, copy. Follow-on patches will
add the ability to batch other commands and include an example of
how to do so via the accel perf tool.  SW engine support for batching
will also come in a future patch. Documentation will also be coming.

Batching allows the application to submit a list of independent
descriptors to DSA with one single "batch" descriptor. This is beneficial
when the application is in a position to have several operations ready
at once; batching saves the overhead of submitting each one separately.

The way batching works in SPDK is as follows:

1) The app gets a handle to a new batch with spdk_accel_batch_create()
2) The app uses that handle to prepare a command to be included in the
batch. For copy the command is spdk_accel_batch_prep_copy(). The
app many continue to prep commands for the batch up to the max via
calling spdk_accel_batch_get_max()
3) The app then submits the batch with spdk_accel_batch_submit()
4) The callback provided for each command in the batch will be called as
they complete, the callback provided to the batch submit itself will be
called then the entire batch is done.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I4102e9291fe59a245cedde6888f42a923b6dbafd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2248
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-10 07:31:09 +00:00
paul luse
cf83cac493 module/accel/idxd: change name of global lock and expand scope
Previous lock protected against a channel counter, renamed as it
now protects multiple values associated with configuration and
reconfiguration.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icfe48aa7df192a7bdf568de418d49c7984d20acf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3175
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-10 07:31:09 +00:00
Ziye Yang
f096f252d0 uring/sock: remove the SPDK_UNREACHABLE
For some exceptional cases (e.g.,
https://github.com/spdk/spdk/issues/1486),
we may detect POLLERR or other events. So for those events,
we can just ingore it, but not use SPDK_UNREACHABLE.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I073575408783ff75e50b40d45ddf09388a2cab96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3262
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-10 07:30:45 +00:00
Seth Howell
f4d0ab7111 Revert "bdev/nvme: add a function for specifying a multipath trid."
This reverts commit 5f13e0f67f38f224f1fb59de27fc47a8dda5f038.

Reverting this change to allow for a secondary review.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iad208603034bd892f3f248836459005d3c2b5297
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3041
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:11 +00:00
Seth Howell
0344ba4064 Revert "bdev/nvme: add failover option to bdev_nvme_reset"
This reverts commit 62ded32fd7d3977abdb065e5f14345364843f2a8.

Reverting this change to allow for a secondary review.

Change-Id: I85a059c434f6e634e014e680ae228e95fffdee92
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3040
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:11 +00:00
Evgeniy Kochetov
37bf49b1d5 sock: Add dependency on socket subsystem to other subsystems
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I665760166d54f6156b54735015bd2b586e91c345
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/616
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-08 07:56:46 +00:00
Evgeniy Kochetov
0f657329d3 event/sock: Add socket subsystem
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: If62b0d0dfb105854b306601f9913bd287ecdc283
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/613
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-08 07:56:46 +00:00
Maciej Szwed
b1bb9917a9 bdev/delay: Return poller status based on completions number
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I54d4a41602f678aed4d2df61d7059adc1ae37f78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3235
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-08 07:54:38 +00:00
Maciej Szwed
eb05cbd677 pollers: Fix pollers to return correct busy status
Poller should return status > 0 when it did some work
(CPU was used for some time) marking its call as busy
CPU time.

Active pollers should return BUSY status only if they
did any meangful work besides checking some conditions
(e.g. processing requests, do some complicated operations).

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id4636a0997489b129cecfe785592cc97b50992ba
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2164
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-07 07:29:31 +00:00
Changpeng Liu
021ebb3529 bdev_nvme/opal: remove unnecessary opal configuration
For base bdevs, we maintain a global list g_opal_base, and
for part bdevs, there is also a global list g_opal_vbdev,
so it's not necessary to use the configuration again, just
unpack it to opal part bdev.

Change-Id: Ia0943610dee3c194a80e44ed8eed8f2370d780fc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3106
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-01 07:50:59 +00:00
Changpeng Liu
c3ace4f40c bdev_nvme/opal: use spdk_bdev_unregister() when deleting a vbdev
The base bdev's reference number is maintained in the bdev
layer, so we don't need to maintain it in this module.

Change-Id: Iffadbde2dbf35d934f273fdcb4b7d19e96394343
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2986
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: wanghailiang <hailiangx.e.wang@intel.com>
2020-07-01 07:50:59 +00:00
Ziye Yang
c833729b22 sock/uring: Fix the nvmf_shutdown_tc3 failure issue.
If the socks parameter(passed in uring_sock_group_impl_poll) is NULL,
we do not need to handle the sock_flush and prep the pollin task,
otherwise it will cause the assert issue when we reap the task when
we handle the nvmf_shutdown_tc3 issue.

Because in uring_sock_group_impl_remove_sock, we finally
set sock->group = NULL. Without this patch,
when we call uring_sock_group_impl_poll in this function,
pollin_task or write_task are prepared, then in the next round,
we will reap those tasks again.

PS: Error info can be found in
https://ci.spdk.io/results/autotest-per-patch/builds/19186/archive/nvmf-tcp-vg-autotest/build.log

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7e6deaa05e958b52e71e0bbf0ccdd20e35583685
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3031
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-29 09:18:13 +00:00
Ziye Yang
d81d7a2c22 bdev/nvme: fix the compilation warning for unused variable.
The error message is:

bdev_nvme.c:456:6: warning: variable ‘rc’ set but not used [-Wunused-but-set-variable]
  456 |  int rc;

Reason: The CI testing pool will always use --enable-debug, but without this
flag, we will see complilation warning.

If we really want to catch this, it is better to really use this rc variable and print
some information.

Change-Id: Iec0ffcec4ec091d36044e3b36a9ac85e677b5c70
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3001
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-24 08:22:11 +00:00
Seth Howell
62ded32fd7 bdev/nvme: add failover option to bdev_nvme_reset
When we fail to process admin completions on a controller
attempt to failover to a previously registered trid

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Icdb43a726969fede4665f2fe59723dd90c5dad0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2883
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-22 10:13:29 +00:00
Seth Howell
5f13e0f67f bdev/nvme: add a function for specifying a multipath trid.
This is part of a larger series enabling failover at the bdev
layer for NVMe.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I0d89e1afab8aeaa90237d0ba780d708154f6e3be
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2881
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-22 10:13:29 +00:00
Seth Howell
af9e73302f bdev/nvme: make the controller trid a pointer.
This lends itself to easily changing the trid during failover.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iae21a98de4f9dd1192a50abac0ec7fb0dd8a2e99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2882
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-22 10:13:29 +00:00
Ziye Yang
a37fd6e03d uring: remove user's ring file descriptor operation.
According to the description of io_uring_queue_exit:

Tear down function for io_uring. Unmaps all setup shared ring buffers
and closes the low-level io_uring file descriptor returned by the kernel.

So we should remove the close operatoin on ring fd.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I910c6e8acd935925b7985c2aa750df385004eb55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2922
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-19 07:18:40 +00:00
paul luse
8d2c520048 lib/idxd: add support for dual cast
Dual-cast copies the same source to two separate destination buffers.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icadae34a75c35e2db672a193287b147416012a5a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2020-06-17 07:21:05 +00:00
Ben Walker
ff24db746f nvmf: spdk_nvmf_tgt_accept no longer takes a new_qpair pointer
Calling spdk_nvmf_tgt_accept() now automatically assigns new qpairs
to the best available poll group.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I3df2a2c5a28dba45c5ba0cbd1e8c28dd7e56cf9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2813
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-16 07:44:49 +00:00
Ben Walker
c37cf9fbd8 nvmf: Make transport-chosen the only valid qpair scheduler
The transport always knows best.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: If66f539033fe11c50dc42e12ec990468588d5739
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2812
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-16 07:44:49 +00:00
Maciej Szwed
e7e10859d3 nvmf: Make spdk_nvmf_tgt_accept return the number of events accepted
This will be usefull for pollers/threads stats.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I4d1651f3ff6410c258c8bc75c2a68640b67d2ed9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2849
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-15 15:28:00 +00:00
Shuhei Matsumoto
82d77ca6f5 bdev/delay: Set zcopy I/O type to be unsupported regardless of backend bdev module
Delay bdev module does not support delaying zcopy I/O. So change
vbdev_delay_io_type_supported() to return false if the passed I/O
type is zcopy to clarify the use case of delay bdev module.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icea5ae351d7139250710ce491e4ac466b0cc7f98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2622
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-10 13:54:40 +00:00
Shuhei Matsumoto
a954d2bf30 bdev/delay: Abort I/O delayed to complete
This addition enables us to test the new abort I/O feature on top of
any bdev which supports I/O type ABORT.

Add the abort I/O handler vbdev_delay_abort() into vbdev_delay_submit_request().

vbdev_delay_abort() aborts the target I/O if it is delayed to complete,
or calls spdk_bdev_abort() otherwise.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I191143357b95eeb56ada708d2d7ce30cf772bdcf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2602
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-10 13:54:40 +00:00
Shuhei Matsumoto
c01fe4c530 bdev/delay: Abort all I/Os delayed to complete at reset I/O
Improve reset I/O to abort all I/Os delayed to complete by parsing
all channels. The generic bdev layer stops new I/O submission during
reset processing. So freezing/unfreezing channels is not necessary.

Additionally, clean up the code which run if I/O submission failed.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I63e1ba78971944558d9317b63befecd86a6d6b60
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-09 07:37:29 +00:00
Shuhei Matsumoto
bfc9ddde2d bdev/passthru: Support I/O type ABORT
Passthrough bdev module sets the passed bdev_io to the cb_arg of
the bdev_io submitted to the underlying bdev. Hence call
spdk_bdev_abort() with bdev_io->u.abort.bio_to_abort for the I/O
type ABORT.

Passthrough bdev module returns the supported I/O type of the
underlying bdev. Hence vbdev_passthru_io_type_supported() is not
necessary to change.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb7627a92409649ab08ae233ba517fbc1bc55a9f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2799
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-09 07:37:29 +00:00
Shuhei Matsumoto
269f4a3c25 bdev/null: Support I/O type ABORT
As malloc bdev module supports I/O type ABORT, it will be helpful
if any bdev module other than NVMe bdev module supports I/O type
ABORT.

Update bdev_null_io_type_supported() to return true for
SPDK_BDEV_IO_TYPE_ABORT and update bdev_null_submit_request() to
try aborting the target I/O if queued.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4614c48840ef069a11991486d184d45789bc3000
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2798
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-09 07:37:29 +00:00
Shuhei Matsumoto
e2d1dfb4fa bdev/malloc: Always return failure but support I/O type ABORT
Malloc bdev module does not manage submitted I/Os by any list,
and so cannot abort any submitted I/O. However malloc bdev module
can always fail any submitted abort request.

Hence let's update bdev_malloc_io_type_supported() to return true
for SPDK_BDEV_IO_TYPE_ABORT and update _bdev_malloc_submit_request()
to complete with failure for SPDK_BDEV_IO_TYPE_ABORT.

This will enable us to use delay bdev + malloc bdev to test I/O
abort feature conveniently.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If4c411f5ab8589291ac90c10264d3ef30c06df83
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2797
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-09 07:37:29 +00:00
Shuhei Matsumoto
7cd20dd3f5 lib/bdev: Add spdk_bdev_abort API
Add spdk_bdev_abort function as a new public API.

This goes all the way down to the bdev driver module and attempts to
abort all I/Os which has bio_cb_arg as its callback argument.

We can separate when only a single I/O has bio_cb_arg and when multiple
I/Os have bio_cb_arg, but unify both by using parent - children I/O
relationship. To avoid confusion, return matched_ios by _bdev_abort() and
store it into split_outstanding by the caller.

Exclude any I/O submitted after this abort command because the same cb_arg
may be used by all I/Os and abort may never complete.

bdev_io needs to have both bio_cb_arg and bio_to_abort because bio_cb_arg
is used to continue abort processing when it is stopped due to the capacity
of bdev_io pool, and bio_to_abort is used to pass it to the underlying
bdev module at submission. Parent I/O is not submitted directly, and is
only used in the generic bdev layer, and parent I/O's bdev_io uses bio_cb_arg.
Hence add bio_cb_arg to bdev structure and add bio_to_abort to abort structure.

In the meantime of abort operation, target I/Os may be completed. Hence
check if the target I/O still exists at completion, and set the completion
status to false only if it still exists.

Upon completion of this, i.e., this returned zero, the status
SPDK_BDEV_IO_STATUS_SUCCESS indicates all I/Os were successfully aborted,
or the status SPDK_BDEV_IO_STATUS_FAILED indicates any I/O was failed to
abort by any reason.

spdk_bdev_abort() does not support aborting abort or reset request
due to the complexity for now.

Following patches will support I/O split case.

Add unit tests together to cover the basic paths.

Besides, ABI compatibility check required us to bump up SO version of
a few libraries or modules. Bump up SO version of blob bdev module simply
because it does not have any out-of-tree consumer, and suppress bumping
up SO version of lvol library because the affected struct spdk_lvol
is not part of public APIs.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I515da688503557615d491bf0bfb36322ce37df08
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-08 09:28:27 +00:00
Tomasz Zawadzki
e757878190 accel: always write JSON with an array of objects for accel modules
Patch below added writing out json configuration for accel modules.
(fad052b0) accel: add write_config json for accel modules

Out of two current modules, only idxd reported its confi out.
When idxd was not compiled in, then "framework_get_config accel"
RPC did not return a valid JSON. Following JSON was returned:
{\"jsonrpc\":\"2.0\",\"id\":1,\"result\":}

Where "result" never had any property given.

This patch adds array object to encompas all accel modules.

Next patch in series will try to address the issue on JSON
level.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib0758db5406896f2a204920417619a3ab6fbde98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2550
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-03 07:38:01 +00:00
Maciej Szwed
a2606d4b5d all: Fix warnings found by gcc 9.3.0 and 9.3.1
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ie7c5b7b1c2d59a1f927eb5602f4ea5a003eff287
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-02 09:51:57 +00:00
Ziye Yang
d8cafc28bb Sock: The created pipe for sock should have a minimal value.
Thus, we can make sure that when read data is larger than
the pipe size, it will not read the data into the pipe.

Change-Id: I87f3b03fd9b81eb693e9eae0fea9eef7d1b9eaa8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2450
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-02 09:48:59 +00:00
Ziye Yang
ef9fd47d07 sock/uring: Fix the connection close issue when the initiator closes the connection unexpected.
When the initiator terminates the connection suddently, we

found the keep alive timeout issue, e.g.,

nvmf_ctrlr_keep_alive_poll: *NOTICE*:
Disconnecting host from subsystem nqn.2016-06.io.spdk:cnode1 due to keep alive timeout.

The root cause is that we did not closes the connection on our target side on time
whening using uring sock, and this patch can fix the issue.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I295f58bbdbae0ac3f5308f6eadef6a75c5ad07d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2544
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-02 09:48:15 +00:00
Evgeniy Kochetov
74b184e73a sock/posix: Add recv_buf_size and send_buf_size socket layer options
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ifb7d542c4da83070344c4baa512da459ba73ec90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/610
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-02 09:48:00 +00:00