7703 Commits

Author SHA1 Message Date
Maciej Szwed
68493f45af event: Add DPDK based governor
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ide0ac38497bc500886e26fd219cb120113896966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4152
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
c6adf304bd event: governor implementation
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9aaa84261144145933d227976a5f89dc89732267
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4028
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
2cffc80079 event: add rebalancing infrastructure
Features:
- does not delay thread creation
- does delay thread deletion
- singlethreaded, but can be made multithreaded.
  By being singlethreaded, we don't waste time,
  because reactors are not paused during rescheduling,
  but we do make statistics less up to date

Change-Id: Ie5a7e8569bc32b3fd4bb887804dfbc3f5c2ea858
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
7148f33360 event: adopt static scheduler to the new API
Move current scheduler to it's own module
and make use of new API.

Change-Id: I4928aed82603d51de01194c9650709e814f7f61b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4054
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
0d8a4af849 event: Add lw_thread helper functions required for scheduler
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ifd21adad6116d75170d5a6ff4b6d03470644feab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4338
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
2a146cd936 event: define scheduler API
Add structures and function for new scheduler API.

Change-Id: I8e379db62189eb7c36092265c137f07368650e9b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4053
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Liu Xiaodong
54a7d73ad1 nbd: apply interrupt
If interrupt mode is set, related poller functions will be
registered to interrupt_handler instead of poller.
interrupt_tgt can run IO with linux nbd.

Change-Id: I39ecf1efa10be76419fb0d25713ea457a5a53b37
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
4bf6e4bb32 interrupt: apply fd_group in thd and reactor
Each reactor and each thread is assigned with one fd group.
At the same time, each thread is treated as one interrupt source
registered into its corresponding reactor.
The egrp function for reacotr is the only block point waiting
for events.

Change-Id: Id092e66591b07b445342f8ae16dc218d28887427
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
7d19e50a54 reactor: extract reactor_post_process_lw_thread
Change-Id: I2761db384f78529525b7f1eb3c9c959a7b885ede
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4756
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
c5d1e7f171 interrupt: new fd_group in lib/util
Interrupt is aiming to support interrupt
mode in SPDK application framework.
fd_group is implemented by epoll on linux.
fd_group can be inserted into thread and reactor
to do epoll_wait internally in order to avoid polling

Change-Id: I9077ad648a97fbd68fd46f43de2e16440bedab0b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4268
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Ben Walker
2ec88c4f8c nvmf: Don't fail a property set command with invalid bits
If the initiator sends a property set command to a valid register, but
the value contains bits that are reserved by the NVMe specification,
don't fail the whole command. Just log an error message.

Previously, any valid bits set would take effect but then the command
would also fail. That confuses the initiator and thinks it must retry
the full property set.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I566bc68c4469b9f41c69902e276f825c86683075
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:35 +00:00
Karol Latecki
507f8ae3ea lib/env_dpdk: update crypto and compress library names
Post DPDK 20.08 crypto, isa-l and qat shared object
libraries had their names changed. Detect which library
names are available and use them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8c9b9b24f9232f303b99c959754a7f7550ffdf90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4780
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 10:14:12 +00:00
paul luse
397cf3f884 lib/idxd: small code cleanup
Suggestions from a prior review... able to remove a boolean by changing
how the batch elements 'index' and 'remaining' are used.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I273e5e231bb30d51eb3ae0a59eec110377d49ab7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4813
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-22 22:43:28 +00:00
paul luse
9d5268b19c lib/idxd: remove unused translations
Added by mistake.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I08fb4d105f4834a551488bcfec413a46f1404e34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4811
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2020-10-22 22:43:28 +00:00
paul luse
63d7ac35c9 lib/idxd: small code simplifcation
Earlier refactoring enables us to not have to keep track of batch completions in
the batch struct as they're always used sequentially now so we can just add
the addresses from the start up to the number of elements in the batch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I00cdcdec3376a1c32c9dab72c68fea868c1cb540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4810
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2020-10-22 22:43:28 +00:00
paul luse
33eac886b9 lib/idxd: refactor batching for increased performance
And to eliminate an artificial constraint on # of user descriptors.
The main idea here was to move from a single ring that covered all
user descriptors to a pre-allocated ring per pre-allocated batch.

In addition, the other major change here is in how we poll for
completions.  We used to poll the batch rings then the main ring.
Now when commands are prepared their completion address is added to
a per channel list and the poller simply runs through that list
not caring which ring the completion address belongs too. This
simplifies the completion logic considerably and will avoid
polling locations that can't potentially have a completion.

Some minor rework was included as well, mainly getting rid of the
ring_ctrl struct as it didn't serve much of a purpose anyway and
with how things are setup now its easier to read with all the
elements in the channel struct.

Also, a change that came in while this was WIP needed a few fixes
to function correctly.  Addressed those and moved them to a
helper function so we have one point of control for xlations.

Added support for NOP in cases where a batch is submitted with
only 1 descriptor.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie201b28118823100e908e0d1b08e7c10bb8fa9e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3654
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 22:43:28 +00:00
paul luse
17bb748a60 accel: Move non-engine specific batch to the accel_fw layer
The new design:

* Supports a generic batching capability in the accel_fw layer
that keeps track of hw accelerated vs sw commands based on
the capabilities of the engine and processes sw commands in the
generic layer while sending a list of commands (not a batch)
to the engines for processing.

* Batch completions are managed via the generic layer, when using
the accel_fw the engines only process commands. With DSA however,
if a list of commands is sent down it will use the DSA public
API to create and send batches but will rely on the generic layer
to complete the batch task itself. When using DSA directly, batching
works as usual (DSA handles batch completion).

* The engine function tables were greatly simplified by replacing
all of the individual entries (copy, fill, crc32c, etc) with one
`submit_tasks` function that is used to both send lists of tasks
for batches or just one task for single shot API.

* Internally batching is now used to re-submit tasks that were queued
for flow control reasons.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I99c28751df32017c43490a90f4904bdabe79a270
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2020-10-22 22:43:28 +00:00
Tomasz Zawadzki
78ad672861 lib/event: remove legacy configuration from event framework
All options -c, --config and --json are valid, but have to
point to JSON configuration file.

Adjusted UT since JSON configs don't work with --wait-for-rpc.

Since this removes last reference to legacy INI configuration,
updated conf library to no longer mention the deprecation.
All uses of conf library are for explicit reason and not
related to SPDK event framework configuration.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic9a702465982daf715ce1c2ab863c48584734611
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 17:08:41 +00:00
Ben Walker
6723bd0c0f nvmf: nvmf_subsystem_remove_host RPC now disconnects hosts
If a host matching the removed hosts' NQN exists, it is now
disconnected.

Change-Id: I0bec29eda2dc220114b9197d4eb765899b9e1517
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
2020-10-22 03:55:57 +00:00
Changpeng Liu
c0bf93145a env/memory: return the memory virtual address's file descriptor and offset
For virtio-user library and coming vfio-user feature, the client needs
to send the memory file descriptors to target so that the two processes
can setup shared memory region to do data processing without memory copy.
Currently virtio-user will read /proc/self/maps to get memory file descriptor,
since DPDK already provides this such APIs, so here we can just use it,
for existing virtio-user library we may replace it with the new added
API.

Change-Id: Icfeae465d53826d0c8d1b335287634b03cd174aa
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4428
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-21 20:47:01 +00:00
Alexey Marchuk
c72a16431a nvme/tcp: Fix check of completion number during icresp handling
The current approach checks "rc == 0". It worked before adding
polling of poll group since a single qpair should return 1
completion for its own icreq while poll group can return
several completions for all qpairs attached to this poll
group (but .e.g not for those qpair who is waiting for the
completion).

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I60d05d8d6640e4e2bbaf3cd533d2f5a3637adea1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-21 20:45:13 +00:00
Tomasz Zawadzki
344bb69312 event: remove printing legacy config for apps
Remove spdk_app_get_running_config() that allowed printing
legacy configuration by the apps.
Along with usr1_handler callback that was used to call that
function.

It was only used in iscsi_tgt so it is removed there.

The app_repeat test was using SIGUSR1 to trigger a
spdk_app_stop/spdk_app_start cycle.  But we can use
SIGTERM for that instead.  While here, do a bit of
cleanup in the app_repeat test app.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie8449e8bcd979b237c68ef94f50fb7a3cb6c19db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4750
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 20:44:47 +00:00
Niklas Cassel
6dec7623be nvme: add support for ZNS zone management receive command
Add support for the ZNS zone management receive command.
An internal nvme_zns_zone_mgmt_recv() function is created
that matches the parameters of the zone management receive
function in the ZNS specification.

Convenience functions are provided for the following
Zone Receive Action: Report Zones.

Zone Receive Actions not implemented: Extended Report
Zones.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I23589a602336da5dffccec7230d07026a868e81b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4793
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 12:18:44 +00:00
Niklas Cassel
be3ff9c0b5 nvme: add function to get ZNS num zones
Add a function to get the number of zones for a zoned namespace.

Since the ZNS specification does not allow zone size == 0,
divide by zero should not be possible on a spec compliant drive.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I15e0ac8a72e244e248b7fb44f7156b182ecd98b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4792
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 12:18:44 +00:00
Niklas Cassel
6abee9a205 nvme: add function to get ZNS zone size
Add a function to get the zone size of a zoned namespace.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I89429fa2fd36307a8b34bdf91e3251c68596a48c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
2020-10-21 12:18:44 +00:00
Tomasz Zawadzki
6511969c3b event: remove subsystem legacy config print
This patch removes callback function for subsystems to
present their options for legacy config.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I63076fc03eff45da5d57fab03501602922a20e3e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4749
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-21 02:25:45 +00:00
Tomasz Zawadzki
28ebb8cc78 bdev_module: remove config_text
This patch removes function for bdev modules to
present options of the bdevs.

blob_bdev.h refers to the spdk_bdev_module, so would need
to be bumped too.
At this time spdk_bdev_module is left unchanged to prevent
that.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3cacb087c998d928c5d8c2722b7f041d82bb43f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4748
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 02:25:45 +00:00
Ivan Betsis
bd3840a729 nvmf/rdma: Add RPC to set new RDMA batching option
Add option to enable/disable work requests batching (RDMA only).

Signed-off-by: Ivan Betsis <c_ivanb@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: I84ca599711cdc2713606444e7ec501c36671e796
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/925
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 02:25:32 +00:00
Ivan Betsis
c818233b41 nvmf/rdma: Add WR batch rdma new transport option
With x86 and low queue depths with multiple QPs/initiators
there is a benefit from disable batch when we have randread IO pattern.

Testing environment: x86, Intel(R) Xeon(R) Silver 4116 CPU @ 2.10GH
Several results:
init_cores | QD | BS | Avg IOPs/BW without batch | Avg IOPS/BW with batch
  8 cores  | 4  | 4K |     1870087 / 7305        |     1594014 / 6226
  8 cores  | 8  | 4K |     1853573 / 7240        |     1576400 / 6157
  8 cores  | 16 | 4K |     1819643 / 7108        |     1569487 / 6130
  8 cores  | 32 | 4K |     1815467 / 7092        |     1569909 / 6132
 16 cores  | 4  | 4K |     1908018 / 7453        |     1566843 / 6120
 16 cores  | 8  | 4K |     1906081 / 7446        |     1562110 / 6102
 16 cores  | 16 | 4K |     1880706 / 7346        |     1555060 / 6074
 16 cores  | 32 | 4K |     1835878 / 7171        |     1548156 / 6046

Signed-off-by: Ivan Betsis <c_ivanb@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Change-Id: Icdbbbdf83f137eb4f05bd2063268ee2a7d87335a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/924
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
2020-10-21 02:25:32 +00:00
Shuhei Matsumoto
d8f4bbeb43 lib/nvme: Add spdk_nvme_detach_async() and spdk_nvme_detach_poll_async()
Add two new public APIs, spdk_nvme_detach_async() and
spdk_nvme_detach_poll_async() to detach multiple controllers in parallel
as a simple manner to users.

Hold the target controller to nvme_ctrlr_detach_ctx because users
will free any object which held it after returning spdk_nvme_detach_async().

spdk_nvme_detach_ctx holds all nvme_ctrlr_detach_ctx in a sequence
by linked list.

spdk_nvme_detach_ctx has a boolean variable polling_started to prevent
the user from calling spdk_nvme_detach_async() while
spdk_nvme_detach_poll_async() is called repeatedly.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>i
Change-Id: Ib049c19f7ef24410b963fd5c777a21184f3012d1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 16:55:55 +00:00
Shuhei Matsumoto
ea1bfd84cd lib/nvme: Make internal of spdk_nvme_detach() asynchronous
Add two new helper functions, nvme_ctrlr_detach_async() and
nvme_ctrlr_detach_poll_async() to make the internal of
spdk_nvme_detach() asynchronous.

Use callback function to remove controller from the attached list after
completing shutdown and before freeing to avoid conflict between
attach and detach.

Update MOCKs in the corresponding unit test cases.

The next patch will add two public APIs spdk_nvme_detach_async()
and spdk_nvme_detach_poll_async() based on this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifbdfec2a1facde9354007c6248f280e245a36eed
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 16:55:55 +00:00
Shuhei Matsumoto
3806b2e1f9 lib/nvme: Make nvme_ctrlr_destruct() asynchronous
Following the last patch, separate nvme_ctrlr_destruct()
into nvme_ctrlr_destruct_async() and nvme_ctrlr_destruct_poll_async(),
but keep nvme_ctrlr_destruct() by replacing the internal by
nvme_ctrlr_destruct_async() and nvme_ctrlr_destruct_poll_async().

Add shutdown_complete to nvme_ctrlr_detach_ctx. If shutdown_complete is true,
we can destruct the controller. The case that nvme_ctrlr_shutdown_async()
failed sets shutdown_complete to true. The case that nvme_ctrlr_disable()
is called sets shutdown_complete to true unconditionally.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3994e259f9d3ccf8fede3ac03aadef911eefb9dd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4415
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-20 16:55:55 +00:00
Shuhei Matsumoto
257fcb7352 lib/nvme: Make nvme_ctrlr_shutdown() asynchronous
This patch is the first of the patch series to make spdk_nvme_detach()
asynchronous.

We have lengthy shutdown notification, i.e., we have to wait a long time
until shutdown processing is completed, in some SSDs. If the running system
has many such SSDs, we see large intolerable delay.

SPDK provides a controller option, no_shn_notification as a workaround.
We can use the workaround if the use case of the detach is to switch to
the next application without system reboot. However, we cannot use the
workaround if we want to do system reboot after detach.

To mitigate such lengthy shutdown notification, we need to parallelize
detachment among SSDs.

Hence the patch series will introduce an asynchronous detach API and
will use the API to parallelize detachment.

This patch adds the following changes.

Introduce a context structure and separate nvme_ctrlr_shutdown()
itno nvme_ctrlr_shutdown_async() and nvme_ctrlr_shutdown_poll_async()
using the context structure.

Name the context structure as nvme_ctrlr_detach_ctx because it will be
used only in internal APIs. The upcoming public APIs will support
multiple detachment and will have the contest structure named as
spdk_nvme_detach_ctx.

Use TSC instead of counter because polling interval will be controlled
by the caller.

Use the convenient macro, SPDK_CEIL_DIV(), to round off the time
value in milliseconds.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9e2355fd24b6d6a4d6c1813577d53822304d4f33
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4414
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 16:55:55 +00:00
Niklas Cassel
67475a58f4 nvme: add support for ZNS zone management send command
Add support for the ZNS zone management send command.
An internal nvme_zns_zone_mgmt_send() function is created
that matches the parameters of the zone management send
function in the ZNS specification.

Convenience functions are provided for the following
Zone Send Actions: Close Zone, Finish Zone, Open Zone,
Reset Zone, and Offline Zone.

Zone Send Actions not implemented: Set Zone Descriptor
Extension.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I9cc04776c249d2b9c515438b81a0f06aea4198fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-20 12:18:56 +00:00
Niklas Cassel
3be2935701 nvme: move ZNS specific functions to their own header file
Move ZNS specific functions to their own header file:
include/spdk/nvme_zns.h.
This is similar to how it's done for e.g. NVMe-oF with
include/spdk/nvmf.h.

This avoids cluttering the nvme.h header with ZNS specific functions.

A SPDK program that wants to use the spdk_nvme_zns_* functions will
need to include include/spdk/nvme_zns.h, however, this is similar
to how e.g. spdk_nvmf namespaced functions work.
This change is safe to do now, since there is no SPDK release with
ZNS so far.

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: If39fab4e7a6924cc3bd67aa0c03e9c6de5f8a324
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4681
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-20 12:18:56 +00:00
Niklas Cassel
198a3ad2fd nvme: if no command set is requested, use the most reasonable default
Since namespace types were introduced in NVMe, the CC.CSS register
has a new value (SPDK_NVME_CC_CSS_IOCS) which indicates that all
supported command sets should be selected/enabled. This possibly
includes command sets other than NVM and ADMIN only.

Therefore, if a SPDK application wants enable all the command sets
that the controller supports, it has to explicitly set
opts->command_set to SPDK_NVME_CC_CSS_IOCS.

To avoid possibly a lot of SPDK applications having to set this
parameter, check if the user requested a command set explicitly,
if not, make SPDK automatically use the most reasonable default,
based on the supported bits set by the controller.
The most common case is that you want to enable (all) the command
sets that the controller supports.

A user will still be able to restrict the controller to only use
the NVM command set (or ADMIN only), by setting opts->command_set
to a specific value.

Since the current default command set value specified by
spdk_nvme_ctrlr_get_default_ctrlr_opts() is SPDK_NVME_CC_CSS_NVM,
which is defined as 0, we cannot know if the user specified a
command set explicitly or not.
To solve this, change the default command set value specified by
spdk_nvme_ctrlr_get_default_ctrlr_opts() to CHAR_BIT (0x8), which
is larger than the largest value that can be set in CS.CSS (which
is only 3 bits wide, thus 0x7).

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I45ec148d3667ab87c41fbfb6d6612a1e0e5c9d9c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-20 12:18:56 +00:00
Tomasz Zawadzki
6d86242dc5 subsystem/iscsi: remove legacy config support
This patch removes legacy config support in iscsi subsystem.
All options through the legacy config are already reflected in JSON.

Following RPCs are corresponding to removed legacy config sections:
Initiator groups - iscsi_create_initiator_group and iscsi_initiator_group_*
Subsystem options - iscsi_set_options
Portal groups - iscsi_create_portal_group
Target node - iscsi_create_target_node and iscsi_target_node_*

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I11326a84d4d580b19db422b8522198eea5a5be0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4747
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 08:53:53 +00:00
Shuhei Matsumoto
9b32f4858c lib/rocksdb Use spdk_bdev_create_bs_dev_ext() to pass bdev_name
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4acce43baa95d0968b7b568e87fade1972efc646
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 08:52:29 +00:00
Tomasz Zawadzki
cda1d2cf11 build: remove unused references to conf library
- Removed slew of conf.h includes
- No longer require mk vars that include conf

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica7e8e8bf1d4a5d0b0200bfe689aa13afd77bfaf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
9103bca73b lib/event: remove deprecated ReactorMask from legacy config
Legacy INI configuration is being removed this release.
This patch removes one of its options and always uses the
SPDK_APP_DPDK_DEFAULT_CORE_MASK as default core mask.

That can only be overwriten by -m/--cpumask arguments in app.

In bdevperf there was no reason to set reactor_mask to NULL,
as that was then still using the defaults.

Meanwhile bdev_svc will now check if user provided different
string via command line args to know if it should
unaffinitize its thread.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id882f8d7dbdd07b7743bd5981c37daa888b7872a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4619
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
5b1e154a5c bdev: remove legacy config support
This patch removes legacy config support in bdev layer.
All options through the legacy config are already reflected in JSON.

Removed configuration can be set via `bdev_set_options` and
`bdev_set_qos_limit` RPC.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I24c365625540659cad425268d2aa41e3bf279d5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
a8de86e9aa lib/accel: remove legacy config support for accel module register
Since legacy config is being removed, SPDK_ACCEL_MODULE_REGISTER
macro no longer needs field corresponding to processing this
type of configuration.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I76ca02487a5fb8f9ecc9d33ac6fa514e7226a03b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
ceea308887 lib/blobfs: remove legacy config support
This patch removes legacy config support in blobfs library.
Mostly reverting patch:
(97f3104) blobfs: Add conf parse for blobfs.

CacheBufferShift option in legacy config is not tested in CI,
and never received corresponding RPC.
If required the RPC can always be added later on.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I54e39f069047a243b2186ae5ea225ed452180488
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4668
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-20 02:42:16 +00:00
Tomasz Zawadzki
b1f76f512f lib/vhost: remove legacy config support
This patch removes legacy config support in vhost_blk/scsi library.
All options through the legacy config are already reflected in JSON.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia63651cdb7433267d1a8839a1739e68b436e5d08
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-20 02:42:16 +00:00
Changpeng Liu
6bd7bd0da2 vhost: deprecate internal vhost library support
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.

For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.

As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.

Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-20 02:42:16 +00:00
Darek Stojaczyk
c77c655987 json: add spdk_json_free_object()
After decoding a JSON object we had to free the parsed
strings one-by-one. Not anymore.

Change-Id: I819f1d533e397aa9babca58b5500c38ac01a963d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2753
Reviewed-by: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-19 10:02:10 +00:00
JimboLuCN
944a480ea7 blobfs: fix the problem that the return code is not returned correctly
In spdk_file_read and spdk_file_write, __send_rw_from_file always returns 0 and does not return the correct error code,
because req has been released in __rw_from_file_done.

Signed-off-by: JimboLuCN <lujinbo@didiglobal.com>
Change-Id: I4437cf8f1817dbb98bf71b808ad1c544e7b09853
Signed-off-by: JimboLuCN <lujinbo@didiglobal.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-19 10:01:21 +00:00
Yao Liu
ed0c3125fb vhost: add missing smp_rmb in vhost_vq_avail_ring_get
Signed-off-by: Yao Liu <yotta.liu@ucloud.cn>
Change-Id: Ie8f42cb41ae923ae0926290019cc79d9f5afc868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4647
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-19 10:01:01 +00:00
Jim Harris
44ff994386 event: remove include of spdk_internal/thread.h
It's not needed.  Add it to one of the UT where it
is needed though.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I06628184c22df68deffce3ca0561878569ccf3f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4717
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-19 10:00:29 +00:00
Jim Harris
95e5339490 nvme: fix signed/unsigned error with cuse ioctls
cuse defines the ioctl handler to take an int for
the command, but the kernel uses unsigned int.  So
cast the int parameter to unsigned int when doing
the switch.

Fixes issue #1613.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7ef13afc755ee79d8a79b3c7a09dccee7bb7aa1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4716
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-19 10:00:29 +00:00