13721 Commits

Author SHA1 Message Date
yidong0635
0e70cb48ab rocksdb/env_spdk: Fix unused warning.
Fuction takes one parameter to print,
others are unused.

spdk/lib/rocksdb/env_spdk.cc: In function
 ‘void rocksdb::base_bdev_event_cb(spdk_bdev_event_type, spdk_bdev*, void*)’:
/spdk/lib/rocksdb/env_spdk.cc:666:70:
error: unused parameter ‘bdev’ [-Werror=unused-parameter]
666 | base_bdev_event_cb(enum spdk_bdev_event_type type, struct spdk_bdev *bdev,
      |                                                    ~~~~~~~~~~~~~~~~~~^~~~
/home/yidong/spdk/lib/rocksdb/env_spdk.cc:667:12:
error: unused parameter ‘event_ctx’ [-Werror=unused-parameter]
  667 |      void *event_ctx)
      |      ~~~~~~^~~~~~~~~
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4929 (master)

(cherry picked from commit a474889bc6cae9a5525f8f519714cf0b07ca836b)
Change-Id: Ic1cf45443ab1dcdf38d1b9c6bdea2905e94df19c
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4933
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-29 08:09:09 +00:00
Liu Xiaodong
494848e3a9 reactor: check calloc failure in gather_metrics
A round of _reactors_scheduler_gather_metrics should be stopped
when there is calloc failure.

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4882 (master)

(cherry picked from commit e2f773aafc2f5515f19a17aeb3e0b556b23fdff9)
Change-Id: Ic2220c561abb07a849ea37d3c88af3f6d5d1ffa1
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4923
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 15:03:17 +00:00
Alexey Marchuk
623d5cc456 nvme: Don't log an error when we can't resubmit all requests
In TCP NVME initiator with zero copy enabled requests might be
completed asynchronously - out of qpair_process_completions
context. At the same time we calculate requests completed
asynchronously so that generic NVME layer can resubmit
queued requests after calling qpair_process_requests (or
poll_group_process_requests).
But there is a time gap between async request complete and
qpair_process_completions and the user can submit new IO
thereby decrease the number of free TCP requests. That means
that there might be less free requests than we excpected when
we try to resubmit queued requests.
The solution is change ERRLOG to DEBUG log since it is not a
fatal case.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4859 (master)

(cherry picked from commit e385cafa7296ef656aae71b869eef0a211e0a0e9)
Change-Id: If045ecd331cc6693e8ef450d8e15432dfa5d8812
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4872
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-28 08:09:03 +00:00
Liu Xiaodong
0bcafaea56 lib/rocksdb: remove redundant linked blobfs_bdev
blobfs_bdev lib is already added into BLOCKDEV_MODULES_LIST
so it shouldn't be included by application who already
uses BLOCKDEV_MODULES_LIST or ALL_MODULES_LIST.

Fixes issue: #1654

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4876 (master)

(cherry picked from commit b45788036fd43c6d9ca73f047d53f8592fe44909)
Change-Id: I46a272e4593e19cf14c3ed8b2965797443c37a0d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4922
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:49 +00:00
Allen Zhu
b5005497db rpc/nvmf.py: pass zero values to SPDK when allowed
in_capsule_data_size/buf_cache_size/sock_priority/max_namespaces can be 0,
which should be passed in nvmf_create_transport/nvmf_create_subsystem commands.

Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3815 (master)

(cherry picked from commit 96ab62802ca344c5b04c249855e1d13095908f32)
Change-Id: Ib557cf9f20f7ec2c0b3c31156cd79dbd670ce7e7
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4921
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Allen Zhu <allenz@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:37 +00:00
Alexey Marchuk
839af8867e nvmf/tcp: Align recv_buf_size to MIN_SOC_PIPE_SIZE
If the user decided to disable ICD then we have several side effects:
1. SPDK prints several warnings/errors
2. SPDK doesn't create recv pipe and doesn't set SO_RCVBUF socket option.

I think that we should not rely on ICD only when we create recv pipe or
set SO_RCVBUF since data may be transferred in sgls via R2T/H2C and
we still need recv_pipe and SO_RCVBUF for better performance.
Alternative option is to set recv_buf_size as a maximum between
ICD and io_unit_size

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4755 (master)

(cherry picked from commit c1fbbfbe560066b8474458ed2cabf76018ed97c7)
Change-Id: Ida71ecc099f9a9355e4617f13315a341872d1cb3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4920
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-28 08:08:37 +00:00
Alexey Marchuk
bf602f12ca nvmf/tcp: Support ICD for fabric/admin commands
According to the SPEC we should support up to 8192 bytes
of ICD for admin and fabric commands. Transport configuration
parameter in_capsule_data_size is applied to all qpair types -
admin and IO. Also we allocate resources when we get a connection
request, so we don't know qpair type at this moment.
Create a list of buffer in TCP poll group to support ICD up
to 8192 bytes when configuration ICD is less than this value.
The number of elements in this pool is hardcoded, it is planned
to add a new configuration parameter later.

Fixes issue #1569
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4754 (master)

(cherry picked from commit 85fa43241bc236d4bcb91a261d565352f6ff3559)
Change-Id: I8589e3e2ea95d515f5503c6de7c1ee40aaf7b6da
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4886
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-28 08:08:37 +00:00
Liu Xiaodong
ad97082bd0 bdev_aio: fix interrupt mode notify error
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4865 (master)

(cherry picked from commit 47962dc7a344c5eb66463bb920842af596706f1f)
Change-Id: Ie4492aa33028e8090da96e2b592b20293d694120
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-27 15:44:16 +00:00
Jim Harris
05216cb7bf event: deprecate opts.config_file member
Just always put the config file name in json_config_file,
since we now only support JSON.

If user specifies both -c and --json, it will just take
the latter of the two.  This is similar to if the user
specified --json twice.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Idc21d73acf0e190eda57a7b0c5d9bcfa14e87030
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4858
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
(cherry picked from commit c31ad668938e0a867933bf41fed60a68a085bb1b)
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4894
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
2020-10-27 15:44:03 +00:00
Liu Xiaodong
69b16a000f thread: fix warning caused by intr
Fixes issue: #1650

Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4869 (master)

(cherry picked from commit a3c3c0b538ebbade039f2e9a24aa875d8de09a35)
Change-Id: I8935d439fb7d1d1c896ef297baa53db0d2cd538f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-27 15:43:46 +00:00
Vitaliy Mysak
a2596f4dc2 event: add scheduler_set RPC
Add RPC that allows to change scheduler at runtime.

Change-Id: I008670f5e936bc25a0fbc923b826277d15343273
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3958
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
v20.10-rc1
2020-10-23 22:35:53 +00:00
Maciej Szwed
1b1e52cb47 event: Do not change scheduler while scheduling
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Idb8cce814b90f7a7a4a06c81507ce79da91e312c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4856
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
b309e8cefa event: Scheduler plugin for using governors
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I2edc6e31eb91fff9e1b7f0a3a05cf7cd5dc79a58
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
68493f45af event: Add DPDK based governor
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ide0ac38497bc500886e26fd219cb120113896966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4152
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
63933afd24 dpdk: Move submodule to latest commit
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I4faa1e65898f50ea9bac0de7f666efd52f6426a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4849
Tested-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
c6adf304bd event: governor implementation
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I9aaa84261144145933d227976a5f89dc89732267
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4028
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
2cffc80079 event: add rebalancing infrastructure
Features:
- does not delay thread creation
- does delay thread deletion
- singlethreaded, but can be made multithreaded.
  By being singlethreaded, we don't waste time,
  because reactors are not paused during rescheduling,
  but we do make statistics less up to date

Change-Id: Ie5a7e8569bc32b3fd4bb887804dfbc3f5c2ea858
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3899
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Vitaliy Mysak
7148f33360 event: adopt static scheduler to the new API
Move current scheduler to it's own module
and make use of new API.

Change-Id: I4928aed82603d51de01194c9650709e814f7f61b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4054
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
0d8a4af849 event: Add lw_thread helper functions required for scheduler
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ifd21adad6116d75170d5a6ff4b6d03470644feab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4338
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Maciej Szwed
2a146cd936 event: define scheduler API
Add structures and function for new scheduler API.

Change-Id: I8e379db62189eb7c36092265c137f07368650e9b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4053
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 22:35:53 +00:00
Tomasz Zawadzki
9e89645b61 CHANGELOG: add missing entries for 20.10
Added missing entries from browsing header and RPC changes.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I326e63cc14278adebd58d1dadd5e444dd2bfc554
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4853
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
75fdd05eea CHANGELOG: fix formatting and typos for 20.10
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibbe6293955e74c236f5a45936aadfcac9879f4fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4852
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
078ef4e5e6 CHANGELOG: move command log entry to correct SPDK version section
Patch introduced during SPDK 20.10 development put this addition put
the changelog entry into SPDK 20.07 section.
(198a3ad) nvme: if no command set is requested, use the most reasonable default

Move it to the right place.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0f3ddd1a551c8ba3ad2e272dea82ab2179063599
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4851
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Tomasz Zawadzki
c7df7a4ce5 CHANELOG: alphabetize 20.10 section
This patch does not add/modify/remove entries.

All sections are put in alphabetic order and lower cased.
Some of the repeated ones are consolidated.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5691f2de1e8de5efa00002361904bd63bddf2ac4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4850
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 18:24:12 +00:00
Liu Xiaodong
147000aa7b interrupt_tgt: add lvol, blobfs and basic bdevs
Logical volume, blobstore and blobfs can directly run in
interrupt mode.
Some bdev modules don't have pollers, so they can be directly
runnning in interrupt mode.
interrupt_tgt can create lvol bdevs and some other type bdevs

Change-Id: I9ad068393c6ff31bb3aafb3e58a41b0e0aeec9af
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4276
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
54a7d73ad1 nbd: apply interrupt
If interrupt mode is set, related poller functions will be
registered to interrupt_handler instead of poller.
interrupt_tgt can run IO with linux nbd.

Change-Id: I39ecf1efa10be76419fb0d25713ea457a5a53b37
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4274
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
ad8c05dc7c bdev_aio: apply interrupt mode
If interrupt is set, related poller functions will be
registered to thread interrupt  instead of poller.
interrupt_tgt can create aio bdev now.

Change-Id: Id777a55c8b7dc7a220f94ab837f7039ecea4fcb4
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
d8618de0df interrupt: add interrupt_tgt app
interrupt_tgt is similar with vhost_tgt, but it is constructed by
libaries supporting event interrupt mode with interrupt functions.
Some basic bdev libraries are linked in it.

"interrupt_tgt -E" will run the app in interrupt mode.
At start, it can work with SPDK RPC methods.

Change-Id: I3a9b253a4657758fc670af835e5bed5e53f412dc
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4271
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
4bf6e4bb32 interrupt: apply fd_group in thd and reactor
Each reactor and each thread is assigned with one fd group.
At the same time, each thread is treated as one interrupt source
registered into its corresponding reactor.
The egrp function for reacotr is the only block point waiting
for events.

Change-Id: Id092e66591b07b445342f8ae16dc218d28887427
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4269
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
7d19e50a54 reactor: extract reactor_post_process_lw_thread
Change-Id: I2761db384f78529525b7f1eb3c9c959a7b885ede
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4756
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-23 16:23:48 +00:00
Liu Xiaodong
c5d1e7f171 interrupt: new fd_group in lib/util
Interrupt is aiming to support interrupt
mode in SPDK application framework.
fd_group is implemented by epoll on linux.
fd_group can be inserted into thread and reactor
to do epoll_wait internally in order to avoid polling

Change-Id: I9077ad648a97fbd68fd46f43de2e16440bedab0b
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4268
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 16:23:48 +00:00
Maciej Wawryk
4ffb15c133 test/vhost: fix shared.sh test to run in vagrant box
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6d255ce3250b97895bd2bb02256f92e14a3c4245
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4666
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:20:08 +00:00
Tomasz Zawadzki
02d4c2b32a ut/iscsi: remove unused legacy config files
The tests did not use the legacy files anyway.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Icffedb7cc7a8574f0f801883b2d9655515cab687
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 16:12:18 +00:00
Michal Berger
dec533bf17 pkgdep/git: Fix git version check
f62834a6bb forgot to actually retrieve version string. Also, use
common function for comparing the strings.

Change-Id: I022ab54ea5b4edf0b596776dd35ca8637506ee46
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4833
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Michal Berger
62df2c1e81 pkgdep/apt-get: Add libpmemblk-dev package
libpmemblk-dev - provides /usr/include/libpmemblk.h (--with-pmdk)

Change-Id: Ie92dc1006cde972e7f77630577de18a016f4b3be
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Michal Berger
22213bb777 pkgdep/apt-get: Move ubuntu-dependent checks to os/ubuntu
Change-Id: I11b4bb2a0db20e6f86749cd951221a3697a72ed6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4815
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-10-23 14:28:50 +00:00
Karol Latecki
17f3add0a4 autobuild.sh: exclude custom dpdk from scanbuild path
Change-Id: Icc668defa2670561f1160a78f68e76452e0862fc
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4774
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:43 +00:00
Karol Latecki
b7e4207b29 autobuild.sh: cherry pick dpdk fix for disabling qat_asym driver
Use the same fix that we already have in spdk/dpdk
repository:
c2c273d5c8

Change-Id: I136d3e90a048c039cb101a190f10dabd043875ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4828
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:43 +00:00
Ben Walker
2ec88c4f8c nvmf: Don't fail a property set command with invalid bits
If the initiator sends a property set command to a valid register, but
the value contains bits that are reserved by the NVMe specification,
don't fail the whole command. Just log an error message.

Previously, any valid bits set would take effect but then the command
would also fail. That confuses the initiator and thinks it must retry
the full property set.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I566bc68c4469b9f41c69902e276f825c86683075
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:35 +00:00
wanghailiangx
6eca984a77 example/bdev/hellow_word: Drop .ini config in favor of json
Change-Id: I019a58860cb020e6a4071efb898b513a11c44554
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4846
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:23 +00:00
Niklas Cassel
6e231f59af examples/nvme/identify: remove exit_and_free_qpair()
Currently, error handling code in examples/nvme/identify frees the qpair
before calling exit().
However, since every SPDK process starts off by triggering a reset
(in state NVME_CTRLR_STATE_INIT), which causes all the qpairs to be
deleted, there is not really a reason to do a nice cleanup on error.

Additionally, other examples, e.g. examples/nvme/hello_world does not
do a nice cleanup on error, so it is inconsistent if identify does so.

Proper freeing of the qpairs is still done in the normal case where
we don't get any errors (in this example, and in other examples).

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I0db612934464e6e84eaf466c12eb35eaa396e511
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4838
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:09 +00:00
Niklas Cassel
6915ad7c34 examples/nvme/hello_world: call exit() on read error
The printout in read_complete() says "Read I/O failed, aborting run",
however, execution still continues, and prints and prints the buffer
where the result was supposed to be stored.
Since we got a read error, we will not have gotten any data stored
in the buffer, so it is simply confusing to print it.

Calling exit() in the read_complete() error handling code code also
makes it consistent with the error handling code in write_complete()
and reset_zone_complete().

Signed-off-by: Niklas Cassel <niklas.cassel@wdc.com>
Change-Id: I6e542758c57efc376f921857c2cdb24e252d49c6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4837
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-10-23 13:47:09 +00:00
Simon A. F. Lund
8a1d6f446d examples/nvme_fio_plugin: add initial support for ZNS
This adds initial support for ZNS by aligning the NVMe spec. defined ZNS
structures and commands with the fio Zone representation and
implementation of the following io-engine functions:

get_zoned_model() / spdk_fio_get_zoned_model(), when namespace is ZNS
and the Zoned-Command-Set is enabled, then this function informs fio
that the device is ZBD_HOST_MANAGED.

report_zones() / spdk_fio_report_zones(), submits a single
zone-mgmt-recv and waits for its completion, converts the spec-defined
zone-descriptors to the fio ZBD_ZONE representation and returns the
number of zones in the converted report.

reset_wp() / spdk_fio_reset_wp(), submits multiple zone-mgmt-send,
covering the range [offset, offset+length] and waits for their
completion.

Four helper-functions are added to assist in the above implementations.

get_fio_qpair(), this helper is added to retrieve the namespace matching
the given fio-file, ensuring that management commands reach the correct
namespace.

spdk_fio_qpair_mdts_nbytes(), this helper is added to assist
report_zones() retrieve the zone-report within the bounds of the
maximum-data-transfer of the device.

The functions pcu() and pcu_cb() provide a means to submit
management-commands and waiting for their completions. They are needed
since, although mgmt-send/recv are IO-commands in the context of NVMe,
then for fio they are not part of the regular queue/event/getevents but
utilized in a synchronous/blocking manner.

Note, in the fio-zone-representation, then the start/len/capacity/wp
fields are in units of bytes, whereas the corresponding values in NVMe
are in lbas/sectors. It is worth noting as the offset <-> lba
conversions do not take NVMe configurations with extended-lba format
into account. Thus, the implementation is initial support for ZNS as
more work is needed to support pi/extended-lba configurations.

Note, a guard FIO_HAS_ZBD checks for the required io-engine ops version
and indirectly testing for available of fio Zone representation by
testing for a macro introduced in the same fio-release as the required
fio Zone representation.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: Id3d1d61a52db2e55019032c724197df4d559271a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4836
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-10-23 13:47:01 +00:00
Simon A. F. Lund
54219afe5a nvme/spec: add accessors to, and commented members of, zone-descriptors
The struct-accessors are added and named matching the fields defined in
the spec. to be used by the fio_plugin/nvme and other consumers of the
driver-layer. Comments to be consumed doc-generators as well as human
readers of the header-file.

The identify example is updated with the change.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I8d6cb82e095c5dcbc06fe892e17ce83dc0062735
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4835
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:01 +00:00
Simon A. F. Lund
6fafaa5dc6 nvme/spec: add ZNS zone states and zone types
Added to be used by the fio_plugin/nvme and other consumers of the
driver-layer.

Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I945b0aa93c675ad381ce0a83883c41688b549aee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4834
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 13:47:01 +00:00
Karol Latecki
c98d7a4adf autobuild.sh: cherry pick dpdk fix for freeing pci devices
Change-Id: I57e380508da69b9c4d7d5219c81a550816966b4b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4732
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
7ba6712604 autobuild.sh: fix ipsec installation for custom dpdk
- Enable shared objects
- Lininkg to libs for meson done with -Dc_link_args
- append lib path to LD_LIBRARY_PATH in case ipsec
  is needed in later build or test steps
- Save LD_LIBRARY_PATH from autobuild.sh and pass it
  to autotest.sh. This is needed for crypto tests
  using ipsec-intel-mb library.

Change-Id: I047c6d3187ea42de8790181428d18596b195ff2e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4730
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
b7b4348233 autobuild.sh: preserve LD_LIBRARY_PATH in sudo call
Apparently LD_LIBRARY_PATH is not preserved even with
"-E" option. Pleaes see sudoers(5) manual as a reference.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7d213abc4322eed3e01199416d5c7880965bdbe2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
d9b311e7fd autobuild: fix isa-l installation for custom dpdk
pkgconfig path must be set for meson/ninja installation
as described in:
https://doc.dpdk.org/guides/compressdevs/isal.html

Change-Id: I9dd01b576c4468c42241ede65781e5d50bd52bbd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4700
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 10:14:12 +00:00
Karol Latecki
507f8ae3ea lib/env_dpdk: update crypto and compress library names
Post DPDK 20.08 crypto, isa-l and qat shared object
libraries had their names changed. Detect which library
names are available and use them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I8c9b9b24f9232f303b99c959754a7f7550ffdf90
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4780
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-10-23 10:14:12 +00:00