Commit Graph

15247 Commits

Author SHA1 Message Date
Michal Berger
24e0f403b8 test/make: Fix LTS tag lookup
Replicating 00afb74669.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I19049e35ad959a34342cd714d805db6bdd933ea4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7814
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:11:05 +00:00
Michal Berger
6c1a1a3dca scripts/rpc: Make sure address argument is properly interpreted
In case the addr argument was not an existing unix socket file the rpc
client would consider it to be an actual ip address. As a result
connect() would be called with improper set of arguments. This could
cause the rpc.py to block for undesired amount of time until connect()
finally decided to return (seen on some fedora33 builds).

This was affecting sh wrapper functions like waitforlisten() which
use rpc.py to determine if given app is ready to be talk to blocking
execution of the tests for way too long then intendent.

To avoid such a scenario determine the format of the address and use
routines proper for given address family.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iaac701d72c772629fa7c6478ff4781b0c5d485d5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7777
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-05-24 10:11:05 +00:00
Michal Berger
ef68667a82 pkgdep/git: Remove 20.01 release workarounds
There's already a new LTS release in place, hence these workarounds
are not needed anymore.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idf597a5cf8da18d75f2630d59a277d4545fadc0f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7987
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:10:41 +00:00
Michal Berger
f8d6541d04 freebsd: Replace use of \s with [[:space:]]
In latest 13.0 release of freebsd sed is not accepting \s anymore,
hence it needs to be replaced. Note that only parts of the repo
which are touched by freebsd during the tests are updated.

Details https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=253893

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I217208511af8f98e7033f8f24a343b5ca9c48825
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7986
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:10:41 +00:00
Michal Berger
022ecaa7f1 vm_setup: Don't fail on freebsd if etc_os-release package is missing
This package is not available under latest 13.0 release, however,
the /etc/os-release we are looking for is already installed there.
For older versions, and in case this package is missing, we still
can determine that we are running under freebsd as part of the
fallback check to uname().

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6a04eb66392d91899a4b55d4eb508bef140e924a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7985
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-24 10:10:41 +00:00
Ben Walker
15ae31fb0c nvmf: Rearrange spdk_nvmf_requset for cache line locality
Get all of the hot stuff to the first cache line.

* Shrink the xfer enum to one byte (it only has 3 values).
* Pull out the dif enabled flag form the dif structure so it
  can be access separately
* Rearrange the members

Change-Id: Id4a2fe90a49c055a4672642faac0028671ebfae9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7827
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:10:19 +00:00
Ziye Yang
d33f44866d rbd: Fix the name comparison bug.
Checked the definition of strncmp. If substring s1
is found in strnmp(s1, s2, len), then it will return 0.

For the len value, it is better to use strcmp. Otherwise,
if s1=cluster1, s2=cluster & len=strlen(s2),
strncmp will return 0. But they are two different strings. For
cluster names, they are different.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I15a06184d834cd1567b329d0322cd6bdea6fee4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7991
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-24 10:09:44 +00:00
Tomasz Zawadzki
1180c390c1 lib/thread: fix iterating over paused pollers
(cd83ea4a)thread: Add SPDK internal APIs spdk_thread_get_first/next_active/timed/paused_poller()
Patch above by mistake iterates over active_pollers list
for function that lists paused pollers.

Fixes #1947

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1b69d942675f34f5f046ec46feacc8d81d89f015
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
2021-05-24 10:08:34 +00:00
Tomasz Zawadzki
ceaa0c7fa9 lib/blob: complete multiple persists
When blob persist starts, there can already be multiple
of such requests pending. It is possible to complete
a set of persists at once, if blob state after their
execution would be the same. This is the case when
persists are already pending when a particular persist
request is started.

This patch implements such mechanism by introducing
persists_to_complete queue, containing entries that
were previously queued up before starting the current
persist request. If there are any entries in this queue,
further requests are put into pending_persists.

When first request from persists_to_complete is persisted,
completions are issued for all requests on that queue at once.
If at that point there are any new entries on pending_persists,
all of them are put into persists_to_complete. Persist process is started
again with the first request from that queue.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I10063e55d6f821b1863de016d3148da6a719a422
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7643
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-24 10:08:00 +00:00
Rajarshi Chowdhury
119158cda8 configure: Added --disable-apps option.
Together with options to disable examples, tests and unit-tests
this will enable us to build SPDK without linking any executables.

This is useful for packaging SPDK in applications - especially
those using own ENV.

Fixes #1877

Signed-off-by: Rajarshi Chowdhury <rajarshi.chowdhury@oracle.com>
Change-Id: I0467ec497d2abe2fad40cdb4ba723a240bee46d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7462
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 10:05:34 +00:00
Michal Berger
7bea0a074f scripts/vagrant: Create user netdev for openstack network
The openstack tests require to have one interface available under
specific ip, 10.0.2.15. This ip is a first lease VM receives from
the user network which is preconfigured in libvirt environment.

Use this instead of defining completely separate network which
by the very default will serve its own DHCP service as well. This
may result in a cosmetic issues (e.g. two ips, static one and
dynamic one on the interface) of having inconsistent config on the
VM depending on how the net device is configured by the underlying
distribution.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie3da26eebcfb0de668166c4a0120c65aec104540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-05-24 09:58:17 +00:00
Michal Berger
0c25e6fbbc test/openstack: Clean up the test
This is done as part of the work to fix currently existing CI job
using this particular test suite.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id5e9e8a814fb3025f192871650b8aa317f2de910
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-05-24 09:58:17 +00:00
Krzysztof Karas
994be25fe1 spdk_top: move mvhline() function call inside show_core
Moves "mvwhline(core_win, 6, 1, ACS_HLINE, CORE_WIN_WIDTH - 2);"
upwards to increase readability by placing this call in order
(to print the line in 6th row after data in 5th row is done printing)

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: Icec340231090aeb57c600f52b3c687dbc78cc4b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7959
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 09:57:47 +00:00
Krzysztof Karas
6f9d124888 spdk_top: change where sore_last_run_counter() is called
Changes where store_last_run_function() is called. Before this patch
this function was used inside refresh_pollers_tab() and it overwrote
last run counters of each poller before show_poller() function had a
chance to use them. As a result poller details window was always
showing zeroes instead of actual difference in run count between each
application loop.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I96219698f7f0b51b94ffe8c0d6bf40f73cbf8a82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7951
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 09:57:47 +00:00
Krzysztof Karas
49bc3005f5 spdk_top: move store_last_run_counter() function
Moves store_last_run_counter() function to allow next patch to
call it earlier in the code, inside get_data() function.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I1e37080ec86309bbf33442fa12cbe9dc575cc864
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7950
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 09:57:47 +00:00
Krzysztof Karas
39950bc8f9 spdk_top: reduce number of poller global structures
Deletes g_pollers_history and store_pollers_last_stats() function
to simplify spdk_top code.

Adds last_run_counter to show_poller() function to get last state
of poller's run counter for comparision with current value and
display in pop-up window.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I236fbcac31e67a617d37ba15888aeb30e3763e01
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7925
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 09:57:47 +00:00
Krzysztof Karas
081a4a0943 spdk_top: reduce number of global thread data structures
Deletes g_thread_history and g_thread_info to use g_threads_stats
across the whole application to simplify spdk_top code.
Now instead of separate struct, fields last_busy and last_idle are
being used.

get_data() function now uses local structure to get RPC data instead
of filling global one. This has been changed so that g_threads_stats
keeps its last_busy and last_idle fields unchanged.
free_rpc_threads_stats has been moved down so that in future patches,
when multithreading is implemented, there is no need to lock
g_threads_stats during RPC call.

Changes places of allocation/deallocation of g_threads_stats, since
we want to save last_idle and last_busy fields instead of zeroing them
out each application loop.

Changes show_thread() function to use local copy of threads array
instead of pointers to global struct. This is for the convenience
in the future patches implementing multithreading to avoid the need
to lock the global struct for details display.


Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I0dc87eac4c1b89fa16f14f5387d94ee176dfdf43
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7587
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-24 09:57:47 +00:00
Krzysztof Karas
22edbe9626 spdk_top: change where get_data() and free_data() are called
Move part of code with get_data(), refresh_tab() and free_data()
inside show_stats() upwards to make sure data structures are
up to date for pop-up details windows.

Delete get_data(), free_data() calls from show_thread(), show_poller()
and show_core functions.

Add data freeing right before rpc calls inside get_data() to let
pop-up details windows to use updated data before freeing it.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I0d78eb7a48b0cdff4284815afc1a214b0effd7fc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7953
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-24 09:57:47 +00:00
Krzysztof Karas
e2b6cf2f96 spdk_top: move sort_threads function
This function is going to be needed in get_data() in the next patch.

Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I9368b4567a92ca20d830c3475e3120ee691b84c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-24 09:57:47 +00:00
paul luse
eb5a111398 lib/idxd: move the crc destination * from the desc to comp record
Was using reserved field in CRC to store the final address of where
to put the result, this not legal.  Move to the completion record
and slightly re-arrange the struct to keep it at 96 bytes.

Refacorted the IO prep function so the caller can udpate both the
descriptor and completion records instead of continuing to add
parameters to the prep function for opcodes that need something
unique in the completion record.  This also allowed for a minor
fix where the prep function was returning NULL when vtophys failed
which would have indicated busy as opposed to failire.  Now we can
proprely fail that path.

fixes #1929

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ic23bc7b68bdd5757c30b7963880677f423368e20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7735
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-22 18:42:35 +00:00
Michal Berger
274f52b527 autobuild: Separate unit tests from autobuild
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If6ce4d3f7bc58855ba61b352be71dff404c87720
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6819
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-21 12:56:48 +00:00
Karol Latecki
7908736c22 autobuild: update patches for mainline DPDK
Patches stopped applying cleanly because of dpdk/dpdk changes:
7d5cfaa7508de0fd248b05effbf421a98317006a
4ad4b20a79052d9c8062b64eaf0170c16a333ff8
Needed to rebase custom patches.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1006f7f6ba21a3cee5b607cfc44adedb4c1d5830
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7903
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-21 08:04:49 +00:00
Mao Jiang
bb27314c03 test/nvme_pcie: cases for mapping io cmd
Change-Id: Icf174668e9b1361bb232db4a0cc53921423a17a4
Signed-off-by: Mao Jiang <maox.jiang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7671
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-21 08:04:39 +00:00
Karol Latecki
b686c82581 scripts/nvmf_perf: update add_listener rpc call
nvmf_subsystem_add_listener() in rpc/nvmf was modified
by 64debe0453 and the
change was not reflected in performance scripts.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7bb8c39ab225421964e37883aa7b0f824e8df683
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-21 08:04:24 +00:00
Jacek Kalwas
dd9350c7e6 build: fix missing pthread dependency for syslibs pc
SYS_LIBS var is used to generate Lib.private section of
spdk_syslibs.pc

lack of pthread can result in linking issue when spdk's *.pc are used

please note that usage of -lpthread and -pthread is not consisent
within system wide pc files

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I2dd327371d5621ea12f1e2f7abdfe6b7aa6602eb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7253
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szulik <maciej.szulik@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2021-05-21 08:04:09 +00:00
Jim Harris
887109bf9e scripts: add bpf/syscalls.bt
This script counts how many times each syscall
is called by the SPDK application.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ife5efccee73e407e1d819293786f2acdfa317f78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7713
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-05-21 08:03:27 +00:00
Jim Harris
dca7ef2759 scripts: add bpf/readv.bt
This script captures a histogram of the returned values
from the readv syscall.  It can be useful for analyzing
the efficiency of our sock/posix implementation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I910eb2e81e3416855cda7a3854982cd02476850e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7712
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-05-21 08:03:27 +00:00
Jim Harris
cbb57e0d96 scripts: add bpf/send_msg.bt
This script tracks spdk_thread_send_msg and
spdk_for_each_channel calls.  It also uses usym
to get the name of the function pointer passed to
these calls.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia0cf5b50538bd26a3e15e35cbab765fe3c327c73
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7711
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-05-21 08:03:27 +00:00
Jim Harris
af0bf0f006 scripts: add bpftrace.sh
This script simplifies execution of bpftrace scripts
for SPDK application analysis. The script should be
invoked as:

scripts/bpftrace.sh `pidof spdk_tgt` script.bt

where 'script.bt' is the name of the bpftrace script
you wish to execute.

The script helps simplify the following:

1) replace __EXE__ markers in the bpftrace script with
   the full path of the spdk executable (required for
   usdt probes);  it uses /proc/<pid>/exe to get the
   full path
2) replace __PID__ markers in the bpftrace script with
   the PID of the spdk process under analysis - this
   allows for doing things like filtering system call
   counts for just the SPDK process
3) invoke bpftrace with the -p <pid> option

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib271f26efe4ba326d951a7b30f61b22be755dda7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7710
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-05-21 08:03:27 +00:00
Jim Harris
5a1c74bfa4 usdt: add User Space DTrace support to SPDK
For now, we will keep this disabled by default,
enable with --with-usdt option to the configure
script.  Long-term we will want to enable this by
default, and only disable via configure.

Modules can include spdk_internal/usdt.h and add
probes such as:

SPDK_DTRACE_PROBE2(probe_name, ptr, val);

When USDT is enabled, these will translate to
DTRACE_PROBE2(spdk, probe_name, ptr, val).  When
USDT is disabled, these will translate to nothing.

Later patches will add some probe points to the
nvmf target, some bpftrace scripts, and instructions
for how to successfully capture data with these
probe points and scripts.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id168e2c800fa5522815a175026386319014cfdaa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7173
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-21 08:03:27 +00:00
Shuhei Matsumoto
9e1c13c5f1 bdev/nvme: Move bdev_nvme_compare_ns() up in a file
bdev_nvme_compare_ns() will be used to check if all namespaces of
one nvme_bdev are identical, and this is the convenient location.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3fa6072c1cceec53268e53bf398fa1e8f069035e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7169
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
9391bc33a7 bdev/nvme: Factor out the internal of _bdev_nvme_check_pending_destruct()
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I27d7e81fe0ecc35b374277c8e40233db2a3ed914
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7273
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
a0ea252399 bdev/nvme: Factor out reset_io completion into a helper function
This will be helpful to simplify the following patches.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I66939f2953c66582bfcb79cfe187814280e89680
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
73b4b9ad68 bdev/nvme: Put all nvme_bdev_ctrlr accesses into a place in bdev_nvme_create_cb()
When multipath is supported, a subsystem has multiple controllers and
bdev_nvme_create_cb() will create a channel per nvme_bdev_ctrlr by
iterating the list of nvme_bdev_ctrlrs, and will hold lock while
doing it.

If the code to access nvme_bdev_ctrlr is put in a place, the
following patches will be easier and smaller.

Hence reorder the code of bdev_nvme_create_cb() as a preparation.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2f2e66758c3374c678cc44bbb0116f4611c6753a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7754
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
81b4379cd8 bdev/nvme: Factor out spdk_bdev_io_complete() calls into a helper function
Factor out spdk_bdev_io_complete() calls into a helper function
bdev_nvme_io_complete().

This simplifies the code a little and will be helpful for the following
patches to retry I/Os. These are not performance critical but we
specify inline explicitly by following bdev_nvme_io_complete_nvme_status().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9fafacfd8571c037c3bc34382c251317309da334
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Mellanox Build Bot
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
e4ec38c336 bdev/nvme: Factor out spdk_bdev_io_complete_nvme_status() calls into a helper function
Factor out spdk_bdev_io_complete_nvme_status() calls into a helper
function bdev_nvme_io_complete_nvme_status().

This simplifies the code a little and will be helpful for the following
patches to retry I/Os. Specify inline explicitly to avoid performance
regression.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1ac451486e1c6a4401842490411e986fac191d59
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7484
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
30d17fff30 bdev/nvme: Delegate processing -EBUSY of bdev_nvme_reset() to its caller
bdev_nvme_submit_request() calls spdk_bdev_io_complete() with failed
if bdev_nvme_reset() returns negated rc other than -ENOMEM.

So let bdev_nvme_submit_request() process it.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2569634ff0f18fb433cb685de1366e43abf5a9fe
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7524
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
5a6cd5b31c bdev/nvme: Inline _bdev_nvme_submit_request()
This is called only in a place. Inlining this into bdev_nvme_submit_request()
will simplify the following patches by removing unnecessary cast.

Besides, use -ENXIO if I/O path is not found. This will be better than -1.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib6c38f89db1c1e651941aad18d31dd0891f380de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7871
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-21 08:02:50 +00:00
Shuhei Matsumoto
b2c0e3761b bdev/nvme: Consolidate exit paths of get_buf_cb()
I/Os will be retried if spdk_bdev_io_complete() is called with
SPDK_BDEV_IO_STATUS_FAILED. To do it easier, consolidate exit paths
of bdev_nvme_get_buf_cb().

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0a67b88a107d616c5a5b0fc5ff963ad1402f5651
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7487
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-05-21 08:02:50 +00:00
John Levon
c41508b7e2 libvfio-user: update submodule
Update libvfio-user to the current version, updating the client for the relevant
changes.

Signed-off-by: John Levon <john.levon@nutanix.com>
Change-Id: Ic64ace08ac0c7e9676f04f8d1f47a9c0388a2652
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
2021-05-21 03:01:21 +00:00
Krishna Kanth Reddy
b73d3e6894 test/nvme: Simple Copy Command takes nlb as a 0's based value
Signed-off-by: Krishna Kanth Reddy <krish.reddy@samsung.com>
Change-Id: I10a33d6478ce10fd2786c8eca2f179a9809764dc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7973
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 20:15:51 +00:00
Maciej Szwed
3703ebc710 doc: Add information about shutdown_cb
Fixes GH 1539

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I5256a53c619b13951382fcc0439a157aa29b61aa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:22:19 +00:00
yupeng
5cc565999e nvmf: support NVME ACRE feature
Support ACRE (Advanced Command Retry Enable) feature. Currently set
all crdt to 0 and only select crdt[0] (crd=1) when the IO has any
error.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: If7bc30f91f5b2d0839002dead17188a4b3a52d5d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7885
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-05-20 15:18:52 +00:00
Alexey Marchuk
103f5c62f5 nvmf: Remove qpair from poll group when it is deleted
That prevents nvmf target from starting to destroy poll
groups prematurely

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I833f6198ef0e3083fdadf70dd3b62844c905aceb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7881
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:18:29 +00:00
Evgeniy Kochetov
d8eb6957a0 ut/nvme: Add tests for NVMe controller namespace handling
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Id2d3d1a15df22512f8d0d4414b5b3d35e79149f4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6505
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
f0e248586e nvme/ctrlr: Always access namespaces via getter function
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I16e327c0c0485057dc90a87cae316c4d6b62720d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6504
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
65ff07719d nvme/ctrlr: Retrieve active NS list before NS construct
This patch changes the order of IDENTIFY_ACTIVE_NS and CONSTRUCT_NS
controller states. It is required to further improve memory management
for namespaces by allocating memory only for active ones.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Ie540442b1bd9e897afcbaa4319c139109dd0c515
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6503
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
75ddfd6511 nvme/ctrlr: Handle namespaces construction failure
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I3b14e2dce3bd4b2ce42e3946592cfde9d4300f28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7361
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Evgeniy Kochetov
aebbce2520 nvme: Refactor active namespace list retrieval
Previous implementation allocated memory just once at the beginning of
active NS list retrieval procedure. It allocated memory for maximum
possible number of active namespaces, i.e. 'cdata.nn'.

This patch changes allocation logic. One page is allocated at the
beginning. If more is needed, reallocation is done with one more
page.

This patch also removes SPDK_MALLOC_DMA flag from allocation since we
don't do RDMA directly into this buffer.

Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: Iaa80c4d70c54daaf71dcbf755c63a01a1d83b772
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6502
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-05-20 15:17:41 +00:00
Michal Berger
fead05c025 test/nvmf: Rework the traps
Declare single trap from nvmftestinit() instead overriding main trap
depending on the use-case (veth vs phy).

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I23c66fc5728510d06eee9837843f0734b2d03dcf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7850
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-05-20 15:16:21 +00:00