Commit Graph

1504 Commits

Author SHA1 Message Date
Michal Berger
3220a05311 pkgdep: Make sure gem is installed on FreeBSD prior installing mdl
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I2ce2ce7294eff1c0cd66a5bf101f37d64ee4645d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9165
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-08-24 07:18:44 +00:00
Karol Latecki
4ce1735929 scripts/nvmf_perf: enable BPF traces
Add option to enable gathering BPF traces while
running the test.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9ea317a37c3638def40a525eca06f2d8325a74c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8644
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-23 08:48:05 +00:00
Karol Latecki
348ff890ad scripts/bpftrace.sh: redirect output to file
Allow redirecting output to file. Using simple
"if" for an environment variable, as script arguments
number is variable and using getops seems like an
overkill.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7e219bb1fd6377bc20c44b76cec3685b32e9316c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8643
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-08-23 08:48:05 +00:00
Maciej Wawryk
6792ae70f1 scripts/pkgdep: add ruby-devel to INSTALL_DEV_TOOLS
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I367b42a84f3af4ca456ea8c2e1263a05a92bbae7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9145
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-08-19 11:24:07 +00:00
Karol Latecki
f60b99b168 scripts/gen_nvme.sh: remove unnecessary wrapper function
create_json_config_with_subsystems contents does not
need to be in a function. Moving it to main simplifies
--with-subsystems condition.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1414a209cc666ebb4639fc25133168482bd96c97
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8775
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-08-10 08:14:30 +00:00
Karol Latecki
65b3c34b14 scripts/gen_nvme.sh: make bdfs var local to a function
No need to keep "bdfs" variable as global variable.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I655ed9f1514f13a6460292c1298ef08f904157fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8774
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2021-08-10 08:14:30 +00:00
Karol Latecki
61f3c72cd2 scripts/gen_nvme.sh: introduce getopts for handling options
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I21b7798e1004b227bcd7f62ee8867667c67dab7c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8773
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-08-10 08:14:30 +00:00
Maciej Wawryk
83440f1ded scripts/pkgdep: move markdownlint install to pkgdep scripts
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If2cc3cf797602f90b26407d35f64840bb2d2697c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9108
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2021-08-10 08:13:00 +00:00
wawryk
2f5c602574 markdownlint: add -g option to check_format and fix mdl errors
-g option using only files known to git, that allow us
to avoid errors from submodules

Also add check if mdl is installed, and gracefull info if not.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib6e1920774ffca81e62d9abebc8d8b4548feb519
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9086
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-10 08:13:00 +00:00
wawryk
12fcbc9b94 markdownlint: enable rule MD013 - line_length
Fixed all MD013 errors

Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: I24846414ae6283e27a17caced16ac798a7e93018
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8938
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-05 07:42:35 +00:00
wawryk
e4d23dc757 scripts: add markdownlint to check_format
Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: I0f5a351fcdbf417a18e388432088c8bea6f12143
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8883
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-05 07:42:35 +00:00
Michal Berger
637d9e6069 scripts/setup: Try to cleanup any lingering spdk_pci_lock files
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I853e278756e53e40d7faf5eb8392057653ca649f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9065
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-08-04 14:11:28 +00:00
Ziye Yang
6434aaa274 scripts/pkgdep: Add the libpmem object library dependency.
Since we will create new bdev relied on libpmem object
library, so need add it.

Change-Id: I7ddcc694fd2ec5a0536abc9813b13fa429fcf068
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9049
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-08-04 11:05:30 +00:00
Jim Harris
d1e67b8b1b bdev/nvme: add fabrics timeout to attach_controller RPC
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I42f6729475fd767087fb0a0c6e30d409672c3b99
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2021-08-02 09:05:41 +00:00
Jim Harris
d9cea939a9 rpc/nvmf: fix typo
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8bb9afd1f3e364105ea1c50d665183b949202d2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8982
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-07-30 08:22:34 +00:00
Karol Latecki
84867c8eaf scripts: remove gdb from Python requirements file
This package is no longer available in PyPI.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3e68b03eabeb4aff14453c90846fd03452c93ae2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8956
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-28 16:19:35 +00:00
Yuri
38a30da5bd nvmf: accepts new parameter to set poll groups mask
Parses and verifies hexadecimal cpu bit mask specified by the user.
Added verification to check for cpu cores range, making sure poll groups cores
assigned within the range of cpu cores allocated for the application.

RPC nvmf_set_config now takes an argument to configure ‘poll groups’,
a new parameter for NVMf subsystem. This parameter sets a CPU mask
to spawn threads which run an event loop for a ‘poll group’.

Change-Id: Ied9081c2213715ec94de00a8b37153730b8ac2ed
Signed-off-by: Yuri <yuriy.kirichok@hpe.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5611
Community-CI: Mellanox Build Bot
Reviewed-by: Matt Dumm <matt.dumm@hpe.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-28 08:06:00 +00:00
Alexey Marchuk
ec1b78dbd7 socket: Remove deprecated enable_zerocopy_send
This parameter is still part of API spdk_sock_impl_opts
structure but it is not used. Keep it to support ABI
compatibility since it is located in the middle of the
structure and removing it may break socket opts initialization
or parsing.

Change-Id: Ib641ad7d965d68bc9ebb65dba531408d88cf6fa1
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8914
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-23 10:30:25 +00:00
Tomasz Zawadzki
29b3da96ba socket: removed deprecated zerocopy RPC options
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I748c3f008a6ba1b969acecdd593940077800091b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-07-23 07:10:03 +00:00
WindYu
49ffdc18af bdev/nvme: Add support to get the health log for NVMe device
Add a new RPC method to get the health log of a certain NVMe
device.

Below is the example:

./scripts/rpc.py bdev_nvme_get_controller_health_info -c Nvme0
{
  "model_number": "INTEL SSDPE2KX020T8",
  "serial_number": "BTLJ72430ARH2P0BGN",
  "firmware_revision": "VDV10110",
  "traddr": "0000:08:00.0",
  "temperature_celsius": 33,
  "available_spare_percentage": 99,
  "available_spare_threshold_percentage": 10,
  "percentage_used_percentage": 2,
  "data_units_read": 1013408619,
  "data_units_written": 346792685,
  "host_read_commands": 30457773264,
  "host_write_commands": 18949677715,
  "controller_busy_time": 4979,
  "power_cycles": 49,
  "power_on_hours": 31114,
  "unsafe_shutdowns": 18,
  "media_errors": 17,
  "num_err_log_entries": 19,
  "warning_temperature_time_minutes": 0,
  "critical_composite_temperature_time_minutes": 0
}

Change-Id: I53125d2ec16cb36011571473430aece99167b803
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8806
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-07-22 13:46:50 +00:00
wawryk
cb19173375 scripts/nvmf: configure adq traffic classes before configuring target
Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibb2c943b555ce4304ff0244af16cc93ab53482cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8837
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-07-21 19:44:13 +00:00
Jonathan Teh
153f03cf42 bdev/nvme: Add bdev_nvme_reset_controller rpc
Implement bdev_nvme_reset_controller rpc, which allows the NVMe
controller to be reset over RPC. Implement bdev_nvme_reset_rpc()
which starts the reset of the controller and returns the result of
the controller reset via the callback function after it completes.

Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: Id98d5e56feb315b7e44e9bb5e5f495e9b1cd1de0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-21 07:45:38 +00:00
Michal Berger
1963d3185c pkgdep/freebsd: Use globbing to match on package names
Some packages include versioning in their name which is not very
friendly in case given package is upgraded - the name of the package
will always change in such a case. To avoid that, use globbing to
match on package names in a more robust way.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I3e23fc8a06f93471f47835e945c24c4a95f80795
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8741
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-20 08:09:36 +00:00
Karol Latecki
8dace9e26c scripts/nvmf_perf: reuse get_connected_nvme_list()
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1ccc18e9a025faea87ef4328bd79be7c3574ebf6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
2021-07-20 07:59:18 +00:00
Karol Latecki
0574f36dea scripts/nvmf_perf: improve error handling
Add exceptions to results parsing and fio run functions
to better inform what's the cause of the failure.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I94e51cb8f77515f7d26a75501011d6111820ae56
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8742
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-20 07:59:18 +00:00
wawryk
2c8baa4685 scripts/nvmf: add io_uring engine option for Kernel Initiator
Enable "kernel_engine" option for Kernel Initiator
class to enable ability to use io_uring engine on
initiator side.

For NVMe-oF this has limitations as apparently it's
not possible to enable polling on the initiator side,
which in turn makes fio "hipri" option also not
possible to use.

Adding #TODO sections with notes in case this is
later fixed in Kernel NVMe driver.

Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I31da52946692015237263abc77d8425b5eae5b98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8134
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-20 07:59:18 +00:00
wawryk
7cb301c042 scripts/nvmf: replace deprecated enable_zerocopy_send by enable_zerocopy_send_server
Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: I98ee388059e522946f4111ab4c414208c8a68daf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8798
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-19 13:50:40 +00:00
Evgeniy Kochetov
2716eb02f1 rpc: Fix disable_zerocopy_send_server in sock_impl_set_options
Signed-off-by: Evgeniy Kochetov <evgeniik@nvidia.com>
Change-Id: I27cfa20c59383e5689dd1795f1737b1676c1bb12
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8799
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-19 08:05:43 +00:00
Matt Dumm
67fbfdd8af nvme: rpc interface updates to manage 'timeout_admin_us' value
Signed-off-by: Matt Dumm <matt.dumm@hpe.com>
Change-Id: Ib97371924e56275ba9b845725d1d42682fd0f94a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8163
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-14 09:15:20 +00:00
Ziye Yang
e5d5b3ebef idxd/rpc: Revise the rpc function to use kernel or user driver
This patch is used to add the support for users to configure
use kernel or userspace idxd library.

Change-Id: Ie159b897bc9595894ad8f333168efaea6c2a3d78
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7332
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-07-13 17:22:30 +00:00
Pawel Piatek
2f0fdf0e48 scripts/nvmf_perf: support for arguments
Support for command line arguments.
This commit will add following ones:
* Configuration file
* Results directory
* CSV results filename
* Usage

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I7166c792661b2fc7ed3b5547f2c3b8faad4e86b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-07-13 09:00:24 +00:00
Pawel Piatek
99465fbd3a scripts/nvmf_perf: remove unused variables
Fix some of pylint W0612 and W0613 findings.

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I34e72bf1e07787689025756c6ddf429711cd250b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8593
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-07-13 09:00:24 +00:00
Ben Walker
e1d06d9954 net: Remove library
Now that we've deprecated the RPCs for a release, we can remove the whole
library.

Change-Id: I0f1a357fcfb3404efac39aa021928841c2f22ff1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4305
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-13 08:57:58 +00:00
Michal Berger
8770e80bda scripts/rpc_http_proxy: Print to stdout POSTed json rpc calls and responses
It's useful to determine what's being sent out to the server. This
may be relevant for e.g. openstack tests where currently it's hard
to see what tempest|cinder is actually doing, SPDK-wise, during the
tests.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie32781621317dd65e3f6fbcfd5110e7dac337f04
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8489
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-07-09 19:19:48 +00:00
Konrad Sztyber
597688b2b1 scripts/trace: use ijson to parse the traces
Since the trace files can get very large (several GBs), parsing them
using python's json module might require an unfeasible amount of memory,
as it needs to load the whole file first.  The ijson [1] library
provides interfaces for parsing files iteratively, only loading a small
portion of a file at a time.

It requires the input JSON to have the tsc_rate and the definitions of
the tracepoints listed before the tracepoint entries.  It's not a big
deal, as this is the way `spdk_trace` generates it, but it's worth
noting, as passing that file through something like `jq -S` might make
it unreadable to the trace script.

[1] https://pypi.org/project/ijson

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I03c0c3fb47091da615a3978b8d63edf4d876b811
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
2021-07-07 09:43:37 +00:00
Konrad Sztyber
e61fbe91e7 scripts/trace: qpair annotations
This patch adds support for annotating qpairs.  The raw pointer values
are replaced with a list of various properties of a qpair collected by
the bpftrace script.

A line like this:

`0 2856282624.000 RDMA_REQ_NEW id: r3256 qpair: 0x26ba550`

becomes (the line was only broken here):

`0 2856282502.669 RDMA_REQ_NEW id: r3256 qpair(ptr=0x26ba550, thread=2, qid=0, \
subnqn=nqn.2016-06.io.spdk:cnode1, \
hostnqn=nqn.2014-08.org.nvmexpress:uuid:729783b4ab38485d8d767b7741108a8)`

To annotate a trace, one first needs to enable and record the DTrace
probes:

```
$ scripts/bpf/trace.py --record `pidof spdk_tgt` > bpftraces
^C
```

Of course, the probe events are only recorded when the script is
executing, so in order to generate the annotations properly, it must be
started before the annotated objects are created.  For instance, for
NVMeoF, it needs to be running before a connection is made.

After the BPF probes are recored, the traces can be annotated:

```
$ build/bin/spdk_trace -p `pidof spdk_tgt` -s spdk_tgt -j | \
	scripts/bpf/trace.py -b bpftraces
```

For now, the script only annotates traces from the rdma module, as it's
the only one with tracpoints recording qpair pointers now, but it could
be extended to support more tracepoints.

Similarly, more objects could be annotated in the future by extending
the `SPDKObject` class and defining additional DTrace probe points.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I2225f4b1ab266b41af561b5e7d942411cd0c13c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-07 09:43:37 +00:00
Konrad Sztyber
01ae68f71d scripts/trace: parse and generate usdt bpftrace scripts
This patch introduces definitions responsible for generating bpftrace
scripts and parsing its output.  That output will be used in subsequent
patches to provide annotations for SPDK traces.

The script has a hardcoded set of probe points that are used to generate
the bpftrace script.  They're also checked against the probes present in
code to sanitize them and make sure that they're in sync.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I1b8c95e1a035bd7affed2c44b056828a5da94abd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-07-07 09:43:37 +00:00
wawryk
8439c53b94 scripts/vagrant: add fedora34
also remove out of support fedora31

Signed-off-by: wawryk <maciejx.wawryk@intel.com>
Change-Id: I2ec224a3fb3849eb62ba46a01aa74c62926174f8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8461
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-07-05 14:36:34 +00:00
Michal Berger
642b89f496 check_format: Fix git-grep pattern
Use extended regexp to resolve any potential ambiguity with matching
on '('. This should fix the following failure as seen on the CI:

fatal: command line, '^SPDK_RPC_REGISTER\(': Unmatched ( or \(

Also, since errexit doesn't see failures inside the process
substitution make sure we return from the function with a proper $rc
in case git fails early on.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia03095e9cc8cf11602dafb5bef28265abb485704
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8577
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-07-05 14:35:50 +00:00
Michal Berger
9d48662103 check_format: Ignore shellcheck's SC2128 and SC2178 directives
These are currently prone to false-positives especially in terms of
scoping. Consider the following:

foo() {
	local bar=(42)
	echo "${bar[*]}"
}

bar=43
foo
echo "$bar"

Some versions of shellcheck, including the latest, 0.7.2, complain
about $bar being reused as a plain string here. This is incorrect
since foo() holds its own copy of bar[@] hence the assignment which
takes place outside of it doesn't affect its content.

SC2178 can be mitigated be reversing the order of declaration:

bar=43
foo() { ... }
...

but the SC2128 still remains.

Currently, in our code majority of these warnings are coming from
false-positives due to initial source'ing which most of our test
scripts do (e.g. they fetch a function where local bar=() is used
and in the test itself $bar happens to be assigned a plain string.
This is still valid code).

To mitigate, disable these directives untill shellcheck is capable
of properly interpreting scoping when checking them.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ifbde973eae6e261d79e1c340eb28644bce5f4e45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-30 22:43:05 +00:00
Michal Berger
8c176db2ca check_format: Fixes for shellcheck's SC2030 directive
SC2030: Modification of var is local (to subshell caused by pipeline)

This directive is raised since the for loop (or rather its contents)
is piped through to another process. In this context it's not harmful
since we are interested only in sending the stdout but since we operate
on variables that happen to be local to the process that handles the
loop shellcheck sees that as a potential issue. To make it happy
remove the pipe and sort pci addresses prior running the loop.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I835c5ac1da9012129c5d01d62880307f70caab1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8501
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-06-30 22:43:05 +00:00
Michal Berger
36f5f0df17 check_format: Fixes for shellcheck's SC2155 directive
This is a first commit in the series addressing potential Bash issues
as discovered by the latest shellcheck release (0.7.2, shipped with
the very latest fedora33 and fedora34). The goal is to either fix,
locally or globally disable given directive(s).

SC2155: Declare and assign separately to avoid masking return values

Simplify the setting of the variable pointing at the root of the repo.
Also, keep it consistent with the rest of the scripts and declare it as
$rootdir.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I63e0b1a85ce16f7983e9ba6dd985046e8a39a650
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-30 22:43:05 +00:00
Jim Harris
79b6771f61 scripts: add escape in check_json_rpc string
On my system, the git grep for ^SPDK_RPC_REGISTER(
fails because the ( is unmatched.  It seems we should
escape it, although clearly our CI systems don't seem
to require it.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6412e27e79488616743f76b2d9eac8f62b996078

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8393
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-06-30 19:23:50 +00:00
Konrad Sztyber
b0d3f29a33 scripts/trace: added script to annotate traces
This script will use the output generated by the spdk_trace app (JSON) and
replace some of the raw pointers with the information gathered through
the DTrace probes.

For now, this only prints out the traces without doing any annotations,
so the output should be nearly identical to what's printed by
spdk_trace.  The annotations will be added in subsequent patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I406e8d4fde9cb88b31c38aa96f11b88e607e9436
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-06-28 16:17:35 +00:00
Konrad Sztyber
a4044e77d6 usdt: pass ticks as the first probe argument
This will allow to match and compare the DTrace probes with SPDK traces
as they'll be using the same source for the timestamps.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I686c3351428c75f9b618a1a909836504fccad828
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8101
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2021-06-28 16:17:35 +00:00
yidong0635
928db41626 rpc: Fix raid strip size method not consistent.
Fix json config file using strip_size error, and
The strip_size parameter was changed to strip_size_kb  in `bdev_raid_create`
     rpc function in order to make it more meaningful to users.

Unify the coding style in rpc help file.
      using --strip-size-kb --raid-level ... instead of --strip-size_kb.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I341af1034c7ea6716ff839bb524ee733c9e0cd34
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8401
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-24 17:30:19 +00:00
Shuhei Matsumoto
11d76e8c81 iscsi: Set resource pool size to the predetermined values to run with little memory
Add three parameters, pdu_pool_size, immediate_data_pool_size, and
data_out_pool_size to the RPC iscsi_set_options to run iSCSI target
with little memory.

For some use cases, we want to keep the max number of connections,
but simultaneously we want to reduce the pool size and let I/Os wait
until resource is provided.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I74dc785310b1d985f3e338c1e13fba3a3840d113
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-17 08:59:18 +00:00
Michal Berger
1347830beb scripts/vagrant: Allow to mix CMB and PMR setup
As of latest qemu (>= 6.0.0) these can now be used together as
different BARs are used to accommodate for both.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icf32f491fd33ef733f0d946f9b5caaeb87137d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8217
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-06-14 08:43:01 +00:00
Michal Berger
a5c3f72a89 scripts/vagrant: Initialize pmr_cmdline as an empty string
Having pmr_cmdline set to nil was for some reason breaking cmb setup
as weirdly enough pmr_cmdline was ending up being set to "true".
Avoid this by having pmr_cmdline set to some value, empty string in
this case.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib41da699dcc2f213f0b18d21be3fd32f398bae41
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8216
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-06-14 08:43:01 +00:00
Konrad Sztyber
a266b6e4be nvmf: add traces around poll group lifecycle
The traces are tracking the lifecycle of a poll group: creating it,
adding and disconnecting qpairs, and finally destroying the group.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I075b7f24d14b8fbb42bb18ddd70a668a8bace118
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-06-10 10:58:05 +00:00