Commit Graph

1426 Commits

Author SHA1 Message Date
Michal Berger
10b7805b0f scripts/rpc: Make sure address argument is properly interpreted
In case the addr argument was not an existing unix socket file the rpc
client would consider it to be an actual ip address. As a result
connect() would be called with improper set of arguments. This could
cause the rpc.py to block for undesired amount of time until connect()
finally decided to return (seen on some fedora33 builds).

This was affecting sh wrapper functions like waitforlisten() which
use rpc.py to determine if given app is ready to be talk to blocking
execution of the tests for way too long then intendent.

To avoid such a scenario determine the format of the address and use
routines proper for given address family.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7777 (master)

(cherry picked from commit 6c1a1a3dca)
Change-Id: Iaac701d72c772629fa7c6478ff4781b0c5d485d5
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/8018
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-05-31 07:11:13 +00:00
Alexey Marchuk
e6e51ea9a2 sock: Deprecate enable_zerocopy_send in sock_impl_set_options RPC
This deprecated parameter will be removed in SPDK 21.07

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7608 (master)

(cherry picked from commit 2fd97e28bf)
Change-Id: I2b2fbcc798bb50fa6f9dfe35045f66e41c1ceaa9
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7639
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-27 16:25:00 +00:00
Alexey Marchuk
d9efb8f004 sock: Add new params to configure zcopy for server, client sockets
When zcero copy send is enabled and used by initiator,
it could significantly increase latency in some payloads.
To enable more fine graing configuration of zero copy
send feature, add new parameters enable_zerocopy_send_server
and enable_zerocopy_send_client to spdk_sock_impl_opts to
enable/disable zcopy for specific type of sockets.
Exisiting enable_zerocopy_send parameter affects all types
of sockets.

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7457 (master)

(cherry picked from commit 8e85b675fc)
Change-Id: I111c75608f8826980a56e210c076ab8ff16ddbdc
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7638
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-27 16:25:00 +00:00
Ben Walker
03a3a5c02d scripts: Add check_format test for JSON-RPC documentation
All JSON-RPCs must be documented in jsonrpc.md

Change-Id: Iec9119a01433fd0cbe4df4477ce94cc2feeb0d54
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4074
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Community-CI: Mellanox Build Bot
2021-04-26 06:55:03 +00:00
Maciej Wawryk
4503500e49 scripts/perf: fix measure_bandwidth failed to open outfile error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9b9f8c9e407425fdab1e84050354481ad76c19c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-22 15:53:34 +00:00
Karol Latecki
9bacb8629d scripts/nvmf_perf: rename remote_nic_ips
Rename to target_nic_ip to better match NVMe-oF
terminology.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0644410cfb063eeea740c54f5860aeefdb52d52d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7089
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
ad7833591c scripts/nvmf_perf: copy SPDK from target early
SPDK directory needs to be copied to initiator systems
earlier for set_irq_affinity to run.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0f9ade02e58c9224e0b2ddc75117baaf653be614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6776
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
02cb1fe86a scripts/nvmf_perf: allow ADQ for SPDK mode only
Need more investigating how to properly enable ADQ
for Kernel mode. Do not run Kernel + ADQ configuration
for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I700ef417b9b398c067a586a1a5ad9947a92057ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6566
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
b06212cfb5 scripts/nvmf_perf: re-write nvmf perf readme file
Readme was frequently missed when adding new stuff or
updating the performance script, and it is out of date.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1710473be576ffbcc2fa8e3701b196bd46cf6654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6538
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
c0fc19f36d scripts/nvmf_perf: configure adq traffic classes
Create a default traffic class (tc0) with minimum
needed number of assigned threads and a priority class
(tc1) with number of assigned threads equal to
number of application threads.
Finally run set_xps_rxqs to configure symmertic queues.

set_xps_rxqs script used from Intel ICE driver
package available at:
https://downloadcenter.intel.com/download/29746/
Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-

Change-Id: Ie0f2db266621a9dabb1621344bfdc5fa64fee03c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
42d2e588d6 scripts/nvmf_perf: Keep track of num_cores for Initiator class
There are multiple cases for counting number of CPU threads
on Initiator side. Keep track of num_cores in all of them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4b8d1d79a2aa17706d206b149c1c8e8816ae2ef5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
31c50e110c scripts/nvmf_perf: disable sysctl busy_read timeout
Set busy_poll to 0 for non-ADQ. Having it enabled with
this value results is huge latency increase.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic68ed2b7dc19b8d4b274ce8622194489693863cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7480
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Michal Berger
3a1d4786f4 pkgdep/common: Add /usr/lib64 to ld's cache
ld on some distros, e.g., ubuntu1604, doesn't include this particular
path in its SEARCH_DIR list. Add it manually to make sure linking
towards liburing is possible in such a case.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I82d7ed870ec9ff257dc72a386f950addb5476d1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-04-19 14:15:38 +00:00
Michal Berger
0cb256c03d scripts: Add helper script for looking up PMR info under qemu
Added mainly to verify if the PMR setup on the VM is correct.

Example of an output with PMR device size of 32M configured for
two nvme devices, and one nvme device with 128M CMB.

[root@fedora31-cloud-1614680581-9ba06c102 ~]# /tmp/get-pmr
nvme0:0000:00:04.0:64-bit:prefetchable:0xf0000000:0xf7ffffff:0x08000000:cmb
nvme1:0000:00:05.0:64-bit:prefetchable:0xf8000000:0xf9ffffff:0x02000000:pmr
nvme2:0000:00:06.0:64-bit:prefetchable:0xfa000000:0xfbffffff:0x02000000:pmr

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I522cebd8b5a16f8a37c6e0e01a0288fb310467fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-04-19 14:15:38 +00:00
Michal Berger
45fc211e5f scripts/vagrant: Add support for setting a PMR device
Available in qemu >= 5.1.0. Note that since CMB and PMR share the same
BAR they cannot be used together. If both are specified, PMR setup is
skipped.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia0c6681b5ac7563efc6c32d2fff1808814196676
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-19 14:15:38 +00:00
Michal Berger
ec632be24f check_format: Make use of shellcheck's -x conditional
-x is supported since 0.4.0 release, make a note of that.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icc64132536c18108029f6c5fa2cbdd5b27018d8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7417
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-04-19 12:47:51 +00:00
Jin Yu
2f600ca75e virtio-blk: add hotplug rpc
Change-Id: I4ed583d91ae9e820be1ee6f4553f29d6650c4922
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5791
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-16 19:21:13 +00:00
Karol Latecki
4b3a60daee scripts/nvmf_perf: rename SPDK Target num_cores to core_mask
num_cores name was misleading, as the parameter actually
contained core mask (or core list). Create separate
class attributes with core mask and number of cores.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibfae7770aea2f2c1c720abf567400deb50028ab4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
128df53566 scripts/nvmf_perf: save subsystem info
Save info about subsystems created subsystems (target
side) and remote subsystems (initiator side) into
a list.

Change-Id: I7ec1c6a4d1f75060b69b68e0b8e0dd61789fc9e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6374
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
696d55bf94 scripts/nvmf_perf: run set_irq_affinity
Run set_irq_affinity script after irqbalance service
is disabled and manually tune IRQs to be run only on
the CPUs local to NIC NUMA node.

set_irq_affinit.sh script must be manually downloaded
as part of https://github.com/Mellanox/mlnx-tools/tree/master/ofed_scripts
package before running tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I67b3c851bcf283caea29ad36f6d4bc0322ca0d27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3992eb2896 scripts/nvmf_perf: set cpu governor to performance
In performance tests always use best performance policy.
Previously this was manually managed on test systems,
but it's better to do this automatically every time.

Change-Id: Iff81863cf8d9cc713a3c4cce1d8edf7ebbf81c84
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2f9103304f scripts/nvmf_perf: add configure_tuned method
Modify CPU power management settings to achieve high,
consistent results.

Change-Id: I3602ae7523c5b83878238928caaeb453f7d2533e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6372
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
81ccc81dde scripts/nvmf_perf: add configure_sysctl method
Tune sysctl parameters for TCP testing. Restore previous
settings after tests have finished. For ADQ-enabled tests
also set proper value for busy_read option.
Commit includes a fix to initiator exec_cmd() method to
allow using command parameters which contains whitespace,
otherwise it's not possible to set some of sysctl params.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3376b69b8d7c0d8a282765db4fe55824f55f9e05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
a35f2b34dd scripts/nvmf_perf: add configure_services method
Disable all unnecessary services which might affect
tests performance.
Managing services implemented in a very simplified way
usign subprocess and configparser modules (because json
output is not available). Proper implementation would
need to use a proper dBus API, but this seems like an
overkill for such a script.

Change-Id: I72feec4293b77442a9d2c9b4afaa032df1c4d5e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6264
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2c5849c7be scripts/nvmf_perf: add adq_configure_nic method
Configure NIC settings as per ADQ configuration guide.

Change-Id: I957aada267474c2a1448c89a1b7c81d4dd261ca6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6263
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3c09b2fb99 scripts/nvmf_perf: add adq_load_modules method
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If2784b74ece6c48b81cc53e63412cd2bc618ffef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6262
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Alexey Marchuk
5c33437b99 bdev_nvme: Add RPC to dump transport statistics
The new RPC bdev_nvme_get_transport_statistics is added.

Change-Id: Ic13d096717c041fd3e0b77eaebca6ae09239698b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6303
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-13 21:30:52 +00:00
Karol Latecki
26d7e3d4bb scripts/nvmf_perf: use common method name for calling subprocesses
Use common method name and parameters for calling
subprocesses for local (Target) and remote systems
(Initiators) instead of using "check_output" (from
subprocess module) and "exec_command" (from paramiko)
separately.
Having these functions wrapped by a single common
method will allow to create common methods in Server
class more eaisly, instead of creating two copies in
Target and Initiator classes.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1c10f6a88f3d7300c227e969ad6fd901763ac52c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
f9dd94b515 scripts/nvmf_perf: add adq_enable option to configuration file
Parse the config file for "adq_enable" flag and enable
the option in SPDK Target and SPDK Initiator configuration.
At this point the flag has no effect on the generated
traffic, as the system and priority queues are not
configured properly. These settings will be added
gradually in following patches.

Change-Id: I5d7bc892f3aa7bbe691b7e0983294b56fcf994f5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
ddb4d24b1d scripts/nvmf_perf: get detailed info about NICs in servers
Get detailed info about NICs installed in servers. This
is using lshw utility to make things easier and not
implement the whole logic for parsing /sys objects and
pci.ids listing in system.

Change-Id: I97871fdc9feaae1c2485574a7b488b88ac3afc4f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
9fec8853fb scripts/nvmf_perf: re-work managing test options
The number of options for test execution grew high,
and we still need to add more. This results in a lot
of parameters to pass around in constructors, which
is hard to read and causes pylint to complain.

Instead of passing each option individually as a
separate parameter, pass them as dictionaries just
like they're defined in .json config file. This
makes managing the default values a bit harder, but
is more readable.

Change-Id: I5d88a2b7fe51d2df93edd9130678a937d34facdd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Maciej Wawryk
1fc0c2d8b9 test: move spdk_test_image.qcow2 to spdk_dependencies dir
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8d75f5fa22e79c93d1f8929b6df048b440d9fb8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6940
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-13 05:13:50 +00:00
Michal Berger
c09b139e61 test/nvmf: Use rxe_cfg to get rxe-enabled net devices
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iafc067f880525f25363db1adf9a94a9fc9215455
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6994
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Michal Berger
4d52dae32a scripts/rxe_cfg: Add env switch to skip printing status header
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ibc3149b5b9591826ff9aeed38457c2aa9ff747e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Michal Berger
e99cfdb6c9 scripts/rxe_cfg: Split collect_devices()
collect_devices() is split into two functions:

 - collect_net_devices(): Collect ethernet net devs from the
                          net class.
 - collect_rxe_devices(): Collect all rxe devices from the
                          infiniband class.

This is done in order to make handling of some conditions easier.
Case and point, in newer kernels, device/net link is not anymore
created for the soft roce devices, instead only ./parent attribute
is available. collect_rxe_devices() is adjusted to handle such
a condition.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idefa39c4a62c9e650a03e237f49940461e9782a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-12 09:41:20 +00:00
Michal Berger
5ddb210e61 scripts/rpc: Support loading plugins in --server mode
Put plugin parser into a separate function and use it to additionally
parse arguments passed to rpc.py via stdin.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I170d762ed9f5483d92b298f4804ee4e9f227a751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7145
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-02 08:22:42 +00:00
paul luse
30d83eb617 bdev/raid: remove deprecated parameter strip_size
For bdev_raid_create RPC, the deprecated parameter strip_size
was removed.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ie9b8d133e29e82e04f92e4a9336b30d8882e37cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6742
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:41:09 +00:00
Maciej Wawryk
0872c7edbd nvmf/perf: Fix type error
TypeError: expected str, bytes or os.PathLike object, not int

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0b395035b43fd4ef5e10ae152889c737ccf66c03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7054
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:40:16 +00:00
Karol Latecki
f88416a1de scripts/ceph: load loop module explicitly
Older losetup versions loaded "loop" in background, but
with 2.36.1 this doesn't happen. We know we're going to
use this module so let's load it ourselves.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If3364e92817ae9f09e9897872bd61ca9167184a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-26 18:46:13 +00:00
Michal Berger
0968ee2fcf scripts/check_format: Run shfmt against all bash files
Currently, reference repos which are used to clone the SPDK repo in
the CI pool's systems don't provide master head refs, hence there's
no easy way to diff against it.

Instead, simply run shfmt against all the bash files in the repo.
Impact on the performance should be minimal.

Also, cleanup some related code which is not used anymore.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie0d4e35b8ad214ceed1b4ea29b01a7423b8ff73b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2021-03-23 11:03:21 +00:00
Michal Berger
9433190fd0 scripts/vagrant: Move the proxy, tsocks and nfs setup to deploy_test_vm()
This is done in order to minimize vagrant work on the CI side. Users
may still provision given VM with full configuration includin these
pieces by running create_vbox.sh with -d argument.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7f494958e06d4e3b047938dda0bd102da824d235
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-23 10:08:37 +00:00
Michal Berger
ca31f3c512 scripts/vagrant: Run vm_setup.sh directly from the synced repo
vagrant's shell provisioner uploads files to /tmp, however,
vm_setup.sh must be run from the repo to find all the source
files. This requires the repo by synced over to the VM as well.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iaf16f066af10739589aa584f57c2b89f5c5202e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-03-23 10:08:37 +00:00
Rafal Stefanowski
5bdaec6322 ocf/rpc: Add bdev_ocf_set_cache_mode method
This method allows to dynamically switch cache mode on OCF bdev

Signed-off-by: Rafal Stefanowski <rafal.stefanowski@intel.com>
Change-Id: I933273e35bb9f29528657f8f6800c6b52dde32e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6547
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-18 14:39:09 +00:00
Michal Berger
360fd2cc5a pkgdep/rhel: Fix typo in alternatives path
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6d3b7c6855899e3a9d355368c41e4d585db7f382
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6767
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-18 14:36:44 +00:00
Jacek Kalwas
6e2c2306f9 build: install generated pkg-config files
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I191ad5e3b153fb563256eba1aa695716f66db788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6377
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-17 11:10:45 +00:00
Richael Zhuang
201aa63471 sock: introduce SO_INCOMING_CPU to get placement_id
Leverage SO_INCOMING_CPU to get the CPU affinity of connections
(sockets). And allocate the connections to specific poll groups,
which aims to utilize cache locality.

From our test:
6 P4600 NVMe on target,target uses 8 cores, NIC irqs are bound to
these 8 cores, and initiator side uses 24 and 32 cores,
we can get 11%~17% randwrite performance boost for posix, and 8%~12%
for uring.

Change-Id: I011e0a21502c85adcccd4a14fbe9838b43f54976
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5748
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-03-09 08:53:52 +00:00
Jim Harris
9713bfe90d rpc.py: print better message if cannot connect
We need to catch the JSONRPCException when we cannot
connect, so that the error message can be printed
more cleanly. Also suggest to the user that maybe
they don't have an SPDK application running when
it cannot connect.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I497bed86573d5bf07a2b48b3d6682a2427aa4987
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6754
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-03-09 08:53:36 +00:00
Michal Berger
9801533f8a scripts/setup: Use uio_pci_generic if it's loaded/compiled in
Also, for consistency, use check_for_driver() while checking if
igb_uio is loaded too.

Change-Id: I00302b3cab169c77032fc0cef0ea384e9e5eb5be
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5303
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-08 10:49:11 +00:00
Michal Berger
fcc35c86cd scripts/setup: Attempt to load uio driver only when igb_uio is picked up
Change-Id: I5c4860c69ccfdae4532498aaea8886ed3905d899
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5302
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-03-08 10:49:11 +00:00
Monica Kenguva
97ef870100 nvmf/rdma: add RPC parameter to specify CQ size with RDMA transport
nvmf_create_transport rpc parameter to configure the CQ size helps
if the user is aware of CQ size needed as iWARP doesn't support CQ resize.

Fixes issue #1747

Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ia9ba2b5f612993be27ebfa3455fb4fefd80ae738
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6495
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-03-04 08:14:57 +00:00