Commit Graph

110 Commits

Author SHA1 Message Date
Maciej Wawryk
4503500e49 scripts/perf: fix measure_bandwidth failed to open outfile error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9b9f8c9e407425fdab1e84050354481ad76c19c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-22 15:53:34 +00:00
Karol Latecki
9bacb8629d scripts/nvmf_perf: rename remote_nic_ips
Rename to target_nic_ip to better match NVMe-oF
terminology.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0644410cfb063eeea740c54f5860aeefdb52d52d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7089
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
ad7833591c scripts/nvmf_perf: copy SPDK from target early
SPDK directory needs to be copied to initiator systems
earlier for set_irq_affinity to run.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0f9ade02e58c9224e0b2ddc75117baaf653be614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6776
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
02cb1fe86a scripts/nvmf_perf: allow ADQ for SPDK mode only
Need more investigating how to properly enable ADQ
for Kernel mode. Do not run Kernel + ADQ configuration
for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I700ef417b9b398c067a586a1a5ad9947a92057ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6566
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
b06212cfb5 scripts/nvmf_perf: re-write nvmf perf readme file
Readme was frequently missed when adding new stuff or
updating the performance script, and it is out of date.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1710473be576ffbcc2fa8e3701b196bd46cf6654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6538
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
c0fc19f36d scripts/nvmf_perf: configure adq traffic classes
Create a default traffic class (tc0) with minimum
needed number of assigned threads and a priority class
(tc1) with number of assigned threads equal to
number of application threads.
Finally run set_xps_rxqs to configure symmertic queues.

set_xps_rxqs script used from Intel ICE driver
package available at:
https://downloadcenter.intel.com/download/29746/
Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-

Change-Id: Ie0f2db266621a9dabb1621344bfdc5fa64fee03c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
42d2e588d6 scripts/nvmf_perf: Keep track of num_cores for Initiator class
There are multiple cases for counting number of CPU threads
on Initiator side. Keep track of num_cores in all of them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4b8d1d79a2aa17706d206b149c1c8e8816ae2ef5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
31c50e110c scripts/nvmf_perf: disable sysctl busy_read timeout
Set busy_poll to 0 for non-ADQ. Having it enabled with
this value results is huge latency increase.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic68ed2b7dc19b8d4b274ce8622194489693863cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7480
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-21 23:59:40 +00:00
Karol Latecki
4b3a60daee scripts/nvmf_perf: rename SPDK Target num_cores to core_mask
num_cores name was misleading, as the parameter actually
contained core mask (or core list). Create separate
class attributes with core mask and number of cores.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibfae7770aea2f2c1c720abf567400deb50028ab4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
128df53566 scripts/nvmf_perf: save subsystem info
Save info about subsystems created subsystems (target
side) and remote subsystems (initiator side) into
a list.

Change-Id: I7ec1c6a4d1f75060b69b68e0b8e0dd61789fc9e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6374
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
696d55bf94 scripts/nvmf_perf: run set_irq_affinity
Run set_irq_affinity script after irqbalance service
is disabled and manually tune IRQs to be run only on
the CPUs local to NIC NUMA node.

set_irq_affinit.sh script must be manually downloaded
as part of https://github.com/Mellanox/mlnx-tools/tree/master/ofed_scripts
package before running tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I67b3c851bcf283caea29ad36f6d4bc0322ca0d27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3992eb2896 scripts/nvmf_perf: set cpu governor to performance
In performance tests always use best performance policy.
Previously this was manually managed on test systems,
but it's better to do this automatically every time.

Change-Id: Iff81863cf8d9cc713a3c4cce1d8edf7ebbf81c84
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2f9103304f scripts/nvmf_perf: add configure_tuned method
Modify CPU power management settings to achieve high,
consistent results.

Change-Id: I3602ae7523c5b83878238928caaeb453f7d2533e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6372
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
81ccc81dde scripts/nvmf_perf: add configure_sysctl method
Tune sysctl parameters for TCP testing. Restore previous
settings after tests have finished. For ADQ-enabled tests
also set proper value for busy_read option.
Commit includes a fix to initiator exec_cmd() method to
allow using command parameters which contains whitespace,
otherwise it's not possible to set some of sysctl params.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3376b69b8d7c0d8a282765db4fe55824f55f9e05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
a35f2b34dd scripts/nvmf_perf: add configure_services method
Disable all unnecessary services which might affect
tests performance.
Managing services implemented in a very simplified way
usign subprocess and configparser modules (because json
output is not available). Proper implementation would
need to use a proper dBus API, but this seems like an
overkill for such a script.

Change-Id: I72feec4293b77442a9d2c9b4afaa032df1c4d5e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6264
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
2c5849c7be scripts/nvmf_perf: add adq_configure_nic method
Configure NIC settings as per ADQ configuration guide.

Change-Id: I957aada267474c2a1448c89a1b7c81d4dd261ca6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6263
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
3c09b2fb99 scripts/nvmf_perf: add adq_load_modules method
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If2784b74ece6c48b81cc53e63412cd2bc618ffef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6262
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-04-15 16:33:26 +00:00
Karol Latecki
26d7e3d4bb scripts/nvmf_perf: use common method name for calling subprocesses
Use common method name and parameters for calling
subprocesses for local (Target) and remote systems
(Initiators) instead of using "check_output" (from
subprocess module) and "exec_command" (from paramiko)
separately.
Having these functions wrapped by a single common
method will allow to create common methods in Server
class more eaisly, instead of creating two copies in
Target and Initiator classes.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1c10f6a88f3d7300c227e969ad6fd901763ac52c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
f9dd94b515 scripts/nvmf_perf: add adq_enable option to configuration file
Parse the config file for "adq_enable" flag and enable
the option in SPDK Target and SPDK Initiator configuration.
At this point the flag has no effect on the generated
traffic, as the system and priority queues are not
configured properly. These settings will be added
gradually in following patches.

Change-Id: I5d7bc892f3aa7bbe691b7e0983294b56fcf994f5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
ddb4d24b1d scripts/nvmf_perf: get detailed info about NICs in servers
Get detailed info about NICs installed in servers. This
is using lshw utility to make things easier and not
implement the whole logic for parsing /sys objects and
pci.ids listing in system.

Change-Id: I97871fdc9feaae1c2485574a7b488b88ac3afc4f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-04-13 08:40:32 +00:00
Karol Latecki
9fec8853fb scripts/nvmf_perf: re-work managing test options
The number of options for test execution grew high,
and we still need to add more. This results in a lot
of parameters to pass around in constructors, which
is hard to read and causes pylint to complain.

Instead of passing each option individually as a
separate parameter, pass them as dictionaries just
like they're defined in .json config file. This
makes managing the default values a bit harder, but
is more readable.

Change-Id: I5d88a2b7fe51d2df93edd9130678a937d34facdd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-04-13 08:40:32 +00:00
Maciej Wawryk
1fc0c2d8b9 test: move spdk_test_image.qcow2 to spdk_dependencies dir
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8d75f5fa22e79c93d1f8929b6df048b440d9fb8a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6940
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-04-13 05:13:50 +00:00
Maciej Wawryk
0872c7edbd nvmf/perf: Fix type error
TypeError: expected str, bytes or os.PathLike object, not int

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0b395035b43fd4ef5e10ae152889c737ccf66c03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7054
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-03-29 07:40:16 +00:00
Karol Latecki
78ce3152b5 scripts/nvmf_perf: add function to get interface names
Interface names will later be needed for additional
test options.

Change-Id: Ic16d305a97087ab1fdc35d5fd5a3d1ed2a021d90
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
2021-02-19 11:29:18 +00:00
Karol Latecki
c4a330c937 scripts/nvmf_perf: add remote_nic_ips field to configuration
So far the information about remote target IP addresses was
stored in "nic_ips" attribute. Add a new "remote_nic_ips"
attribute to be able to differantiate which IPs are local
to given Server class object.

Change-Id: I03dac140241341996674b350c51c18da6e2ee447
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-02-19 11:29:18 +00:00
Simon A. F. Lund
f79d4b8baf scripts/nvmf_perf: fix pylint E741
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I039a2e22a665e0d52082e6876f2c7c1cd9a336c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-02-16 08:43:24 +00:00
Karol Latecki
4dd1548be3 scripts/nvmf_perf: check "extra_params" key berofe accessing it
If config without this field was used then
script resulted in KeyError exception.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Icf0b270d28dcc6bf44b66c4b9ed583a6b3ef08b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-02-04 08:33:43 +00:00
Karol Latecki
62c2c1c110 scripts/nvmf_perf: fix pylint E1111 error - assignment on no return
Fix assigning variables from functions which return nothing.

Change-Id: Ic54dc0d42e3339cda23970ed347ac6f7bf92e795
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6203
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
2021-02-04 08:33:43 +00:00
Karol Latecki
6882e04563 scripts/nvmf_perf: fix pylint C0411 wrong import order
Standard lib modules first, then pip installed modules
and local modules last.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If0159fad29bab5bef0b69aa803e3cea429cc5f25
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6202
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-02-04 08:33:43 +00:00
Karol Latecki
22e240eb87 scripts/nvmf_perf: do not use "shell" for python subprocess
Using shell=True was unnecessary, as it spawned additional
shell process for any subprocess with this option set.
We don't need that in this script and it's generally
discouraged.
Also removing check=True from some calls as don't have
these subprocess calls in try/excepy anyways.

Change-Id: Ibb9b5d71119c9dd877209ea77b8351f9b610cade
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
abe273b8d1 scripts/nvmf_perf: unify PCM options into single setting
Instead enabling each measurement (cpu/power/memory) just
use a single flag and enable them all at once.

Change-Id: I8a639b697285aa741f28b5ddec11ce70c34fc760
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5980
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
90761053ef scripts/nvmf_perf: stop pcm-memory using sigterm
Use terminate() instead of kill() to exit more gracefully.
Using kill() resulted in (sometimes) output not being saved.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ff1dec8cb0804f6e0674c5ed61bbe4ec9ea70ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6045
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
79199d4164 scripts/nvmf_perf: create results dir early
"measure_*" threads in script use the same results dir
as the results parsing function. To avoid race try to
create results destination directory early.

Change-Id: Ic391a7c2fb3decff64f1fdf51782abb940706d38
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6044
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-01-22 18:32:19 +00:00
Karol Latecki
55e5baf993 scripts/nvmf_perf: add PCM power measurements
Enable pcm-power measurements with appriate option.
This will be used for scheduler tests.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I57acc9905c6f1acd7bc3ecdbf886b209a2cdb653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5979
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2021-01-21 14:24:19 +00:00
Maciej Wawryk
e7fb0e9a8a scripts/nvmf: Add scheduler_setting option in NVMe-oF performance scripts
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1933b1c3026c86b55c3e2b712bff02fa8f711bc7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5446
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-01-05 09:38:00 +00:00
Maciej Wawryk
da5c9b328a scripts/nvmf: enable zcopy on target side
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I5fb99ebf57e7989f23d877798feddddd70179d4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5297
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-01-05 09:38:00 +00:00
Maciej Wawryk
632703f681 scripts/nvmf: Capture configuration in perf tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I343e568f3c4f6d16a7159c8cdc460df2d2c842ca
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5252
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-12-11 11:01:58 +00:00
Karol Latecki
1bc2a2d9e9 scripts/nvmf_perf: fix clat percentile calculations
In case of latest fio versions "percentile" leaf for clat
latency measurements does not get created in case no IO
was run (e.g. write clat will not have "percentiles"
created in case rw mode was set to randread). This
caused KeyError exception when trying to access non-
existent dict key.

Change-Id: Ie0c28dc932481e5aab971505ba0f38b6798bff36
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-12-10 14:50:09 +00:00
Jim Harris
9289521630 test/vhost: replace master_core with main_core
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ibdf9612a27266ef739d9d10db3fd1d353729644a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-12-03 09:41:07 +00:00
Maciej Wawryk
c4c37f1cf1 Modifications for using universal qcow2 image in tests
We want to replace few qcow2 images with one universal.
This commit contains:
 - change password in autotest.sh
 - change image path
 - change image name
 - use snapshot mode in hotplug.sh instead of copying base image

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I75c457fe75f005b0ab43ca909be7886529ed115b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-10-28 15:07:56 +00:00
Karol Latecki
dc45a73d9e perf/nvmf: replace bdev INI configs with JSON
INI is deprecated, use JSON configs for fio plugin instead.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3075e79ed7601116e837536a818821372362eb8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-10-22 10:12:02 +00:00
Karol Latecki
a8e5914833 perf/nvmf: remove pylin no-member error
Add an abstract method in Initiator class to get
rid of pylint "no-member" error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I83ac1a498d55f6390a5b36038a9532121a9e5d0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3922
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
f252ff1f16 perf/nvmf: add option for TCP dif_insert_or_strip
Change-Id: Id5b6082f0ac87e76335f807d1758693d6813bf98
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3920
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
66f42f3703 perf/nvmf: set protection information on null bdevs
Use --dif-type and --md-size option for bdev_null_create
rpc when creating null bdevs. Needed for DIF/DIX comparison
tests.

Change-Id: I15de25d498ff7c239cfca291e10aa16acb4f09eb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3919
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
7049fe936f perf/nvmf: Allow using multiple nullblock devices
So far only single null block device was allowed in
the setup for the purpose of latency measurements.
Rename setting to "null_block_devices" and use an int,
rather than a bool, so that multiple null block
devices are created.

Change-Id: Id543cf3bfbc7d9c171f8605909e1971cabbdfcc7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3900
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
c7f8e73906 scripts/nvmf_perf: fix syntax error
Missing comma causes Python interpreter to
raise syntax error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id86bd10fb55ae2284ebc1c0c36babfe946302d1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3897
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 17:59:13 +00:00
Maciej Wawryk
d1fc288376 scripts/nvmf: add dpdk memory measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6c836114b71b1c5ee1dbbf7900c7314bb5619a48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3551
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-21 08:25:22 +00:00
Maciej Wawryk
e90d35578a scripts/nvmf: Remove unnecessary function call
Function get_nvme_devices_count in kernel mode is
unnecessary called, this caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie1c613f9a46707b9e4f821d03644a8b45bf52966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3508
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
bf601d5ae4 scripts/nvmf: Add Linux to grep nvme list
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1a0d11e14df481f62752da7dcf622e5eb38e26e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3499
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
efd3f83d72 scripts/nvmf: move get_nvme_devices_count function
Getting number of nvme devices before check if this is
null_block test caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I55b0399561ddb11b2f6c36408a56c525e813d239
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00