Commit Graph

1575 Commits

Author SHA1 Message Date
Michal Berger
a3ac80c105 test/config_converter: Drop config_converter tests
It's been some time since SPDK switched from ini to json,
there's no much benefit from keeping these tests around.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0cf7fcd6587abe872553211e41bee12e14bf234f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-12-02 08:09:45 +00:00
Maciej Wawryk
dda5efbc8a scripts/nvmf: fix type error in ethtool_after_fio_ramp
TypeError: ethtool_after_fio_ramp() argument after * must be an iterable, not int

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I12403967b5b8ce7282c65296a54e00bd047cf903
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10367
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-12-02 08:09:30 +00:00
Michal Berger
6dcfa49f61 qat_setup: Don't use absolute path to igb_uio driver
This driver is now built by vm_setup.sh and installed under
/lib/modules.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icea6b96e60390932c14ff49405614c186e61a99b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6099
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-12-02 08:08:10 +00:00
Shuhei Matsumoto
cf8a9bf286 rpc: Fix example of initiator_list of the iscsi_create_initiator_group RPC
We provided list of IP addresses as the example of the list of the
initiator names of the iscsi_create_initiator_group and
iscsi_initiator_group_add_initiators RPCs.

However, they were very confusing and gave little hint about how
to use initiator name.

Fix them by providing IQNs instead.

Fixes issue #2209

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8ce19c4ed0af54fd8a6f1ab2a74b2e6f865d3d1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10236
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-11-30 11:59:31 +00:00
Allen Zhu
b42699966c rpc: Fix failure to set config_param of bdev_rbd_register_cluster
A python error caused failure to set librados configuration.

Change-Id: I72c35e9ff819350ffe231b51450e0dc7b4467ec1
Signed-off-by: Allen Zhu <allenzhu@nvidia.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10343
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-11-26 08:10:09 +00:00
Michal Berger
85395e2895 scripts/setup: Fallback to /proc/sys/vm/nr_hugepages if NUMA is not available
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1801c36ac889e2a25eb937b65382addbbcf7d8b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10394
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-25 08:32:29 +00:00
Michal Berger
ddb42b2e69 scripts/setup: Clear hugepages while cleaning up
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I214e6b45f9784df74196de6e8e40a01feba9f102
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10209
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-24 08:34:47 +00:00
Michal Berger
7e0fc66786 scripts/pkgdep: Disable mdl installation on centos
Versions of centos we support - 7 and 8 - don't ship with ruby which
can handle installation of the mdl gem due to unmet dependencies -
it's simply too old.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id0a3843bcab12b8de332d1d3f64956f043c347e3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9483
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-11-22 08:34:00 +00:00
Richael Zhuang
fe09e7d494 scripts/pkgdep: support shfmt for arm64
Get shfmt package according to arch.

Change-Id: Ib51feb64368e1ef559c886d9899dd7a1f28cff95
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9996
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-22 08:33:29 +00:00
Alexey Marchuk
5e1e850bdc bdev/malloc: Add optimal IO boundary
Allow to specify optimal IO boundary for
malloc bdev, it can be used to test split
of IO requests on generic bdev layer

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ic3529dc00cf852ea5cf40d0553d846a698fff6c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2021-11-18 08:21:43 +00:00
Michal Berger
0b6a29d4a2 pkgdep: Fix ID lookup
In case of linux systems, which don't provide /etc/os-release, the
ID would become plain "linux" (due to a fallback to uname). This
would result in a confusing message stating that we don't support
a "linux" platform. Instead, be specific as to what we are looking
for and what distros we support.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If6a71ea08b315f54c906206165e3dbcd4c21379c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10260
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-18 08:17:50 +00:00
Jim Harris
6a298e87f1 syscalls.bt: print syscall map every second
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3f1589880a00a0788fe5fbb2104789b3f81e1497
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10188
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-11-17 10:58:17 +00:00
John Kariuki
e18cd48cda scripts/nvmf_perf: fix matching result files
When I use a single json config to run 4k and 64k together, the IOPS
for the 4k test were lower than expected because the script is using
metrics from both the 4k_64_randrw_m_100_* and 64k_64_randrw_m_100_*
to calculate the average metrics because 4k_64_randrw_m_100_* is in
64k_64_randrw_m_100_*. Fixed issue by checking, if the result file
startswith the name of the fio config file.

Change-Id: I630385c34223b38694d7e2bb578779627c89866d
Signed-off-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10127
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2021-11-10 19:54:47 +00:00
Michal Berger
182adb70a7 scripts/vagrant: Add option to specify box version
From the CI perspective, this allows for proper binding of given
VM builds to specific tasks (e.g. use this image for LTS, this
one for release, etc.) since by default vagrant always uses the
"latest" version of given box.

Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I0cbd2a47e7dbdcbe83d18e3fc3100fd2a2400983
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10132
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2021-11-10 19:54:28 +00:00
Michal Berger
264e0a0a2b scripts/vagrant: Add option to force boot of given distro
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I1b437ee8f63e7cc15e60f611fe0c814542c73ece
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10131
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-11-10 19:54:28 +00:00
Jim Harris
1e2c5b1f3d nvmf: add dtrace probe for nvmf io path
This can be used for multipath validation.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba62c5e90b22a9a85078fbc783d3a7273c029cde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/10137
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2021-11-10 19:53:32 +00:00
Alexey Marchuk
1421a35e44 rpc: Fix help message of nvmf_subsystem_listener_set_ana_state
Correct ANA state is non_optimized

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I48c79ab29e714bc93ae99cc552e9a8fcf1da3edd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9004
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Dong Yi <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-10 19:51:46 +00:00
Blachut, Bartosz
dc82055029 rpc: fix removing a listener with an empty trsvcid
If trsvcid is not set in the nvmf_subsystem_remove_listener call,
rpc client treats it as 'None' and sends "null" to the application,
which causes decode failure in SPDK.

Trsvcid is described as optional and should be treated this way.
For such a listener removal call with an empty trsvcid, "" should be
sent as trsvcid instead of "null" in the json request.

Signed-off-by: Blachut, Bartosz <bartosz.blachut@intel.com>
Change-Id: Ic26907c22a2600c9618354ce8ff63e93e00aec09
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9659
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-11-02 15:24:21 +00:00
Jim Harris
4c5774fd6f scheduler/dynamic: add bpftrace script
This will allow runtime observation of the
dynamic scheduler in a release build.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7436b5dbcaa0df1529f828ef75f4e9335a092893

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9672
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-28 11:33:32 +00:00
Shuhei Matsumoto
85c77ec183 bdev/nvme: Add "multipath" to the bdev_nvme_attach_controller to enable multipath
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I57aad8ffe9e60d8535e523de55f88649bfffdb39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9869
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Shuhei Matsumoto
43adb646b8 bdev/nvme: Retry failed I/O up to retry_count times
Add bdev_retry_count to spdk_bdev_nvme_opts and retry_count to
nvme_bdev_io, respectively.

Set type of both to int because we want use -1 for infinite retry.

Set the default value of bdev_retry_count to zero for the backward
compatibility.

bdev_retry_count is configurable by the RPC bdev_nvme_set_options.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9bc746fcea54aa8722c76f79c70c2ae2b375aa53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9864
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Shuhei Matsumoto
4495bda43f rpc/bdev_nvme: Deprecate retry_count and add transport_retry_count instead
retry_count of struct spdk_bdev_nvme_opts controls the number of retries
in the transport layer, and is set to transport_retry_count of struct
spdk_nvme_ctrlr_opts.

The next patch will add bdev_retry_count to struct spdk_bdev_nvme_opts
to control the number of retries in the bdev layer.

For clarification, rename retry_count to transport_retry_count of
struct spdk_bdev_nvme_opts. Then deprecate the retry_count parameter
and add and use an new parameter transport_retry_count instead for
the RPC bdev_nvme_set_options.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I0689c54aa1c96ee99d24236e8ff1a594ad7208e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9924
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-27 11:53:31 +00:00
Alexey Marchuk
3d8904c66b nvmf: Add discovery filtering rules
SPDK nvmf target reports all listeners on all subsystems
in discovery pages, kernel target reports only subsystems
listening on a port where discovery command is received.

NVMEoF specification allows to specify any addresses/
transport types. Ch 5: The set of Discovery Log entries should
include all applicable addresses on the same fabric as the
Discovery Service and may include addresses on other fabrics.

To align SPDK and kernel targets behaviour, add filtering
rules to allow flexible configuration of what should be
listed in discovery log page entries.

Fixes #2082

Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: Ie981edebb29206793d3310940034dcbb22c52441
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9185
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
2021-10-25 22:57:48 +00:00
Ben Walker
b098640f05 bdev/nvme: bdev_nvme_detach_controller now understands host parameters
You can now detach specific paths based on the host parameters. This is
useful for two paths to the same target that use different local NICs.

Change-Id: I4858bfda7d940052ca77ffb0bbe764a688fb315d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9827
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-22 04:28:22 +00:00
tanlong
8a154b2c0d bdev/rbd:Add uuid to bdev_rbd_create to support multipath when rbd used as ns of nvmf target
This patch is to solve the issue that two nvmf target connect the same rbd image and used for multipath.
The scenario is host wants to access the same rbd image via two gateways, host and gateways are working as nvmf ini and tgt, and two gateways connect with the rbd image, io can switch to another gateway once one is broken. The targets of multipath must have the same uuid, so this patch add a new argument for bdev_rbd_create, like malloc dev.

Signed-off-by: tanlong <948985618@qq.com>
Change-Id: I593fedb6c5d94f625f1b331fdc40e2db488f7fb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9935
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-22 00:38:18 +00:00
Maciej Wawryk
1aa72efad3 scripts: exit from tests when no qat card
Also moved qat_setup.sh to functional tests
fixes: GH#2203

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iab69b192a29013a69f4cc4a251a34dcc6fbf6b7d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9952
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-21 21:26:44 +00:00
Krzysztof Karas
3c271d5e4c scripts/bpf/trace.py: enable printing relations for NativeProvider
Currently the only way to see the relations, while using trace.py script,
is to feed it a JSON formatted file. This patch allows trace.py
to do the same for the for the binary trace files.

Change-Id: I7237896ede608080d3bec896a43586e34c297b04
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9906
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-21 17:57:33 +00:00
Krzysztof Karas
5594c7c847 scripts/bpf/trace.py: print relation information
Make use of the changes introduced in previous patches while
printing trace information.

Change-Id: I69edadfb4a7d34fa9c8c1c520b083372317cabbb
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9638
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-21 17:57:33 +00:00
Krzysztof Karas
70c171602a trace: enable adding relations between traces
Currently we do not have any way to connect traces from different
modules in SPDK. This change modifies our trace library
and app/trace to handle adding relations between trace points
and a trace object.

Additionally this patch adds classes and fields to structs
inside trace.py to prepare it for future patches implementing
printing relation information.

Change-Id: Ia09d01244d923957d589fd37e6d4c98f9f7bbd07
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9620
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
2021-10-21 17:57:33 +00:00
Konrad Sztyber
a78d8ea4f2 scripts/bpftrace: force basic regex syntax in grep
Forced grep to use use basic regular expression syntax (-G option) and
removed escape characters from "{", as they're not needed in this mode.
It fixes the following errors:

fatal: command line, 'enum spdk_nvmf_subsystem_state \{': Unmatched \{
fatal: command line, 'enum nvmf_tgt_state \{': Unmatched \{

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7dd15207d6ff22d229d8f66668d33e0f5596379e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 23:39:59 +00:00
Konrad Sztyber
143aa5cb1f scripts/bpftrace: use SO names in bpftrace scripts
If an USDT probe is defined within a shared object, bpftrace expects the
path to that shared object instead of an executable.  This means that we
need to replace __EXE__ in the bpftrace scripts differently, depending
on whether the application was linked statically or dynamically.

This is now done by the `scripts/bpf/gen.py` script.  It lists all
available probes, along with their locations, of a process.  Then, it
matches them to those described in a bpftrace script replacing the
__EXE__ markers with the listed location (either a path to the
executable or a shared library).  If a bpftrace script uses a probe that
isn't listed, its __EXE__ will be replaced by a path to the executable.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I7c323d5f7d948ea57cf8d4d3132e4d59a2de594f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9807
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 23:39:59 +00:00
Michal Berger
b794726807 scripts/common: Extract SPDK_GPT_PART_TYPE_GUID from source
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id22c28c46a5fbdf5bd76b31d16f07f2672dfe14e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9306
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-20 21:18:52 +00:00
Michal Berger
76f840c049 autotest: Check if nvme devices are in use before the wipe
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4f838df0bfb91398eb5d179a982165adb8af3476
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9291
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-20 21:18:52 +00:00
Michal Berger
84a9119bb0 scripts: Add python script to read SPDK's GPT GUID from device
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ib24714363405f926ac8483244b24296fe35d5a86
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9591
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-20 21:18:52 +00:00
Maciej Wawryk
ce7a40b6ee scripts/nvmf: disable channel packet inspection optimization after fio ramp time
A temporary workaround for issues seen when using "adq_enable"
is to set channel-pkt-inspect-optimize on before fio ramp_up and
turn it off when ramp_up is ending.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I74f71d11e3b9d1a064b36de8acc7a2a23f51a34b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9879
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 07:23:52 +00:00
Maciej Wawryk
6f2f94e634 scripts/nvmf: fixes for ADQ
* Added sleep as sugested in config guide
 * Moved adq_configure_tc() before nvmf_create_transport()
 * Added extra logs to output

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Icc253b98a0fadbe917e912342f91e2bc5ca5064b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9703
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 07:23:52 +00:00
Krzysztof Karas
6d4ad9a2af scripts/bpf/trace.py: add tcp events
Add tcp events changed in previous patch to the trace.py for
enhanced event information.

Change-Id: I8f176905a8283ba1b588d95bdbc075c38677acbe
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9307
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@gmail.com>
2021-10-20 07:22:00 +00:00
Konrad Sztyber
e89224d1b7 scripts/trace: improve line formatting when printing
Changed the way a line is formatted to avoid unnecessary operations.
This slightly (~5-10%) decreases the runtime when the script is executed
on large trace files.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I5549462d56f866bea99609f746aa53890b98d622
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9442
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2021-10-20 07:22:00 +00:00
Konrad Sztyber
364dbc8f48 scripts/trace: suppress KeyboardInterrupt and BrokenPipe exceptions
The stack trace is no longer printed if a user presses <C-c> while the
script is running or pipes its output to tools such as head or less.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0951c38d914986b5c9bf2ee98bcd046f0e957ff0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9441
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2021-10-20 07:22:00 +00:00
Konrad Sztyber
bc71c2e6d7 scripts/trace: add build/lib to LD_LIBRARY_PATH before starting
It makes it more convinient for a user to start the script, as it's no
longer necessary to update the library path manually.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I0dbeb54cad2585a50162c91f48c5ef199fa133b9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
2021-10-20 07:22:00 +00:00
Konrad Sztyber
5d5d9cbb3e scripts/trace: use trace library for parsing traces
Added trace provider based on the trace library.  It uses ctypes to
define the trace objects and execute library calls.  It significantly
(~2x) speeds up the execution time on large trace files.

The script automatically detects the type of the trace file and will use
the appropriate engine (either JSON or the trace library) using
python-magic for file detection.  When piping a file through stdin, JSON
format will be assumed.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Icfa216edee4662714266968eeb72f0097b3779b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9439
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-10-20 07:22:00 +00:00
Jim Harris
df9f1e99c2 bpftrace: add escape char to gen_enums.sh
At least on my system, I need an extra escape for
the opening brace in the generated enums.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2f1d55f817bc10b3b1084d8280f330cdbd50edf6

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9671
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-18 07:47:38 +00:00
Jim Harris
88f9411281 bpftrace.sh: append enums at end of script
bpftrace scripts do not allow struct definitions after
the first probe definition (including BEGIN/END).
This is problematic for an upcoming patch which will
include some struct defintions.

So instead just add the generated BEGIN/END probes
with enum definitions to the end of the user-specified
script, instead of the beginning.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7bc8c18c0891ee26edc099aef96ea18ed63ccb72

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9670
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2021-10-18 07:47:38 +00:00
Maciej Wawryk
328a75c0cb scripts/nvmf: fixed type error in sar measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibcc7a77d1ebcc013b5050564fcc31b23b1202912
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9840
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2021-10-14 23:20:36 +00:00
Ben Walker
a24d549038 bdev/nvme: bdev_nvme_attach_controller now has a multipath parameter
This parameter may have the values "disable" or "failover". The default
is failover to match existing behavior. In the future we expect to
change the default behavior to disable.

Further, we expect to add an "enable" option soon to do full multipath.

Change-Id: Iebbdc9b95f23101f18d64e085933463498e627be
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9343
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-12 08:52:01 +00:00
Maciej Wawryk
e50d1e0f59 scripts/nvmf: fixed sar mesure error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1a4d32de6a83800cdcd65dde3895a36731c52838
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9710
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-07 09:23:22 +00:00
Konrad Sztyber
b14196fa7f scripts/trace: add owner/object type to tracepoint definition
These fields aren't needed by the JSON provider, as the trace entries in
this format already has the owner and object fields filled, but it'll be
needed for the provider using the trace library.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I9db974793193fe59bfffc8dd3b975d5343036be9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-06 07:30:17 +00:00
Konrad Sztyber
d1732fcedc scripts/trace: create trace provider abstraction
This abstraction will make it possible to support different sources for
the traces.  The current JSON implementation is moved to a provdider and
a new one using the trace_parse library will be added in the following
patches.

Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: I322e7984ffe19ce8b8e1bb551e8339655383623f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Krzysztof Karas <krzysztof.karas@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2021-10-06 07:30:17 +00:00
Maciej Wawryk
c76f45ee87 scripts/nvmf: set correct port in adq_configure_tc
fixes #2178

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I89981459e2d5917bf46cde496550e28b1a3175f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9717
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2021-10-05 08:24:01 +00:00
Michal Berger
8a0fedf4e6 scripts/vagrant: Remove OCSSD support from vagrant configuration
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ifdd1d02b466b6496469c9ea1f09633b6e368ab57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/9617
Community-CI: Broadcom CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2021-09-30 08:07:57 +00:00