Added 'rate_iops' FIO option to the performance scripts.
This will be used during tests where limiting factor
has to be artifically imposed, rather than come from
the target/initiator system or enviroment.
One case for this are scheduler tests, to limit
each connection and balance the threads accordingly.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I10bb09be9205a8b7ba90b36cd4e0bf922b0ecbc3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7840
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Revise bdev_rbd_create rpc call to add an optional
parameter "--cluster-name", e.g., "--cluster-name Rados".
Then users can create a rbd bdev with registered
Rados Cluster. This shared strategy can be used to
remove the thread creation overhead if multiple rbds
are connected to the same Ceph cluster.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ide5800f8fc6b2074805272a59731c666fe279b9a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7584
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch is used to add two rpc calls:
bdev_rbd_register_cluster
bdev_rbd_unregister_cluster
Then in the next patch, rbd bdev constructed on the same cluster object
can share the common Rados_t structure in order to remove the thread creation
overhead and improve the scalability.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I898cc4ffabb8e6721ba5bef099cbf948c64d2c98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7551
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Current `test -f` never succeeds, because `/dev/cpu/0/msr` is a
character device file.
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Change-Id: Iaa28f04866ab24140668ad43351a37f0ccccef65
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7166
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is needed before we can start looking up attributes
of the block subsystem under sysfs. In some occasions they
may simply not exist yet at a given time.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I4261d17e92e02acec31b8affaccd0aa11c2b851e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7863
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add {min,max}_cntlid to spdk_nvmf_subsystem, defaulting to 1 and
0xFFEF, respectively, and add nvmf_subsystem_set_cntlid_range() to
allow the controller range to be configured in the range [min_cntlid,
max_cntlid].
Also add {min,max}_cntlid to the nvmf_create_subsystem RPC to allow
the controller ID range to be specified when creating an nvmf
subsystem.
Signed-off-by: Jonathan Teh <jonathan.teh@mayadata.io>
Change-Id: I936db3bb0c9a38569063a6fd3c11df262dfad776
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7322
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
With https://review.spdk.io/gerrit/c/spdk/spdk/+/7684 there are no
more active uses of it in the test suites.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: If6f8227b68f1fe1a4038cec2b6107eac82ad44de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7698
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
fio.py simply wraps itself around fio and doesn't do anything that
would require python to be in use. Having it in a simple bash form
makes it easier to integrate it with autotest's common sh tooling
and to debug any potential issues with the underlying tests.
This also fixes#1919 by making sure only proper nvme devices are
selected for the nvmf targets.
Fixes: #1919.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I111d00df3c7b2517f431cae865e258a665c2ecb3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7684
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The changes in the nvmf_create_transport show
how this command parser work. And there have
two benefit for this changed.
1. Simplify the definition of rpc method. no need add
so many args anymore. Also it retains its original
functions, so we can also check the input args.
2. Make the rpc call more versatile, for example.
when user extend the subparsers(add new args
into subparsers), they can pass some private args
into the rpc method by command parser.
Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Change-Id: Iaf916e3454f23715cf9216794bb80c65b2b4603f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6652
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Older version wasn't providing full support for PMR setup, this one
does.
Rewrite get-pmr such that it supports CMB/PMR sizes greater than 4GB.
Also, since CMB and PMR can coexist in newer versions of qemu try to
detect both under single device.
E.g.:
nvme0:0000:00:04.0:64-bit:prefetchable:0x700000000:0x707ffffff:0x08000000:cmb
nvme0:0000:00:04.0:64-bit:prefetchable:0x400000000:0x5ffffffff:0x200000000:pmr
nvme1:0000:00:05.0:64-bit:prefetchable:0x708000000:0x70fffffff:0x08000000:cmb
nvme2:0000:00:06.0:64-bit:prefetchable:0x600000000:0x6ffffffff:0x100000000:cmb
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ic159f5c12d3ef39db77617f7d64f825356c255a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7539
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
When zcero copy send is enabled and used by initiator,
it could significantly increase latency in some payloads.
To enable more fine graing configuration of zero copy
send feature, add new parameters enable_zerocopy_send_server
and enable_zerocopy_send_client to spdk_sock_impl_opts to
enable/disable zcopy for specific type of sockets.
Exisiting enable_zerocopy_send parameter affects all types
of sockets.
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Change-Id: I111c75608f8826980a56e210c076ab8ff16ddbdc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7457
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
All JSON-RPCs must be documented in jsonrpc.md
Change-Id: Iec9119a01433fd0cbe4df4477ce94cc2feeb0d54
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4074
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Community-CI: Mellanox Build Bot
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9b9f8c9e407425fdab1e84050354481ad76c19c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6655
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Rename to target_nic_ip to better match NVMe-oF
terminology.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0644410cfb063eeea740c54f5860aeefdb52d52d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7089
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
SPDK directory needs to be copied to initiator systems
earlier for set_irq_affinity to run.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0f9ade02e58c9224e0b2ddc75117baaf653be614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6776
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Need more investigating how to properly enable ADQ
for Kernel mode. Do not run Kernel + ADQ configuration
for now.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I700ef417b9b398c067a586a1a5ad9947a92057ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6566
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Readme was frequently missed when adding new stuff or
updating the performance script, and it is out of date.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1710473be576ffbcc2fa8e3701b196bd46cf6654
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6538
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Create a default traffic class (tc0) with minimum
needed number of assigned threads and a priority class
(tc1) with number of assigned threads equal to
number of application threads.
Finally run set_xps_rxqs to configure symmertic queues.
set_xps_rxqs script used from Intel ICE driver
package available at:
https://downloadcenter.intel.com/download/29746/
Intel-Network-Adapter-Driver-for-E810-Series-Devices-under-Linux-
Change-Id: Ie0f2db266621a9dabb1621344bfdc5fa64fee03c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6537
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
There are multiple cases for counting number of CPU threads
on Initiator side. Keep track of num_cores in all of them.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4b8d1d79a2aa17706d206b149c1c8e8816ae2ef5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6536
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Set busy_poll to 0 for non-ADQ. Having it enabled with
this value results is huge latency increase.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ic68ed2b7dc19b8d4b274ce8622194489693863cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7480
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
ld on some distros, e.g., ubuntu1604, doesn't include this particular
path in its SEARCH_DIR list. Add it manually to make sure linking
towards liburing is possible in such a case.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I82d7ed870ec9ff257dc72a386f950addb5476d1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Added mainly to verify if the PMR setup on the VM is correct.
Example of an output with PMR device size of 32M configured for
two nvme devices, and one nvme device with 128M CMB.
[root@fedora31-cloud-1614680581-9ba06c102 ~]# /tmp/get-pmr
nvme0:0000:00:04.0:64-bit:prefetchable:0xf0000000:0xf7ffffff:0x08000000:cmb
nvme1:0000:00:05.0:64-bit:prefetchable:0xf8000000:0xf9ffffff:0x02000000:pmr
nvme2:0000:00:06.0:64-bit:prefetchable:0xfa000000:0xfbffffff:0x02000000:pmr
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I522cebd8b5a16f8a37c6e0e01a0288fb310467fb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Available in qemu >= 5.1.0. Note that since CMB and PMR share the same
BAR they cannot be used together. If both are specified, PMR setup is
skipped.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia0c6681b5ac7563efc6c32d2fff1808814196676
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
-x is supported since 0.4.0 release, make a note of that.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Icc64132536c18108029f6c5fa2cbdd5b27018d8b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7417
Reviewed-by: Thanos Makatos <thanos.makatos@nutanix.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Change-Id: I4ed583d91ae9e820be1ee6f4553f29d6650c4922
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5791
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
num_cores name was misleading, as the parameter actually
contained core mask (or core list). Create separate
class attributes with core mask and number of cores.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ibfae7770aea2f2c1c720abf567400deb50028ab4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6535
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Save info about subsystems created subsystems (target
side) and remote subsystems (initiator side) into
a list.
Change-Id: I7ec1c6a4d1f75060b69b68e0b8e0dd61789fc9e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6374
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Run set_irq_affinity script after irqbalance service
is disabled and manually tune IRQs to be run only on
the CPUs local to NIC NUMA node.
set_irq_affinit.sh script must be manually downloaded
as part of https://github.com/Mellanox/mlnx-tools/tree/master/ofed_scripts
package before running tests.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I67b3c851bcf283caea29ad36f6d4bc0322ca0d27
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6534
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
In performance tests always use best performance policy.
Previously this was manually managed on test systems,
but it's better to do this automatically every time.
Change-Id: Iff81863cf8d9cc713a3c4cce1d8edf7ebbf81c84
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Modify CPU power management settings to achieve high,
consistent results.
Change-Id: I3602ae7523c5b83878238928caaeb453f7d2533e
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6372
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tune sysctl parameters for TCP testing. Restore previous
settings after tests have finished. For ADQ-enabled tests
also set proper value for busy_read option.
Commit includes a fix to initiator exec_cmd() method to
allow using command parameters which contains whitespace,
otherwise it's not possible to set some of sysctl params.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3376b69b8d7c0d8a282765db4fe55824f55f9e05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6265
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Disable all unnecessary services which might affect
tests performance.
Managing services implemented in a very simplified way
usign subprocess and configparser modules (because json
output is not available). Proper implementation would
need to use a proper dBus API, but this seems like an
overkill for such a script.
Change-Id: I72feec4293b77442a9d2c9b4afaa032df1c4d5e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6264
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Configure NIC settings as per ADQ configuration guide.
Change-Id: I957aada267474c2a1448c89a1b7c81d4dd261ca6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6263
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If2784b74ece6c48b81cc53e63412cd2bc618ffef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6262
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Use common method name and parameters for calling
subprocesses for local (Target) and remote systems
(Initiators) instead of using "check_output" (from
subprocess module) and "exec_command" (from paramiko)
separately.
Having these functions wrapped by a single common
method will allow to create common methods in Server
class more eaisly, instead of creating two copies in
Target and Initiator classes.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I1c10f6a88f3d7300c227e969ad6fd901763ac52c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Parse the config file for "adq_enable" flag and enable
the option in SPDK Target and SPDK Initiator configuration.
At this point the flag has no effect on the generated
traffic, as the system and priority queues are not
configured properly. These settings will be added
gradually in following patches.
Change-Id: I5d7bc892f3aa7bbe691b7e0983294b56fcf994f5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6260
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Get detailed info about NICs installed in servers. This
is using lshw utility to make things easier and not
implement the whole logic for parsing /sys objects and
pci.ids listing in system.
Change-Id: I97871fdc9feaae1c2485574a7b488b88ac3afc4f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
The number of options for test execution grew high,
and we still need to add more. This results in a lot
of parameters to pass around in constructors, which
is hard to read and causes pylint to complain.
Instead of passing each option individually as a
separate parameter, pass them as dictionaries just
like they're defined in .json config file. This
makes managing the default values a bit harder, but
is more readable.
Change-Id: I5d88a2b7fe51d2df93edd9130678a937d34facdd
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6207
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
collect_devices() is split into two functions:
- collect_net_devices(): Collect ethernet net devs from the
net class.
- collect_rxe_devices(): Collect all rxe devices from the
infiniband class.
This is done in order to make handling of some conditions easier.
Case and point, in newer kernels, device/net link is not anymore
created for the soft roce devices, instead only ./parent attribute
is available. collect_rxe_devices() is adjusted to handle such
a condition.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Idefa39c4a62c9e650a03e237f49940461e9782a6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6992
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Put plugin parser into a separate function and use it to additionally
parse arguments passed to rpc.py via stdin.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I170d762ed9f5483d92b298f4804ee4e9f227a751
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7145
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Older losetup versions loaded "loop" in background, but
with 2.36.1 this doesn't happen. We know we're going to
use this module so let's load it ourselves.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If3364e92817ae9f09e9897872bd61ca9167184a1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/7086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Currently, reference repos which are used to clone the SPDK repo in
the CI pool's systems don't provide master head refs, hence there's
no easy way to diff against it.
Instead, simply run shfmt against all the bash files in the repo.
Impact on the performance should be minimal.
Also, cleanup some related code which is not used anymore.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ie0d4e35b8ad214ceed1b4ea29b01a7423b8ff73b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is done in order to minimize vagrant work on the CI side. Users
may still provision given VM with full configuration includin these
pieces by running create_vbox.sh with -d argument.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I7f494958e06d4e3b047938dda0bd102da824d235
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
vagrant's shell provisioner uploads files to /tmp, however,
vm_setup.sh must be run from the repo to find all the source
files. This requires the repo by synced over to the VM as well.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iaf16f066af10739589aa584f57c2b89f5c5202e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I191ad5e3b153fb563256eba1aa695716f66db788
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6377
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Leverage SO_INCOMING_CPU to get the CPU affinity of connections
(sockets). And allocate the connections to specific poll groups,
which aims to utilize cache locality.
From our test:
6 P4600 NVMe on target,target uses 8 cores, NIC irqs are bound to
these 8 cores, and initiator side uses 24 and 32 cores,
we can get 11%~17% randwrite performance boost for posix, and 8%~12%
for uring.
Change-Id: I011e0a21502c85adcccd4a14fbe9838b43f54976
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5748
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We need to catch the JSONRPCException when we cannot
connect, so that the error message can be printed
more cleanly. Also suggest to the user that maybe
they don't have an SPDK application running when
it cannot connect.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I497bed86573d5bf07a2b48b3d6682a2427aa4987
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6754
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Also, for consistency, use check_for_driver() while checking if
igb_uio is loaded too.
Change-Id: I00302b3cab169c77032fc0cef0ea384e9e5eb5be
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5303
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
nvmf_create_transport rpc parameter to configure the CQ size helps
if the user is aware of CQ size needed as iWARP doesn't support CQ resize.
Fixes issue #1747
Signed-off-by: Monica Kenguva <monica.kenguva@intel.com>
Change-Id: Ia9ba2b5f612993be27ebfa3455fb4fefd80ae738
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6495
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
There's almost no benefit coming from it CI-wise since all our VMs are
already fully provisioned. The speed up of nightly jobs, which do run
package updates, is minimal. Overall, it only slowers deployment of the
autotest VMs.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I68d9530533f0b0d7d2fb72ce82706d6bc0cf2e6e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
For OCSSD disk case we create two files, but only one was later
chown-ed.
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I411f0717f5416ba012e2e90582b60ec892d2744e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
- throw error instead of silently skipping create disks on
non-linux OS
- early return/exit from script
- add quotes around disk name
- use case/esac to check WHICH_OS
- other small syntax changes
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I71879bf635ecb275e805b6fcb3ec9d6bc3bc299f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Also, add some minor tweaks for the add|rm routines.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I39cdbec2c0aca12e7077d8db161e65fdce7ca19f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6241
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Rationale behind e663dc82ad which put this package as a dependency
into pkgdep was that it shouldn't be exclusive to vm_setup.sh.
However, this particular package causes unnecessary kernel upgrades
whenever newer version is detected by dnf|yum. This behavior is out of
pkgdep's scope and as such similar upgrades should be performed via
vm_setup.sh only where its sole purpose is to configure system on a
wider spectrum.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I066105f95834f3f8d436b9ff52ec1f2bf97844af
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6437
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Interface names will later be needed for additional
test options.
Change-Id: Ic16d305a97087ab1fdc35d5fd5a3d1ed2a021d90
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
So far the information about remote target IP addresses was
stored in "nic_ips" attribute. Add a new "remote_nic_ips"
attribute to be able to differantiate which IPs are local
to given Server class object.
Change-Id: I03dac140241341996674b350c51c18da6e2ee447
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6205
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
It has been confirmed that there is no affected use case in
the SPDK community when we remove the range parameter from
the bdev_ocssd_create RPC.
Hence, remove the range parameter from the bdev_ocssd_create RPC,
remove range parameter from bdev_ocssd_create_bdev(), remove range
info from ocssd_bdev_config_json(), and then update unit tests
accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1b0a541b61bf26732fd028dc43becb7ca2384f8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6220
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Signed-off-by: Simon A. F. Lund <simon.lund@samsung.com>
Change-Id: I039a2e22a665e0d52082e6876f2c7c1cd9a336c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Function bdev_nvme_create accpets new parameter - ctrlr
opts which is passed and filled by RPC handler. That
will allow us to add config parameters for other ctrlr
options with minimal changes.
Change-Id: I96ac1b21e7a3816c652765cddade75423eb843ca
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6023
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Checking only the .sh suffix was omitting ./configure and potentially
other bash scripts which don't necessarily have to include said suffix
in their name.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I6196559e8875de46f80d9c8426577a1a79b5996d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Recent builds with mainline DPDK failed due to missing
python dependency "elftools". This patch adds it to
our system provisioning scripts.
Fixes#1770
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I9e3879d9bddd64bb0f7be81120e5fadebefa9eb8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6103
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
If config without this field was used then
script resulted in KeyError exception.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Icf0b270d28dcc6bf44b66c4b9ed583a6b3ef08b6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6204
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Standard lib modules first, then pip installed modules
and local modules last.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: If0159fad29bab5bef0b69aa803e3cea429cc5f25
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6202
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Users can use "--with-vfio-user" to enable it when testing it.
For CI configuration, we add a new test flag SPDK_TEST_VFIOUSER
to enable CI tests.
Change-Id: Id284df721171d01cc52491ebf4088bcc17eee147
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6139
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added rpc to get name and period of currently set spdk scheduler
and name of currently set spdk governor.
Signed-off-by: Krzysztof Karas <krzysztof.karas@intel.com>
Change-Id: I5562a81a7f9e4879bd48a765c9467f70b43f73ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5917
Community-CI: Broadcom CI
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Using shell=True was unnecessary, as it spawned additional
shell process for any subprocess with this option set.
We don't need that in this script and it's generally
discouraged.
Also removing check=True from some calls as don't have
these subprocess calls in try/excepy anyways.
Change-Id: Ibb9b5d71119c9dd877209ea77b8351f9b610cade
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6046
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Instead enabling each measurement (cpu/power/memory) just
use a single flag and enable them all at once.
Change-Id: I8a639b697285aa741f28b5ddec11ce70c34fc760
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5980
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Use terminate() instead of kill() to exit more gracefully.
Using kill() resulted in (sometimes) output not being saved.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ff1dec8cb0804f6e0674c5ed61bbe4ec9ea70ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6045
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
"measure_*" threads in script use the same results dir
as the results parsing function. To avoid race try to
create results destination directory early.
Change-Id: Ic391a7c2fb3decff64f1fdf51782abb940706d38
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6044
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
DPDK 20.11 moved the kernel modules to separate
dpdk-kmod repository. It has to be built separate
from DPDK.
If needed for testing vm_setup.sh script now contains option
to build this driver.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I98a5eb956eb0cc60ec402d88fcdbd66d4854f19a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/6033
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Enable pcm-power measurements with appriate option.
This will be used for scheduler tests.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I57acc9905c6f1acd7bc3ecdbf886b209a2cdb653
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5979
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Users can now generate the necessary linker args for their
own applications using something like:
PKG_CONFIG_PATH=build/lib/pkgconfig pkg-config --libs spdk_nvme
Dependencies between libraries are included in the generated
.pc files, so the user only needs to pass the top-level subsystems
or individual SPDK libraries they are using in their application.
Modules will automatically be added to the output if the associated
library is specified. For example, specifying "spdk_bdev" will include
the libraries not only for spdk_bdev, but also all of the bdev modules.
Users still need to supply the -Wl,--no-as-needed or -Wl,--whole-archive
flags. They cannot be added to the .pc files without increasing the length
of the argument string by a factor of 15x to 20x.
Modify the test/external_code/hello_world Makefile to use pkg-config to
ensure this gets tested at some level in our autotest environment.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie48a75f11969d5d775d514cf10bcb82d197eabfd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4371
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Usually null bdev is used in testing.
The resize function provides flexibility
Change-Id: I603db0b85ff13a86d38ccc8dd6f260807393df93
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5797
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Period parameter is marked as optional in rpc.py. This works
as long the rpc call is issued with rpc.py, as argparse
sets period to "None" by default. This patch makes it
an optional argument as well directly in app.py.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I2e3cae6df7edd7fc6e1120ee3d4a2655dc60813a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5942
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Sections 'Nvme', 'Nvmf', 'Bdev' and 'iSCSI' were always present in
the output of config_converter. Even when they were not present in
the legacy config.
This posed an issue with iSCSI section. Support for iscsi_set_options
is part of lib/iscsi, that is not compiled with all applications
(such as nvmf_tgt).
Fixes#1741
This patch checks if there is corresponding section in the legacy
config, and only then presents the JSON output.
Added test to verify output from empty legacy config.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ief94e92115dbd80c890dcb434b7c6d5376421c9e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5879
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
JSON configuration that does not contain a particular
subsystem is a valid one. If needed, the defaults are always
present in the subsystem itself.
accel, interface, net_framework, scsi, nbd do not have
legacy configuration to convert. Including those in the
output of config_converter does not serve any purpose.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6442ac1b8f65ac7271102e782068846f86c2d09f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5878
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: <dongx.yi@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch is for packed ring and recover the ring
base when vhost target reconnect to QEMU.
Change-Id: I73f791b4a55adf9834112afd7dd7bb26c75a135d
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4128
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
JSON-RPC allows for methods that do not contain any params.
Turns out that so far this scenario has not occured for load_config or load_sybsystem_config
RPC, which processes JSON config file saved from previous app run.
Next patch writes out bdev_wait_for_examine to bdev subsystem
configuration. As that methods does not contain any params following
error was seen:
# /home/vagrant/spdk_repo/spdk/scripts/rpc.py load_config
Traceback (most recent call last):
File "/home/vagrant/spdk_repo/spdk/scripts/rpc.py", line 2671, in <module>
call_rpc_func(args)
File "/home/vagrant/spdk_repo/spdk/scripts/rpc.py", line 2601, in call_rpc_func
args.func(args)
File "/home/vagrant/spdk_repo/spdk/scripts/rpc.py", line 100, in load_config
include_aliases=args.include_aliases)
File "/home/vagrant/spdk_repo/spdk/scripts/rpc/__init__.py", line 143, in load_config
client.call(elem['method'], elem['params'])
KeyError: 'params'
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a56a9d0caa4b752fb9c1fad4ae581aad8a4c2c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5860
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add spdk_bdev_wait_for_examine() API to be called
in order to report when examine on all registered bdevs finished.
It will be built in to most bdev modules RPC.
New RPC added to allow
- building it into bdev submodule
- user/orchestration to verify examination status manually
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I27db3ae42eea3e692faeea4c2a01d04586bff438
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5480
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
py27-pycodestyle was dropped and is no longer available for the
amd64 arch. Use py37-pycodestyle instead (it should be available for
other common architectures too).
Change-Id: Icb56860d358e2fa06f202a47302a5005b32708e3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
This is done in order to track core dumps in a more efficient
manner. Till now, some cores could be missed if the binary was
executed outside of the cwd of the autotest (i.e. outside of
the spdk repo) but which was part of the critical path of the
actual test (e.g. fio in vhost-initiator tests). Also, since
core_pattern was set to plain "core", impact on the underlying
storage wasn't controlled either - if core was 20G in size,
this is what we would get. This could easly exhaust storage in
case error-prone patchsets were submitted on the CI side.
The collector will try to mitigate all the above by doing the
following:
- collecting all the cores, regardless of their cwd
- limiting size of the core to 2G
- compressing the cores (lz4)
Also, limit of 2 collectors executing at once is set - if more
processes crashes at approx. the same time, they will be logged
in the kernel log instead.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I5956a9030c463ae85a21bfe95f28af5568c5c285
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1933b1c3026c86b55c3e2b712bff02fa8f711bc7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5446
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This patch includes the following work:
1 Add two fields in spdk_bdev_opts structure and make the
two fields configurable.
2 Update the unit test in bdev_ut.c
3 Revise the set_bdev_options rpc call and add the support
to use the two new fields.
Change-Id: Idd6073b3a3ffaba8161e3ffa9444a9e533e67f6d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5664
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There are transport types like vfio-user or fc for which this field is
optional so there is no need to explicitly specify it for the add
listener rpc. Still validation of this filed is preserved for rdma and
tcp transport.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I6aef55bed45b3653472fd356e5a7cfead22c6f2b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5605
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This reverts commit 6d87bc7a8a.
There is an issue with the desing here i.e. lifetime of the subsystem
opts shall be associated with the subsystem but the transport specific
layer is not having any notification about that. As an alternative to
the transport specific subsystem opts listener interface was extended
in a previous commit.
Change-Id: I75c4e329e411a91694959db18ff1955774f0993e
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5571
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In recent Centos8 updates name of the "PowerTools" repo was changed to
"powertools". Since config-manager will fail if we provide a faulty
name of the repo, we need to make sure we tell it to enable what's
actually present on the system.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Iff0c6103d1375fc42f5c967cd713e3e5089f4723
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
If the sriov_numvfs sysfs attr is not available for given controller
report that. Also, verify if number of VFs matches initial value
of 16 devices. If not, report that as well. This number was also
being overwritten with the set value from previous device - this
could false-positive the confirm check so keep the set value in
a seperate variable.
Change-Id: I518464518dd75585c4692c8ec1b3268e798dcf6c
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
In case of latest fio versions "percentile" leaf for clat
latency measurements does not get created in case no IO
was run (e.g. write clat will not have "percentiles"
created in case rw mode was set to randread). This
caused KeyError exception when trying to access non-
existent dict key.
Change-Id: Ie0c28dc932481e5aab971505ba0f38b6798bff36
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5466
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The coming libvfio-user needs json-c and libcmocka library, so install
the libraries in CI environment.
Change-Id: Ib4871169a4d5b140cda11a2cc449201c963a9f12
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
cuse kernel module is not required to build nvme-cuse,
but any usage still requires it.
Packages for that were part of vm_setup script,
but should not be exclusive to it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia105454f65ee87ab3f56580c43d33214beac7b2a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Add keep alive timetout parameter in bdev_nvme_set_options.
NVMe bdev can set this value especially when we test with
bdevperf.
Fix github issue: #1690
Change-Id: I255c935671b74cdb615a8d393e7d7e84524f3c23
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5306
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is done in order to make sure IOMMU extensions, which DPDK is
looking for, are enabled.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Ia9ff8e5329d14a5eb60721f08ff7063cfb1d6eb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Mike Gerdts
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
master_core will still be available but deprecated.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I6fe9eb7fecb2919795c439a02cb7d9d06a2774d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5358
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
NVMf subsystems can have certain hosts that are allowed.
We were previously using the term "whitelist" to refer
to those hosts. Replace that usage as part of removing
this term from the rest of SPDK.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I2878352d83f3911aa8b59629061b818c7f1e5c96
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5282
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
These replace the pci-whitelist and pci-blacklist params
which are now deprecated.
pci-blocked will still use the -B short name. pci-allowed
will use a new -A short name.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Icf1c46c40ad6a0f49fb539b18cdd8d0c8daac636
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5281
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Similarly replace PCI_BLACKLIST with PCI_BLOCKED.
Use of ALLOWED/BLOCKED matches similar changes made
in DPDK.
While here, replace use of term "blacklist" with "blocked"
in one of the nvme perf scripts. The usage there was
associated with how devices are blocked by using the
environment variables that are changed by this patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I720d99118ba5e050f436612c9fd415db44294a63
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5275
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This can be done via explicit options on the
application command line instead.
This has the added benefit of removing a usage of
whitelist/blacklist which we are working to
remove from the SPDK repository.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5739e281f1c29fc8a5d175f5bbc916cd7d926fe4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5274
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
cmake is required by the coming libmuser, it's already installed
in some environments, here install it with more environments.
Change-Id: Ia7ce5ddaad95342e88ddcec327ac9c0eee51297d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5207
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If DPDK was not told to unlink the hugepage backing files during the
hugepage_init, the application will keep them open while holding a
lock on each file. Check if said lock is held, if not, remove the
file.
Change-Id: I738a3d7756c335b63388a2efeb03debd734d9c9e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5065
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Main changes:
- By default, allocate all hugepages on node0. On NUMA-aware systems,
processes will most often use default policy with local node as the
preffered one for allocations. This usually is node0, thus splitting
hugepages evenly across the nodes, in default setup, would force
allocations from a remote node, impacting overall performance (in
case hugepages on node0 run out). See 68740678e1 as a reference.
- Introduce HUGE_EVEN_ALLOC - force setup.sh to evenly distribute
hugepages across all the nodes.
- Introduce HUGEPGSZ - overrides default page size
- Introduce CLEAR_HUGE - remove all hugepages on demand before
allocation is performed.
- HUGENODE - this now can be a list of nodes to use. NRHUGE will be
set across all the nodes from the list.
Change-Id: I084829edde3c416e7fc6b7b6abe369cc0631fcd7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is done so user can easily filter out parts of the output which
are not relevant for additional processing (greping, seding, etc.).
Change-Id: I67eafd1fbeb723224cca6b0026e149cb7f85bfe9
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5152
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is done in order to improve the grepability of the output. Sort
is based on the BDF address.
Hugepages statistics are intact.
Change-Id: I121e9a46bfb1550380f233c1d5c06cefb9d4e7a9
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add an new optional parameter wait to the RPC, iscsi_create_portal_group
not to listen on portals until it is started explicitly.
Fixes the issue #1676.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic217f1ccceb618e70fdb2aff3f710d262a8a9bdb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5091
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Add an new RPC, iscsi_start_portal_group, to start listening on
portals if the specified portal group is not started yet.
The next patch will add an new parameter wait to the existing RPC,
iscsi_create_portal_group.
The RPC allows the specified portal group to be already started,
and returns a success response in this case.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I631d6bccffb38092c95694f922f10648e24d6ff7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5090
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Each nvme BDF can have multiple namespaces, hence multiple block
devices, return them all.
Change-Id: I93fe8acf5b1904f05514445eb3e970ef4254caed
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5170
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
lsblk will include all the potential holders|virtual devices given
block device may be part of. This is relevant in case a nvme|virtio
device is part of the lvm|md setup.
Change-Id: I32198ddcf71746c3cbad4a96496a1762a5c201cd
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5167
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We don't use this script to create qcow2 images for our tests in CI.
Also this script is using random vagrant box images, and we don't know
what is inside them.
Script maintenance is a tedious affair because fedora boxes use xfs file
system, and we can not shrink them virtual disk to minimal size, and this
will caused fails in tests in future.
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7b35f16bbfd6135377995f7b20e1402d14b2b1b1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4738
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
It could potentially override cmp binary, causing failures in some
of the tests.
Change-Id: Ia0bd88222c60e5b8a13d3809a6213b457b887b41
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5005
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We want to replace few qcow2 images with one universal.
This commit contains:
- change password in autotest.sh
- change image path
- change image name
- use snapshot mode in hotplug.sh instead of copying base image
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I75c457fe75f005b0ab43ca909be7886529ed115b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4551
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
In bdev_nvme_set_options command, timeout_us/retry_count/arbitration_burst/low_priority_weight/
medium_prioity_weight/high_prioity_weight/nvme_ioq_poll_period_us/io_queue_requests can be 0,
which should be passed in the rpc command.
Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I2ebff51b7b7de418c5ac8fbd0265444fd27e1e10
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3816
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
in_capsule_data_size/buf_cache_size/sock_priority/max_namespaces can be 0,
which should be passed in nvmf_create_transport/nvmf_create_subsystem commands.
Signed-off-by: Allen Zhu <allenz@mellanox.com>
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ib557cf9f20f7ec2c0b3c31156cd79dbd670ce7e7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3815
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This parameter represents the number of control messages to be
allocated per poll group, specific for TCP transport.
The new parameter can't be zero.
Change-Id: I8ae198c0b46e9a5850a80492aa6260f0c6ef885e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4829
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add RPC that allows to change scheduler at runtime.
Change-Id: I008670f5e936bc25a0fbc923b826277d15343273
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3958
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
gen_nvme.sh will no longer generate the legacy configuration.
"--json" option will still work for any current users of the script.
Tests were modified to no longer use the "--json" option.
Meanwhile others were simplified with switch to "--json-with-subsystems".
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I8450be98660e54c64c27d8401fc40d649f9403ea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
INI is deprecated, use JSON configs for fio plugin instead.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3075e79ed7601116e837536a818821372362eb8d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
If devices are not picked up from the cache, iter_all_pci_class_code()
will pick lspci in first instance to iterate over the pci bus. If said
tool is installed on FreeBSD, it will return BDFs in format which won't
be understood by pciconf (which is used to determine which driver given
device is bound to):
pciconf: cannot parse selector pci0000:00:06.0
To make sure pciconf understands the argument it's given, simply
replace .function with :function (0000:00:06.0 -> 0000:00:06:0).
Change-Id: I59d4f7050c65df99626a3d449aa0a5bb122d4081
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4665
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If they are not, there's no point to continue setup on FreeBSD.
Change-Id: I6d94712f1507c0597fbd569ff73c52f262ab9df9
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4658
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
In case contigmem is loaded, but the kernel environment hasn't been
properly updated with hw.contigmem.* options, kenv will fail,
breaking the check to determine if contigmem should be unloaded.
Avoid the above scenario by determining first if given option is
actually set in kernel's environment.
Change-Id: I4118b7622fd876b28f0c07d081c583f5782f0357
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
“Requirements files” are files containing a list of items
to be installed using pip install. We're going to need
this for SNYK scans. We can also later use this file
in our pkgdep.sh installation script.
Change-Id: I3224f21ee4cbe170c9840aac38156eeee627c257
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4532
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Monica Kenguva <monica.kenguva@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.
For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.
As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.
Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There should be no Qemu param "namespace" added for
emulated nvme drive (i.e. default should be used).
This condition was faulty because ENV variable storing
this information is a text and we check for != 1 as for
and integer.
Change-Id: Ic87aadbbc7eb1ba1ec700bccdb8864726a6562e1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4159
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Mellanox Build Bot
Previous check was always returning true forcing vagrant to add
additional net device to each VM with the same local ip address.
This interface is not needed by vagrant for communicating with
most of the VMs so make sure it's added for the openstack tests
only.
Change-Id: I64092d5c113d3b5f4575f83db640bf5dd40d4bf8
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4542
Reviewed-by: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Original code removed $symbol prefix from every entry in
$defined_symbols. This was incorrect shown by example
patch next in series.
Original function spdk_json_decode_object() was moved
and spdk_json_decode_object_relaxed() was added.
This resulted in $defined_symbols containing entry "_relaxed".
Now each entry in $defined_symbols is checked separatly to
match exactly to the removed $symbol.
Reported-by: Jacek Kalwas <jacek.kalwas@intel.com>
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1d9931d2e93dc85465ce47a838a176c6ab5f1587
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4357
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Some 4.18.x kernels are shipped with a broken uio driver due to the
following change:
git.kernel.org/pub/scm/linux/kernel/git/stable/linux.git/commit/?id=9421e45f5
Try to detect if we are running against a faulty uio driver and if so
fallback to igb_uio driver if present. The priority of picking up the
drivers has not changed.
Note that this commit may be deemed as not needed since from CI
perspective the https://review.spdk.io/gerrit/c/spdk/spdk/+/4342 may
be simply enough.
Change-Id: I9b12511c203c0be0e8f3f462c9c96babde52dc6e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Recent refactors removed the behavior introduced with a6edaa9600.
Bring it back and don't try to rebind the VMD devices if they were
not explicitliy allowed in the setup.
Also, shuffle the code a bit and put verification pieces under one
block where $mode is being determined.
Change-Id: Ie2cc41e402f20147b98ab288d623ac76a4472839
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4398
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It seems that RHEL systems can be actively used without an actual
subscription. Since we don't really want to bother with this side
of the system, simply ignore all subscription-manager's failures.
See https://github.com/spdk/spdk/issues/1542 as a reference.
Change-Id: I76a58cb38bb77b3b6deb12e0cc8ccca5f883d5a7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4360
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Also, put it on yum's cmdline separately since it doesn't really
belong to repos[] - it provides proper tooling not the actual repos.
Change-Id: I26c0c90b52f5b4fa914c116477092df525b4448f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4359
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Updated required NASM version to 2.14,
since intel-ipsec-mb v0.54 now requires it.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I10cee6aea941593b828a6a171297d7e997d0f30c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Error log as below:
06:27:53 # nvme_namespace_revert
06:27:53 # /home/storage/workspace/opal-end-to-end-reservation-autotest/spdk/scripts/setup.sh
/home/storage/workspace/opal-end-to-end-reservation-autotest/spdk/scripts/setup.sh: line 109: /sys/bus/pci/devices/0000:80:04.2/driver/remove_id: No such file or directory
home/storage/workspace/opal-end-to-end-reservation-autotest/spdk/scripts/setup.sh: line 110: /sys/bus/pci/devices/0000:80:04.2/driver/unbind: No such file or directory
06:27:54 # trap - ERR
Change-Id: I302e58143f0e165b17a9b5edd14121e6282f55a7
Signed-off-by: wanghailiangx <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The main purpose of this is to force vagrant into using its own
ssh keys when password authentication is requested. Normally,
when ssh.password is defined, vagrant will attempt to inject its
own ssh key, by using provided password first, and re-use the key
for the new ssh session to provision the VM.
However, some vagrant boxes may come with their own embedded
Vagrantfiles which define custom ssh keys. If password auth is
requested for such boxes, vagrant will not use its own key, nor
the ones that may be defined by the vagrant box. Instead, it will
fallback to interactive, password authentication. From the CI pool
perspective this is not desirable.
On other note, this ties to the ongoing work of building indepdent
images for the CI where vagrant boxes will be deployed with a
custom ssh key alredy provided inside the box.
Related work:
trello.com/c/gAfo9mH1/208-vagrant-improvements-box-packaging
trello.com/c/9Dxp2Y9c/248-packaging-centos78-and-freebsd1112-with-packer
Change-Id: I49035b426519d9b24bcdab573d335ee622130560
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4283
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This is a bool, hence it doesn't require any additional
arguments.
Change-Id: Ibc438f77c5c63ec5dc38007826268a494d8e661d
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifc5e28feacc27f1607a2aa19f3147c6cc9cad106
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
This function is now being called from common.sh instead.
Change-Id: I90df5e3ea543a96fcede33b90369371191dffd07
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
shellcheck.log
shellcheck does not seem to reliably return error codes, at least
on the 0.7.0 version packaged on Fedora. Instead, look at
whether shellcheck.log contains any output.
Change-Id: Iec3c4f416844eed6a6c75b9204e24c2bc7243f7a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Even on version 0.7.0, the diff mode throws up a bunch of errors
that don't appear in the tty mode. We'll have to deal with those
at a later date.
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ic7001b9cf793e696bbf4bab0b0630e8d705a6905
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4071
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I68766e986268352159114dc2ddc9104a17a31718
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4137
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an new RPC, nvmf_subsystem_listener_set_ana_state.
Find the specified subsystem listener, and then set the ANA state
of the listener by calling nvmf_subsystem_listener_set_ana_state().
By adding a string and an enum to the existing context structure,
nvmf_rpc_listener_ctx, and adding an operation type to the existng
enum, nvmf_rpc_listen_op, reuse the existing code and data as much
as possible.
Besides, insert line break into a few long lines and fix wrong
error log.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6fb2dfbb1f9c5f56848eba21d2a733fbed802614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Not sure how this one was missed back during the big rename party.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia00819a2c2e8c81ad0499f4081667dde04fa726e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4140
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This patch is used to enable placement_id getting
in sock layer and also add the rpc support.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I70de57b0ed392a0aefce9d3ff1f61ef924015a87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4146
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add an new RPC, nvmf_subsystem_get_listeners.
ANA state is per listener and per subsystem, and is stored in
subsystem listener. We can return ANA state by the existing
nvmf_get_subsystems RPC but it's confusing that listen addresses
have ANA states.
To change ANA state, we will provide a RPC to change ANA state of
only one selected subsystem listener.
To query ANA state, it will be convenient to get ANA states of all
listeners of one selected subsystem.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic3baad6eac65d7af6e0cab2c4059e1458d41e6e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an new RPC, nvmf_subsystem_get_qpairs to retrieve the list of
qpairs of an NVMe-oF subsystem.
This RPC will be usable to verify if NVMe ANA works.
Pause and resume the subsystem to access the qpairs safely.
One subtle issue remains. The JSON RPC returns success even if
resuming the subsystem fails. Write FIXME to address this.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9d90a01b1117dee00d85b2e21b4f4d02d80db531
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4050
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
liburing.so is a link to actual lib created under /lib, not
/usr/lib{,64}.
Instead of checking hardcoded paths, simply check the ld cache.
Change-Id: I65954490d0b79e17de791d8045a1758cbbb381f5
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4020
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
ELRepo's elrepo-testing holds missing btrfs packages.
On Centos, system repos provide centos-release-ceph-* packages which
install necessary Ceph repositories to provide ceph package.
Also, group these repos into single array and install them in one
bulk. Use yum-config-manager to enable EPEL's|ELRepo's repos instead
of mixing it with unnecessary install step.
Change-Id: I3ba9c6f713ac7e28c7f1ba1e4482d2f2839eeb6a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are 2 issues:
1. building of ninja fails with missed skbuild dependency.
This dependency can be installed with scikit-build module
2. The build of ninja wheel fails due to cmake version. E.g. centos 7.6
uses cmake 2.8 and cmake 3.x can be installed as another package.
It is needed to create a soft link to cmake3 with `cmake` name so
that cmake3 is used instead of cmake 2.8
Change-Id: I53910d8d73b233d3f3a9059ab10938477c5db57b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Most distros ship these drivers as modules, however, some, like
clearlinux, have them compiled directly in the kernel (e.g. nvme).
In case modalias lookup fails, have a fallback prepared just in case.
Change-Id: Ib9da8ff69edbb0c4ce427aa089fa7d04b399a20b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4047
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The previous approach didn't take into the account systems where root
partition is representend in a virtual form of /dev/root (see
clearlinux). In case the device was bound to virtio this would fool
setup.sh into thinking nothing is mounted and proceed with unbinding
the entire rootfs.
Fix this by checking the maj:min numbers of the device instead of its
name against the mountinfo list.
Change-Id: I0feb2584869f6bb72df3e9a4e619620240cfce3b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4046
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Make sure that setup.sh waits for block devices during the tests.
This is to make sure that the underlying controllers are ready to
be talked to and minimize the amount of their flakiness.
Change-Id: Id09445de7ac7ccf4c9f679ee2b6c4bdd14c89733
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This script tries to be smart by mainly looking up all the kernel
events caught over the netlink. This is done in order to mitigate the
amount of flakiness test scripts may suffer from while waiting for
given device to pop up. This mainly concerns nvme devices when they
are moved back from user space - the time from when the ctrl is bound
to the driver until the block subsystem finally gets the device can be
substantial (even > 2s). Consider:
$ setup.sh reset; sleep 10s
vs
$ setup.sh reset; sync_dev_uevents.sh block/disk nvme0n1
Where the latter could recover some of the seconds back:
$ time { ./scripts/setup.sh reset; ./scripts/sync_dev_uevents.sh block/disk nvme0n1 ; }
0000:82:00.0 (8086 0953): uio_pci_generic -> nvme
0000:00:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:80:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:00:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:80:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:00:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:80:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:00:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:80:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:00:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:80:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:00:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:80:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:00:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:80:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:00:04.7 (8086 0e27): uio_pci_generic -> ioatdma
0000:80:04.7 (8086 0e27): uio_pci_generic -> ioatdma
* Found nvme0n1
real 0m2.427s
user 0m0.465s
sys 0m0.408s
Change-Id: Ib91efc73742fb00b058d4f54976d27390489ddbf
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3655
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifeadc3fac3f5d09143dd13ef3d510de9cecffb8a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4032
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On its own these values don't uniquely identify a particular pci
device, hence there's no much point in storing them like so.
Change-Id: Icc5cfec28fcccae6cd935dff03fe3ef01826edc4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3950
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are some devices for which nvme driver takes a long time to
finalize the unbind stage. With that in mind, each device would
add up a significant amount of time needed for setup.sh to complete.
To mitigate such a scenario, make sure the controllers are unbound
in a parallel fashion.
Examples taken from the system with 19 nvmes on board:
[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null
real 0m36.250s
user 0m1.024s
sys 0m1.990s
[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null
real 0m4.848s
user 0m0.867s
sys 0m17.605s
Also, take note that this is currently done only for the nvme
devices since other, i.e., ioatdma, seem to trigger a BUG in the
kernel when unbound in parallel. Some details here:
https://bugzilla.kernel.org/show_bug.cgi?id=209041
Change-Id: Icaeb2b2ecb306f149587bc5da73743b1519bc5d6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
