Commit Graph

1464 Commits

Author SHA1 Message Date
Michal Berger
aa46af008d scripts/vagrant: Don't expect an argument for -u option
This is a bool, hence it doesn't require any additional
arguments.

Change-Id: Ibc438f77c5c63ec5dc38007826268a494d8e661d
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4239
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-18 07:36:52 +00:00
Michal Berger
684b104389 scripts/pkgdep: Disable liburing installation on FreeBSD
Change-Id: Ifc5e28feacc27f1607a2aa19f3147c6cc9cad106
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4161
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
2020-09-18 07:36:52 +00:00
Michal Berger
3b102b4f57 scripts/pkgdep: Remove call to install_sfhmt from freebsd.sh
This function is now being called from common.sh instead.

Change-Id: I90df5e3ea543a96fcede33b90369371191dffd07
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4160
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-18 07:36:52 +00:00
Ben Walker
d61da0ccda scripts: Break check_format up into functions
Change-Id: I3e3bd44826453e0ff5cc5b01769e81b5fdd660ff
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4073
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-18 07:25:15 +00:00
Ben Walker
9f6f02f22e scripts: Detect shellcheck errors by looking at the length of
shellcheck.log

shellcheck does not seem to reliably return error codes, at least
on the 0.7.0 version packaged on Fedora. Instead, look at
whether shellcheck.log contains any output.

Change-Id: Iec3c4f416844eed6a6c75b9204e24c2bc7243f7a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-18 07:25:15 +00:00
Ben Walker
0ee7697e4d scripts: Always use tty mode for shellcheck
Even on version 0.7.0, the diff mode throws up a bunch of errors
that don't appear in the tty mode. We'll have to deal with those
at a later date.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: Ic7001b9cf793e696bbf4bab0b0630e8d705a6905
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4071
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-18 07:25:15 +00:00
Maciej Wawryk
2e2a0b947d scripts/vagrant: Change host ip in local.conf
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I68766e986268352159114dc2ddc9104a17a31718
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4137
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-16 07:58:21 +00:00
Maciej Wawryk
0b510f0e83 scripts/vagrant: Add network interface for openstack tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I094f8e3c98fb9be4f952efa9ff751e0b1144e391
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4125
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2020-09-16 07:57:56 +00:00
Shuhei Matsumoto
c8cb51d4bc lib/nvmf: Add nvmf_subsystem_listener_set_ana_state RPC
Add an new RPC, nvmf_subsystem_listener_set_ana_state.

Find the specified subsystem listener, and then set the ANA state
of the listener by calling nvmf_subsystem_listener_set_ana_state().

By adding a string and an enum to the existing context structure,
nvmf_rpc_listener_ctx, and adding an operation type to the existng
enum, nvmf_rpc_listen_op, reuse the existing code and data as much
as possible.

Besides, insert line break into a few long lines and fix wrong
error log.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6fb2dfbb1f9c5f56848eba21d2a733fbed802614
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-16 07:55:28 +00:00
paul luse
2b29a2b16c bdev/compress: rename RPC to match standard naming
Not sure how this one was missed back during the big rename party.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia00819a2c2e8c81ad0499f4081667dde04fa726e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4140
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-09-14 09:19:19 +00:00
Ziye Yang
a6db2f3590 sock: enable placement_id configuration in sock layer
This patch is used to enable placement_id getting
in sock layer and also add the rpc support.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I70de57b0ed392a0aefce9d3ff1f61ef924015a87
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4146
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-11 10:04:22 +00:00
Shuhei Matsumoto
9c1d648315 lib/nvmf: Add nvmf_subsystem_get_listeners RPC
Add an new RPC, nvmf_subsystem_get_listeners.

ANA state is per listener and per subsystem, and is stored in
subsystem listener. We can return ANA state by the existing
nvmf_get_subsystems RPC but it's confusing that listen addresses
have ANA states.

To change ANA state, we will provide a RPC to change ANA state of
only one selected subsystem listener.

To query ANA state, it will be convenient to get ANA states of all
listeners of one selected subsystem.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic3baad6eac65d7af6e0cab2c4059e1458d41e6e2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4059
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-09 07:20:16 +00:00
Shuhei Matsumoto
83eb352b3c lib/nvmf: Add nvmf_subsystem_get_qpairs RPC
Add an new RPC, nvmf_subsystem_get_qpairs to retrieve the list of
qpairs of an NVMe-oF subsystem.

This RPC will be usable to verify if NVMe ANA works.

Pause and resume the subsystem to access the qpairs safely.

One subtle issue remains. The JSON RPC returns success even if
resuming the subsystem fails. Write FIXME to address this.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9d90a01b1117dee00d85b2e21b4f4d02d80db531
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4050
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-09 07:20:16 +00:00
Michal Berger
6182bff272 pkgdep/common: Fix liburing's .so lookup
liburing.so is a link to actual lib created under /lib, not
/usr/lib{,64}.

Instead of checking hardcoded paths, simply check the ld cache.

Change-Id: I65954490d0b79e17de791d8045a1758cbbb381f5
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4020
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-09-09 07:19:36 +00:00
Michal Berger
97ab0917e7 pgkdep/rhel: Disable liburing on Centos7|RHEL7
Change-Id: I9986004a2573af239286eef2fb6f67662f2598d5
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4019
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-09-09 07:19:36 +00:00
Michal Berger
8ef9f7b2af pkgdep/rhel: Add ELRepo, Ceph repos for Centos, RHEL
ELRepo's elrepo-testing holds missing btrfs packages.

On Centos, system repos provide centos-release-ceph-* packages which
install necessary Ceph repositories to provide ceph package.

Also, group these repos into single array and install them in one
bulk. Use yum-config-manager to enable EPEL's|ELRepo's repos instead
of mixing it with unnecessary install step.

Change-Id: I3ba9c6f713ac7e28c7f1ba1e4482d2f2839eeb6a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-09 07:19:36 +00:00
Alexey Marchuk
f7b4dd3492 pkgdep: Add a workaround for Centos7.x aarch64
There are 2 issues:
1. building of ninja fails with missed skbuild dependency.
This dependency can be installed with scikit-build module
2. The build of ninja wheel fails due to cmake version. E.g. centos 7.6
uses cmake 2.8 and cmake 3.x can be installed as another package.
It is needed to create a soft link to cmake3 with `cmake` name so
that cmake3 is used instead of cmake 2.8

Change-Id: I53910d8d73b233d3f3a9059ab10938477c5db57b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:28:09 +00:00
Michal Berger
342c08d0b0 scripts/setup: Define fallback driver for each device type
Most distros ship these drivers as modules, however, some, like
clearlinux, have them compiled directly in the kernel (e.g. nvme).

In case modalias lookup fails, have a fallback prepared just in case.

Change-Id: Ib9da8ff69edbb0c4ce427aa089fa7d04b399a20b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4047
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:55 +00:00
Michal Berger
d6c499f3ee scripts/setup: Fix check for mounted devices
The previous approach didn't take into the account systems where root
partition is representend in a virtual form of /dev/root (see
clearlinux). In case the device was bound to virtio this would fool
setup.sh into thinking nothing is mounted and proceed with unbinding
the entire rootfs.

Fix this by checking the maj:min numbers of the device instead of its
name against the mountinfo list.

Change-Id: I0feb2584869f6bb72df3e9a4e619620240cfce3b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4046
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:55 +00:00
Michal Berger
44775a8062 scripts/setup: Add env switch for block dev sync
Make sure that setup.sh waits for block devices during the tests.
This is to make sure that the underlying controllers are ready to
be talked to and minimize the amount of their flakiness.

Change-Id: Id09445de7ac7ccf4c9f679ee2b6c4bdd14c89733
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4029
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
0897e4dbfe scripts/setup: Keep track of the driver given device is bound to
Change-Id: I4c3a54d017f2ed26ce1fa5afb539063f73fadbc1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4031
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
4f1701f60f scripts: Add tool for keeping uevents in sync
This script tries to be smart by mainly looking up all the kernel
events caught over the netlink. This is done in order to mitigate the
amount of flakiness test scripts may suffer from while waiting for
given device to pop up. This mainly concerns nvme devices when they
are moved back from user space - the time from when the ctrl is bound
to the driver until the block subsystem finally gets the device can be
substantial (even > 2s). Consider:

  $ setup.sh reset; sleep 10s

  vs

  $ setup.sh reset; sync_dev_uevents.sh block/disk nvme0n1

Where the latter could recover some of the seconds back:

  $ time { ./scripts/setup.sh reset; ./scripts/sync_dev_uevents.sh block/disk nvme0n1 ; }
0000:82:00.0 (8086 0953): uio_pci_generic -> nvme
0000:00:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:80:04.0 (8086 0e20): uio_pci_generic -> ioatdma
0000:00:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:80:04.1 (8086 0e21): uio_pci_generic -> ioatdma
0000:00:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:80:04.2 (8086 0e22): uio_pci_generic -> ioatdma
0000:00:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:80:04.3 (8086 0e23): uio_pci_generic -> ioatdma
0000:00:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:80:04.4 (8086 0e24): uio_pci_generic -> ioatdma
0000:00:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:80:04.5 (8086 0e25): uio_pci_generic -> ioatdma
0000:00:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:80:04.6 (8086 0e26): uio_pci_generic -> ioatdma
0000:00:04.7 (8086 0e27): uio_pci_generic -> ioatdma
0000:80:04.7 (8086 0e27): uio_pci_generic -> ioatdma
* Found nvme0n1

real    0m2.427s
user    0m0.465s
sys     0m0.408s

Change-Id: Ib91efc73742fb00b058d4f54976d27390489ddbf
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3655
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
1c8a890bff scripts/setup: Reorder the status's header
Change-Id: Ifeadc3fac3f5d09143dd13ef3d510de9cecffb8a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4032
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-07 09:27:42 +00:00
Michal Berger
d2cbb15c40 scripts/common: Don't cache pci vendor,device ids separately
On its own these values don't uniquely identify a particular pci
device, hence there's no much point in storing them like so.

Change-Id: Icc5cfec28fcccae6cd935dff03fe3ef01826edc4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3950
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-02 14:43:02 +00:00
Michal Berger
904ac49f4c scripts/setup: Configure binding of the controllers in parallel
There are some devices for which nvme driver takes a long time to
finalize the unbind stage. With that in mind, each device would
add up a significant amount of time needed for setup.sh to complete.

To mitigate such a scenario, make sure the controllers are unbound
in a parallel fashion.

Examples taken from the system with 19 nvmes on board:

[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null

real    0m36.250s
user    0m1.024s
sys     0m1.990s

[root@supermicro4 spdk]# time ./scripts/setup.sh &>/dev/null

real    0m4.848s
user    0m0.867s
sys     0m17.605s

Also, take note that this is currently done only for the nvme
devices since other, i.e., ioatdma, seem to trigger a BUG in the
kernel when unbound in parallel. Some details here:

  https://bugzilla.kernel.org/show_bug.cgi?id=209041

Change-Id: Icaeb2b2ecb306f149587bc5da73743b1519bc5d6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3893
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 14:43:02 +00:00
Michal Berger
5ea549466e scripts/setup: Commonize the driver binding and BDF verification|status
Change-Id: I8c8f4297891a027c3f4c650bba915d0786f46a47
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3892
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
85501619b4 scripts/setup: Use modalias to verify the bdf's driver
Change-Id: I52afb29b69f758b4cbebf4866099e430953ed681
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3891
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
69ae10f785 scripts/setup: Unify block name lookup
Change-Id: Icde3b528d37f6c9a2d8a2fd9f3548f3ee75ceff0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-02 07:38:48 +00:00
Michal Berger
51b5fa8579 scripts/setup: Group all devices into respective arrays
This is a first patch from a series attempting to merge similar
routines used throughout setup.sh.

Change-Id: I4c71b88f7556b0e9e2f65a37b8175914022486a4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3889
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-09-02 07:38:48 +00:00
JimboLuCN
dd3d5800b0 script/rpc: fix bdev_set_qos_limit's usage error
Signed-off-by: JimboLuCN <jimbolucn@gmail.com>
Change-Id: I67db5c7ef81ae77edefcc5ecf058826d3be8d2e8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3981
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-09-01 07:51:38 +00:00
Michal Berger
e989faa936 scripts/pkgdep: [rhel] Add warning about working under RHEL system
Additionally, fall through potential yum failures to try and install
what's available in repos given system is using.

Change-Id: Icf7f5f3db1d72b67414c7d7e4badb8416a9d25b1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3873
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
1338e698d1 scripts/pkgdep: [rhel] Configure EPEL repo prior installing any packages
For some RHEL releases, especially 8.x, EPEL repo is crucial in order
to install most of the package dependencies, hence we need to make
sure it's in use.

Change-Id: Iaaa1f901e6fffddb65b98fa98b21b9e77e59760f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3872
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
f724f6b9c4 scripts/pgkdep: [rhel] Use python36
On rhel 8.x standard appstream and baseos repos don't provide python
package. Use python36 instead as it's available there.

Change-Id: I125aa3370bdb11f67f2774e00e70196df9680f13
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3871
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-09-01 07:50:41 +00:00
Michal Berger
d70ecbe935 scripts/pkgdep: [rhel] Don't use hard link for a python binary
This limitation may simply fail if /usr is part of a separate
mount|fs so softlink it instead.

Change-Id: I1e0387c2792c0bbd20d83e9739b3882a7cbbbf50
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3870
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-09-01 07:50:41 +00:00
Karol Latecki
7ff4e2af79 bdev/null: make md_size inclusive for dif_type
When using --dif-type option --md-size should be
required as well.
Update & improve bdev_null_create rpc.py help
messages as well.

Change-Id: I6588a97aef6c8792bab7a41ece17c0461bb36844
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3949
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-31 07:47:48 +00:00
Vitaliy Mysak
d267d0e874 lib/log: add RPC to toggle timestamps
Allow toggling log timestamps on and off by adding new RPC call.

Change-Id: I34c84bf89fae352ade266fbf7fd20594ff67bced
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2024
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-31 07:46:36 +00:00
Karol Latecki
a8e5914833 perf/nvmf: remove pylin no-member error
Add an abstract method in Initiator class to get
rid of pylint "no-member" error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I83ac1a498d55f6390a5b36038a9532121a9e5d0a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3922
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
f252ff1f16 perf/nvmf: add option for TCP dif_insert_or_strip
Change-Id: Id5b6082f0ac87e76335f807d1758693d6813bf98
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3920
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
66f42f3703 perf/nvmf: set protection information on null bdevs
Use --dif-type and --md-size option for bdev_null_create
rpc when creating null bdevs. Needed for DIF/DIX comparison
tests.

Change-Id: I15de25d498ff7c239cfca291e10aa16acb4f09eb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3919
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Karol Latecki
7049fe936f perf/nvmf: Allow using multiple nullblock devices
So far only single null block device was allowed in
the setup for the purpose of latency measurements.
Rename setting to "null_block_devices" and use an int,
rather than a bool, so that multiple null block
devices are created.

Change-Id: Id543cf3bfbc7d9c171f8605909e1971cabbdfcc7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3900
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-08-28 07:30:36 +00:00
Michal Berger
60f3b6b218 rxe_cfg_small: Align the way how devices are added|removed
This is done to be more conscious of what net|rxe devices ar currently
in use to make sure we don't unnecessarily write to infiniband class.

Also, this makes use of net/infiniband class more consistent between
both the add and remove actions.

Change-Id: I0ba3bc3a14047e12a804bfdf3aca8c089d34d6b0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:36:17 +00:00
Pawel Piatek
dca3fdc482 script/vagrant: vagrantfile refactoring
- reorganize the code to cleanup main loop
- other small syntax improvements

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I0a2b9ac9e78db2f96ed49d3e7e4d938425c76fa9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3720
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
282b1945ef script/vagrant: split into functions - setup libvirt
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I3ff2bec5f016c4436efb99824d7891858391b657
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3719
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
2e617732ba script/vagrant: split into functions - setup virtualbox
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I69516c217a1517d064f88f3700eeb88066568180
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3718
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
b9e3b917e7 script/vagrant: split into functions - deploy test vm
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I301a100f24f8bb7d20756ccfd0fcd29405956cfc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3717
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
211be73970 script/vagrant: split into functions - setup ssh and vagrant-cachier
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I871ec3f64ba3680ea382ac5e71c5bab4e340c94a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3716
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
221c5b5ee2 script/vagrant: split into functions - setup disks functions
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Ibbb6ab720dc8a95bcb1daebc3f8f555c3fce69c4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3715
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
dc7f3206a1 script/vagrant: split into functions - clear cflags
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I8842fda241403d7bfc0ec0da144848777d44921a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3714
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
d0426c8775 script/vagrant: split into functions - spdk local copy of nfs
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I613e02a88ce4fbfcb67e913574fe3978ba9cb7b2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3713
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
b5fea1314b script/vagrant: split into functions - copy functions
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I9cfa65e6b447bd629e984a495b133c186ff76f55
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3712
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
e08eae22f6 script/vagrant: split into functions - setup proxy
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I82e101afca9e6bc43c49aa6a755e0c3afffb349e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3711
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
0b46c777bd script/vagrant: avoid unnecessary strings interpolation
Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: Ie6e306011249fcdaeb96d3dcdf9807f6ec2d9926
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3710
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Pawel Piatek
9620760bea script/vagrant: vagrant file - checkboxtype refactoring
- rename to get_box_type
- use hash to map distro to box type

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I5608b04d813ad1b37ae4dbbc418217a61bd96a52
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3480
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-27 08:34:43 +00:00
Maciej Wawryk
b5fd1f86f4 scripts/pkgdep: Fix mkdir liburing in clearlinux
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I29630dd873134f6617dcbbdd4fee7c9483121af7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3883
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-08-26 15:50:25 +00:00
GangCao
0e91da2f78 Script: update the JSON content generation
Change-Id: Id34e7fc85cb992d76e35a7370a49dbd79a0f5922
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3846
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-24 07:37:53 +00:00
GangCao
bb0f64daa9 Vagrant: update the used Bdev
To solve below issue:

https://github.com/spdk/spdk/issues/1550

The default one is Malloc0 which is not used.

Change-Id: I36b64457e8583c8243b8b9d6714c42e2d43ade7d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3831
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-24 07:37:53 +00:00
Michal Berger
34cbbf3bf0 scripts/pkgdep: Add bash-completion for SPDK apps and tools
This adds basic bash-completion support for apps commonly used within
the repo. It's meant to work with standard bash-completion package
available for most of the distros out there.

Change-Id: If2f92c85537aa7a56d32b46450f2afa0298e7fed
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3526
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-24 07:37:21 +00:00
Karol Latecki
c7f8e73906 scripts/nvmf_perf: fix syntax error
Missing comma causes Python interpreter to
raise syntax error.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Id86bd10fb55ae2284ebc1c0c36babfe946302d1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3897
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-21 17:59:13 +00:00
Maciej Wawryk
d1fc288376 scripts/nvmf: add dpdk memory measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6c836114b71b1c5ee1dbbf7900c7314bb5619a48
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3551
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-21 08:25:22 +00:00
Michal Berger
8e43a261ea rxe_cfg_small: Remove unused function
link_up_rxes() is not used anywhere so remove it. That said, make sure
that link_up() is called while adding the rxe device(s).

Change-Id: Ic0f9573b911d83d58b42f62d104ef916aa010e4e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3824
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-21 08:24:29 +00:00
Shuhei Matsumoto
1da94ed7b8 rpc/nvmf: Add ana_reporting parameter to nvmf_create_subsystem RPC
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9adc8373050e68872a4d9e89518c137e61005254
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3852
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-21 08:22:40 +00:00
Shuhei Matsumoto
4cc04a1251 lib/nvmf: Add nvmf_subsystem_get_controllers RPC
Add an new RPC, nvmf_subsystem_get_controllers to retrieve the list
of NVMe-oF controllers of an NVMe-oF subsystem.

One of the main use cases will be to get identification information
of NVMe-oF controllers to configure their ANA states dynamically.

Pause and resume the subsystem to access the controllers safely.

One subtle issue remains. The JSON RPC returns success even if
resuming the subsystem fails. Write FIXME explicitly to address this.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibf8d1cf56850a705e343b86022d101b4c7204199
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3848
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-21 08:22:40 +00:00
Michal Berger
98678abe89 vm_setup: Add some missing dependencies
btrfs-progs and xfsprofs are required by some of the tests, e.g.
the nvmf/target/filesystem.sh.

abigail package provides abidiff needed for check_so_deps.sh tests.

Change-Id: I5199fafef2ac03ce04ea3709e9ff76fefa1ac717
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2923
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-20 09:34:23 +00:00
yupeng
2d30df9b0b bdev: add bdev_examine_bdev API
The bdev_examine_bdev api will examine a bdev explicitly. After
disabling the auto_examine feature, a user could call
bdev_examine_bdev to examine a specific bdev he/she wants.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: Ifbbfb6f667287669ddf6175b8208efee39762933
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3219
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-20 08:56:53 +00:00
Shuhei Matsumoto
05cd697757 lib/iscsi: Add MaxR2TPerConnection to iSCSI options
Add MaxR2TPerConnection to iSCSI global options and make it configurable
by JSON RPC.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ida95e5c7dac301a22520656709e1aa4d611f31ef
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3777
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-18 07:27:45 +00:00
Shuhei Matsumoto
5aaf754f81 lib/iscsi: Add MaxLargeDataInPerConnection to iSCSI options
Add MaxLargeDataInPerConnection to iSCSI global options and make
it configurable by JSON RPC.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ibcd16da2eac64241217bedeb89a7929bbdc67871
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3756
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-18 07:27:45 +00:00
Shuhei Matsumoto
e97fd6c936 lib/iscsi: Add iscsi_target_node_request_logout RPC
For the login redirection feature, the current implementation works
only if a portal is redirected from an initial portal to a redirect
portal. However, the login redirection feature should work even if a
portal is redirected from one redirect portal to another redirect
portal.

A public portal group knows only a redirect portal and does not know
the portal group of the redirect portal.

Moreover, it is very likely that an initial portal and a redirect portal
exist in different SPDK iSCSI target applications.

To cover all these concerns, add an new iscsi_target_node_request_logout
RPC to request connections whose portal group tag match for the target
node.

To cover potential use cases, make the second parameter portal group
tag optional.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I612672490722fb22fd4eba055998b7408ab84ca5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-17 08:26:59 +00:00
Shuhei Matsumoto
8cf5581fa9 lib/iscsi: Remove async logout request from iscsi_target_node_redirect()
As written in doc/iscsi.md, typically the login redirection feature
will be used in scale out iSCSI target system, which runs multiple
SPDK iSCSI target applications.

In scale out iSCSI target system, the initial portal, the current
redirect portal, and the next redirect portal are likely to be in
different SPDK iSCSI target applications.

In this case, asynchronous logout request should be sent independently
from the iSCSI target application which has the current redirect portal.

However, we had added asynchronous logout request into the iSCSI target
application which has the next redirect portal. This idea works only
for the case that login is redirected from the initial portal to a
redirect portal.

We remove asynchronous logout request from iscsi_target_node_redirect()
in this patch, and update the corresponding help documents.

The next patch will add an new RPC to send asynchronous logout
request to all connections to the specified portal group and the
specified target.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib0ac72e8cdad7e8c64e446b7495e572fac4b5bae
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3779
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-17 08:26:59 +00:00
xjjx
b70fc69b32 scripts/vagrant: do not try to install vagrant plugins
We assume that vagrant sshfs and proxyconf plugins
should be already installed and we do not want to install
them implicitly. In future this logic will be implemented
in vm_setup.sh script.

Signed-off-by: Pawel Piatek <pawelx.piatek@intel.com>
Change-Id: I421f6fd691312289ce15d432f989c6f7a7cf35b3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3330
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-08-13 07:09:01 +00:00
rafalste
4d91b4ef01 bdev/ocf: add cache line size
Define cache line size when creating cache device.

Signed-off-by: rafalste <rafal.stefanowski@intel.com>
Change-Id: Ica536a9bda35c355b4dd4e406a241defc1453f53
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3449
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-12 10:39:56 +00:00
Seth Howell
8bff48f63c module/bdev_rpc: modify detach_ctrlr rpc to accept trid.
This will allow us to use this RPC to detach only specific
paths from controllers.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib52e38aa7d4ea096418a6dc0328481c2e8db6c54
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3070
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-08-11 08:29:14 +00:00
Seth Howell
ca8c9be341 lib/bdev: fix python docs for attach_controller RPC.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ie4e722bd289e29691cad34418d42088c73ac002a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3614
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-08-11 08:29:14 +00:00
Shuhei Matsumoto
4ea197883c lib/iscsi: Add iscsi_target_node_set_redirect RPC
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I090f55debe4ecdc47459bcfe0571a937f064313b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3439
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-11 08:27:43 +00:00
Shuhei Matsumoto
10d6218444 lib/iscsi: Create portal group as public or private portal group
In SPDK iSCSI target, portal group works almost as identifier of
portal.

To support iSCSI login redirection, we need to have two types of
portal groups, public and private portal groups.

We need portals of public portal groups to redirect to a portal in
a private portal groups at login via temporary login redirection
funciton, and we need to make SendTargets return only portals in
public portal groups.

To do these simply, we mark primary or secondary portal group expicitly
at its creation by this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iccf87a4b9dd1f4a8fbb857a399b8f2dbc7c0b3ab
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3491
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-08-11 08:27:43 +00:00
Shuhei Matsumoto
61cd9d308e lib/sock: Add option to enable or disable quick ACK
TCP delayed ACK can be disabled or enabled by enabling or disabling
quick ACK, respectively.

The recently added spdk_sock_impl_opts is helpful for sock library
to control quick ACK.

Hence this patch adds and uses an option enable_quickack. The option
is effective only for the POSIX sock module.

We have spdk_sock_opts now too but spdk_sock_impl_opts will be better
for this case.

This option is not supported on FreeBSD. FreeBSD users can set the
option globally via sysctl if desired.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic89620267acce5872dc8ecaf7a99bb70ae97e993
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3603
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-08-10 11:29:20 +00:00
Michal Berger
20708f3531 scripts/pkgdep: Move distro independent parts to a common place
This will avoid adding similar components into multiple scripts
instead of just one.

Change-Id: I83cff1ad883f3a7054d0f21ec20210c81cb6b9c1
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3525
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-08-10 11:28:20 +00:00
WANGHAILIANG
8f7cb5e6dd script/pkgdep: fix pip-3 to pip3
Change-Id: Ibda02ab0efcd333d0623a6efe9c540ddd65b0af2
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3627
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-08-04 16:57:22 +00:00
Maciej Wawryk
817d077a10 scripts/pkgdep: Add liburing to install_all_dependencies function
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I84cbd8c7a5563b704349c95df0c83513dcf91c45
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3541
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-07-29 07:36:50 +00:00
paul luse
62b3b171cb module/compress: add new parm to RPC for create compress vol
To specify the desired logical block size. Must be 4K or 512.
If no block size is provided a default of 0 means to use the
underlying bdev block size. For cases where something other
than 4K or 512 is desired, format the underlying device
accordingly and don't specify a logical block size on creation
of the compress vol.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I58b71e210cfa77b3237c0c454585c734e2e22aea
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3177
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-27 22:39:29 +00:00
Maciej Wawryk
e90d35578a scripts/nvmf: Remove unnecessary function call
Function get_nvme_devices_count in kernel mode is
unnecessary called, this caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie1c613f9a46707b9e4f821d03644a8b45bf52966
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3508
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
bf601d5ae4 scripts/nvmf: Add Linux to grep nvme list
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1a0d11e14df481f62752da7dcf622e5eb38e26e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3499
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
efd3f83d72 scripts/nvmf: move get_nvme_devices_count function
Getting number of nvme devices before check if this is
null_block test caused fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I55b0399561ddb11b2f6c36408a56c525e813d239
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-24 15:36:04 +00:00
Maciej Wawryk
cb94476cfc scripts/nvmf: Set minimal job_section_qd value
job_section_qd cannot be equal 0 because tests will fails.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I910ae8dbbe7e18bc79f8d69a8ae157162b3b861b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-07-24 15:36:04 +00:00
Michal Berger
e1287f7f59 scripts/vagrant: Use custom rsync cmdline
By default, vagrant executes rsync with --copy-links argument which
instead of copying the actual symlink is actually copying the file
given link points to. Since symlinks are part of the spdk repo,
they need to be synced "as is" - when symlinks are replaced with
actual regular files, git will report that as a typechange, messing
up status checks that our tests perform.

Change-Id: I72814547c3e876ab2adbc779159ca991f18cbe86
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3375
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
2020-07-24 09:41:08 +00:00
Evgeniy Kochetov
29f31a90e1 sock: Add sock_impl option to disable zero copy on send
Zero copy send can cause performance degradation with small
payloads. This patch adds an option to disable it if required. By
default zero copy is enabled.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I14f2b21ad375e770cb08f850360898bac675b351
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 00:30:45 +00:00
Evgeniy Kochetov
63c5e51ebc sock: Add sock_impl option to disable receive pipe
Receive pipe reduces number of system calls and gives significant
performance improvement with kernel TCP stack and relatively small IO
sizes. With user space TCP/IP implementations there are no system
calls and double buffering introduced by pipe has negative impact on
performance. Receive pipe remains enabled by default.

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ic5ddee42293df2c233ba7ffbe6662de7917ac586
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3343
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-24 00:30:45 +00:00
Maciej Wawryk
7dbaf54bf5 scripts/nvmf: Fixed python syntax error
Missing comma in nvmet_cfg caused this error.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I71864c8dc443e2bb73cd9f575eb6540e16bab40e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3470
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Piatek <pawelx.piatek@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-23 07:29:01 +00:00
Michal Berger
35babaded1 scripts/common: Use separate function for nvme driver lookup
Check if given pci address is bound to nvme driver via a separate
function - this makes the task of getting list of the nvme devices
that can be used in tests a bit easier since we don't have to relay
on, e.g., gen_nvme.sh where the extra step to get said devices is
to parse the actual config.

Change-Id: I1f15f08a3678c0c41b051348af259e225356c340
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3014
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-23 07:24:08 +00:00
Maciej Wawryk
03382cf795 scripts/vagrant: Add Fedora32
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibf0d4b51485ad0060afa7117236f1217354d7c0d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3314
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-22 23:27:45 +00:00
yidong0635
0b09b65147 scripts/pkgdep: Fix unrecongnized os ID of opensuse.
Running pkgdep on opensuse, but report error about:
'Not supported platform detected.'

ID=opensuse-leap must link to sles.sh.
So add this.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: I059378da2b1a9cb7b7f6953d158956690d80053b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3220
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-21 09:22:46 +00:00
Vitaliy Mysak
efb72886b3 scripts/vagrant: add fedora31 to create_vhost_vm script
Adding fedora31 to the list of available choices
in create_vhost_vm script.
This requires some changes in Vagrant_vhost_vm because
fedora is configured differently than ubuntu.

Fedora VMs will very useful for vhost tests because
they run on fedora.

Change-Id: Iad2820a9c3fe2621a368be34d9a72b80ad3f4941
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1677
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-16 07:30:22 +00:00
Maciej Wawryk
1cab8c9f19 scripts/perf: Choose only nvmf device
During nvmf perf test, on machines with have local
nvme device figure out that they are added to
fio.conf file, that can caused bad for results.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ibb6aa7b29ee63a8c41cd80909bc26d1dc224dd0c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3113
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-16 07:30:06 +00:00
Shuhei Matsumoto
26e0ef9a36 lib/nvmf: Make abort execution timeout value optional
Make the abort execution timeout value as optional.

Zero is acceptable and means immediate timeout.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia4b03c65b8bd15899f48be9476ee657446147581
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3104
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-16 07:29:46 +00:00
Maciej Wawryk
c7d1abba18 test/vm_setup: Add clearlinux package manager
Also remove unnecessary jq from pkgdep

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I259e8f00a4a6e9c5463fa9e8bdefb19f9bdaa93d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-07-14 07:55:19 +00:00
Maciej Wawryk
3c8451e69a scripts/perf: Add bandwidth measurement to nvmf test
Bandwidth measurement is realized by bwm-ng tool.
Using this measure tool will help us to check network saturation.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I7b95b9b4ac3ca3b1847ec72ab7ea333807befe89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3190
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-14 07:55:08 +00:00
Karol Latecki
140a772746 scripts/vagrant: use IDE bus for Arch Linux VMs
The original image was prepared using SCSI bus.
Using IDE explicitly speeds up boot process as the
system does not have to look where to boot from.

Change-Id: Ia9f649d1c46c591135833c02ed60b3c960ad8b98
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2840
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-13 08:40:05 +00:00
Karol Latecki
778312c8d2 scripts/pkgdep.sh: install libffi for Arch Linux
Arch Linux instance is failing on that CI. Seems that
libffi is a dependency for installing configshell_fb
and it's missing.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I988f2fb5e879b9e8882409c9c2772d5bd2c5e045
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2839
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2020-07-13 08:40:05 +00:00
Ziye Yang
08b9603531 scripts/pkgdep: Install liburing lib in /usr/lib64.
In some OSes (e.g., Fedora30), /usr/lib is not one of the default
library loading paths. So Let's change it into /usr/lib64.

To address: #1471

Change-Id: I89a66b6096fc1096be7f7e7bb6414f0998b3d974
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3261
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-10 07:30:39 +00:00
Seth Howell
3b830202ca lib/nvmf: add an acceptor_backlog transport opt.
This allows users to configure the number of
connection requests outstanding to an rdma port
at once.

RPC included.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8a2bb86b2fb7565cb10288088d39af763b778703
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3097
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-07-08 07:57:04 +00:00
Seth Howell
5faea08e71 scripts/check_format.sh: fix naming convention check.
I had previously implemented a portion of the naming convention
check that looked for symbols that had been moved by performing
two checks:

git diff commit1 commit2 -- libname | grep for added function
git diff commit2 commit1 --libname | grep for added functions

and then subtracting the values returned by the second check from
the values returned by the first check. That works as long as the two
diffs are reciprocal (i.e. the first diff is a mirror image of the
second). However, this has proven to not be the case.

This change fixes that check by performing the smae diff twice and
grepping for removed functions the same time.

Change-Id: I09c81921d68436baeee706f2d9a6d30db1d23976
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3229
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-08 07:56:06 +00:00
Maciej Wawryk
2ba684aa20 script/vagrant: Change plugins_sync_backend to nfs for clearlinux
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie00ca3d18ebbe94baa272da1bca4baf43df6903a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3110
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-08 07:55:47 +00:00
yidong0635
8c883b977f scripts/spdkcli: Stay on command input.
It will except in the middle of creation if an error command inputted.
usrs hope it could stay on consle. So change this.

For example:
'''
SPDK CLI v0.1

/> lsss
Command not found lsss
/>
'''

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ib8d38b5d3f6db1bae965c56e3b78ff715a574189
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2886
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-08 07:55:15 +00:00
Evgeniy Kochetov
b1074c4905 sock/rpc: Add sock_impl_get/set_options RPC methods
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: If1193ef647c85b045d7f14a03a9942783583f553
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/612
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-07-07 07:32:32 +00:00
Maciej Wawryk
9de6d03ee4 scripts/perf: Fix kernel fio filename generator
Also check nullblock subsystem number.
There was missing subsystem number definition in tgt_start
for null_block tests, and wrong filenames generator in
kernel section.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iae5bc6f497176d6c2a12ab430f4c6c92104fa4a5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2570
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-07 07:30:44 +00:00
Michal Berger
131e6b0208 check_format: Mitigate #1455
Make sure that git diff ignores all submodules while looking for
changes made by astyle. This should prevent the following issues
from occurring:

object directory /var/ci_repos/dpdk-submodule.git/objects does not exist; check .git/objects/info/alternates
object directory /var/ci_repos/intel-ipsec-mb.git/objects does not exist; check .git/objects/info/alternates
error: bad tree object HEAD
fatal: 'git status --porcelain=2' failed in submodule intel-ipsec-mb

Change-Id: Ie9a39e324674337fdbd6bf627da6e80775ceaeec
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3223
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-07-07 07:30:14 +00:00
Ziye Yang
cd8b9455ad test: Enable uring test in CI pool.
This patch is used to enable uring test in some VMs or machines
in the CI pool.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I75c2ad477f5f648289d8dbb344b75b2408d56a38
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3107
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2020-07-03 07:31:08 +00:00
Maciej Wawryk
7303918290 scripts/pkgdep: Add uring install function
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Idabeee49f9a0b9093136cde214e8528f1abc37e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2702
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
2020-07-03 07:31:08 +00:00
Maciej Wawryk
af0057e6e1 scripts/perf: Add effective QD counting
During perf test we figured out, that global qd
settings is split between disks number in filename.
Setting this parameter (multiplied by section disks number
and numjobs) in filename section give as expected value.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I89a44e48f10da131e2e9128f3ff2fbfb197076ad
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2749
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:53:54 +00:00
Karol Latecki
c0ff295493 perf/nvmf: add option to skip SPDK build on initiators
Skip copying and building SPDK on host systems if
specified in configuration file.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4f9c0dd77e020dfd866f52dc138526e63dd9222d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3075
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
d3f46b279b perf/nvmf: aggregate read and write results
Aggregate separate read/write results into
a single stat. Reduces the number of additional
fields and formulas in spreadsheets when preparing
performance reports.

Change-Id: I5fdab35c1bb69aad98f7b9808885801db0416449
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
9228dd5dd7 perf/nvmf: sum results from all test initiators properly
Test result IOPS and BW are a sum from across all initiators,
but latencies should be an average from accross initiators.

In previous version it was mistakenly a sum, and that had
to be manually handled in Excel spreadsheets when gathering
data for performance reports.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I17e38e3cdd593dcda9730c19decc2d6a72a11130
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
baa1f8a4a1 perf/nvmf: save per initiator results to a file
Save each initiator results to a .csv file. Useful
for a bit more insight than aggregated results.

Change-Id: I1fafe52ba78f2c62abfd0e3c863505515a363cd3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3053
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-07-01 07:53:32 +00:00
Karol Latecki
afa0404c4b Revert "scripts/vagrant: Remove inline shell scripts"
This reverts commit 436b9be9d0.

Reason:
these change makes Vagrantfile rely on env variables (eg. spdk_dir),
which are only set when running scripts/vagrant/create_vbox.sh script.
After create_vbox.sh has finished env variables are not accessible,
but still needed to work with Vagrantfile using Vagrant commands
like 'vagrant ssh' or 'vagrant reload', etc. This results in
configuration errors from Vagrant and scripts being unusable.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I135c42d248c13dfa93c2210dabce49df8116e8f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3055
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-07-01 07:51:29 +00:00
WANGHAILIANG
74050eacd7 script/perf: move "make clean" before "./configure"
I suggest putting "make clean" before "./configure". Like Migration-tc3a.sh (spdk/test/vhost/migration):
ssh_remote $MGMT_INITIATOR_IP "cd $spdk_repo_share_dir/spdk; make clean; ./configure --with-rdma --enable-debug; make -j40"
Otherwise, once the configuration is changed, some outdated compiled files may remain after "make clean".

Change-Id: I7300749986129cbfaef44f8ff72fad0449f4e081
Signed-off-by: WANGHAILIANG <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2805
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-26 07:21:15 +00:00
Maciej Wawryk
436b9be9d0 scripts/vagrant: Remove inline shell scripts
Moved inline provisioning steps to external script,
to tidy up Vagrantfile

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I65ffae8180c2fb28701be34384b13928877db5fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2946
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-23 08:58:33 +00:00
Maciej Wawryk
6fac35a3f4 scripts/vagrant: Move getting variables into one place
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id63cd5fa330dd21f73875f11ecfd964e67dfd949
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2945
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-23 08:58:33 +00:00
Maciej Wawryk
8f6fdefd53 scripts/vagrant: Add missing equal sign
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iddee2b60137bd22a2a4e28552e27cb2f7e8185e6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2944
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-23 08:58:33 +00:00
Ben Walker
c37cf9fbd8 nvmf: Make transport-chosen the only valid qpair scheduler
The transport always knows best.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: If66f539033fe11c50dc42e12ec990468588d5739
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2812
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-16 07:44:49 +00:00
Michal Berger
5327de1ffa scripts/vagrant: [nvme] Enable CMB buffer on demand
Allow to specify additional argument which will tell Vagrant to enable
CMB on given nvme drive. The format of the argument is any string which
is then simply converted to plain "true". E.g.

./create_vbox.sh -b "/path_to_nvme_img,nvme,3,true"

Change-Id: I7de55e46491318b082bf4e2165260091b44d60f0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2725
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-06-15 15:27:32 +00:00
Ben Walker
85bdd43b40 build: Example applications build to build/example
This mirrors build/bin and build/lib

Change-Id: Ide1d17dfc8b425adfbe67f381c93c236653bd301
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2380
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Ben Walker
c3b5ae0c60 build: Place fio plugins into build/fio
This makes them easier to find. They are now named
spdk_nvme and spdk_bdev.

Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Change-Id: I1f6736d8f7f5b9669583731fb0b710ba54f0d50a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2679
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Ben Walker
6b9b448e53 build: Output executables from the app directory to build/bin
Automatically place binaries produced from the app directory
into build/bin. This matches with the output in build/lib
that already exists.

Change-Id: I13cd2da71d2f88592e22308fe8a907bf458458b5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2379
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-06-15 15:27:16 +00:00
Darek Stojaczyk
00b52bf5dd scripts/rpc.py: add new parameter to retry connecting
This can potentially replace waitforlisten() in our test scripts.

$ ./scripts/rpc.py -h
  -r CONN_RETRIES       Retry connecting to the RPC server N times with
                        0.2s interval. Default: 0
[...]

$ ./scripts/rpc.py -r 10 spdk_get_version
<SNIP the usual python exception>
rpc.client.JSONRPCException: Error while connecting to
/var/tmp/spdk.sock
Error details: [Errno -2] Name or service not known

real    0m2.452s
user    0m0.242s
sys     0m0.020s

Change-Id: I204da366360647fcce8f8de0eb25b0f576aabc5b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2751
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-08 09:35:16 +00:00
Dayu Liu
415abe87fc check_format: fix pattern error of regular expression
In a few tools or languages, slash(/) is maybe a delimiter by default in regex.
An unescaped delimiter must be escaped with a backslash (\).
Otherwise the pattern of regular expression will be error
when delimiter is slash(/) in a few special environment, i.e. in MinGW32 git bash.

Signed-off-by: Dayu Liu <liu.dayu@zte.com.cn>
Change-Id: I73b556416b98e135eb240078dcc68f80e08c5e74
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2488
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-08 09:34:35 +00:00
Michal Berger
623c3bc62b rxe_cfg: Fix reporting for net devices with >10G speed
Change-Id: I1bb68d0f5bbc1255671bbeefc98a8d65f66b5918
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2783
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-06-08 09:34:18 +00:00
Maciej Wawryk
90190aef02 scripts/pkgdep: Add clearlinux dependencies for spdk cli
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6c8d705e98d901cb79b6ed66804046bc2ad06e57
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
2020-06-08 09:33:15 +00:00
Seth Howell
c2312f6cb3 test/check_format.sh: remove moved symbols in naming check
There was an edge case in this check where a public symbol
definition was just moved and that movement triggered this
check. Going forward, check the diff for symbols that were
both added and removed in the same patch. This indicates a
symbol that was just moved instead of newly added.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I71bbcd6d6b3e0a2133e77c29f4ec7a4f2b09e3c2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
2020-06-05 09:02:44 +00:00
Michal Berger
cba9514dec rxe_cfg: Drop librxe and port minimal rxe_cfg functionality to Bash
Since rdma-core handles the entire libibverbs suite, and because it
dropped rxe_cfg in favor of iproute2's rdma tool, building librxe is
not needed anymore. That said, the functionality of the rxe_cfg that
our test suites are using is basic and useful enough to be preserved
in a form of a simple Bash script. This can be used to avoid full
overhaul of the code which would need to be adjusted for iproute2's
tooling. In case more complex rdma configuration link-wise is needed,
iproute2 dependency can be added then.

Additionally, some of the nvmf functions have been simplified to make
use of the rxe_cfg port.

The formatting of the status cmd is left compatible with the rxe_cfg.

Change-Id: I594a24b73472a16d51401bcd74fd30c415b24ddb
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Community-CI: Mellanox Build Bot
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-05 09:02:27 +00:00
Darek Stojaczyk
d3030f6b95 scripts/setup.sh: add a newline before VMD status section
VMD section wasn't separated from virtio.

$ scripts/setup.sh status
<SNIP>

IDXD Engine
BDF             Vendor  Device  NUMA    Driver

virtio
BDF             Vendor  Device  NUMA    Driver          Device name
VMD
BDF             Numa Node       Driver Name

Change-Id: Iffecf06a036ec1a9738c8edd74664eb2d22c45fa
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2763
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-05 09:02:20 +00:00
Alexey Marchuk
1551197db5 rpc: Deprecate max_qpairs_per_ctrlr parameter
This parameter describes the number of admin and IO
qpairs while admin qpair always exists and should not
be configured explicitly.
Introduce a new parameter `max_io_qpairs_per_ctrlr`
which configures the number of IO qpairs.
Internal structure of NVMF transport is not changed,
both RPC parameters configure the same nvmf transport parameter.

Deprecate max_qpairs_per_ctrlr in spdkcli as well

Side change: update dif_insert_or_strip description -
it can be used by TCP and RDMA transports

Config files parsing is not changed since it is deprecated

Fixes #1378

Change-Id: I8403ee6fcf090bb5e86a32e4868fea5924daed23
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Anil Veerabhadrappa <anil.veerabhadrappa@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom CI
2020-06-04 07:20:45 +00:00
Darek Stojaczyk
922d90c806 log: remove backtrace printing
It was broken since long ago. --enable-log-bt doesn't change
anything. log.c expects SPDK_LOG_BACKTRACE_LVL to be defined
for backtrace to work, but it's not defined anywhere.

Apparently nobody needs this, so remove it.

Change-Id: I2313fd24198b0bf718663f2eafee9b5c6efa0a7f
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2194
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-03 07:39:08 +00:00
Maciej Wawryk
4e6f8167ea scripts/vagrant: Clear environment variables in clearlinux
Clearlinux after installation has no empty environment variables,
which caused test failtures.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8e0673a22c1fe3cf93232704bf8950be357a21b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2717
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2020-06-02 09:52:08 +00:00
Maciej Wawryk
8a1160fb77 scripts/pkgdep: Add ClearLinux dependencies to pkgdep.sh
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id3ce1b6ffe06c4d32a8585f7a8afc336b8372629
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2643
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-06-02 09:52:08 +00:00
Maciej Wawryk
9839e8cbf7 scripts/vagrant: Add clearlinux to Vagrant scripts
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ia3b9255abf84ace0d8eec7db3cd9b7293285a9c8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2642
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-06-02 09:52:08 +00:00
Darek Stojaczyk
59ff57b93c scripts/check_format: use version_lt to compare shfmt version
It turns out we have a function to compare version numbers.
It additionally handles cases like "3.0.10 > 3.0.9". The
previous code returned false, the new one returns true.

version_lt() actually tests for less or equal, so compare
against 3.1.0 - the real minimum shfmt version we need.

Change-Id: Ibd18c4505e5b81c0073a907f23c3fc034cfbafbe
Suggested-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2574
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-02 09:51:14 +00:00
Seth Howell
a354903737 check_format: compare commits against previous commit.
Previously when doing the naming conventions check, we were
checking against master which was causing some weird issues on
patches which weren't rebased. The much simpler check is to just
compare each patch against the previous one.

This method still works to prevent any bad changes from getting
merged, but handles naming issues on a patch by patch basis.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ic031ce07ca1c67819b792dc292dcf6c50df5b1a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2733
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-06-02 09:48:43 +00:00
paul luse
230f827a38 misc comment/echo cleanup, replace use of word DMA for consistency
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I186c68250e7e882e80ed49964dd9dbbba988612d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2111
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-06-01 09:22:54 +00:00
Michal Berger
f385076a1b pkgdep: Make pkgdep.sh to use distro-specific source files
Change-Id: Iee7ba04c489ee1192f6fd65c6647c78536440246
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2508
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
af8f939e2f pkgdep: Intrdouce FreeBSD-specific source
Change-Id: I3617a08089b11ace54f34ce2f21bd120a9aa05a3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2507
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
511de06bb5 pkgdep: Introduce SUSE-specific source
Change-Id: Ie106459d7dbe92e32933dad569b79c348a8f0052
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2506
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
75febdfba7 pkgdep: Introduce RHEL-specific source
Change-Id: Iaa29cdb1920ec3283c1fa62dd4fc20d4c58b1340
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2505
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
ca95a71863 pkgdep: Introduce Debian-specific source
Change-Id: I71b129b3928f1d5881f42ebfd1510a3fb4055cd3
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2504
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
b4c74897f3 pkgdep: Introduce arch-specific source
Change-Id: Ib28364c84a46b98c36ea7202412b34231324f226
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-06-01 09:20:24 +00:00
Michal Berger
c988348361 check_format: Use $output_dir to store shfmt patch in
If $output_dir is set, putting the shfmt diff there will make it
easier to extract it and/or look it up from the side of the per
patch build.

Change-Id: Iefb601a29f165e59b7670a8878e95bb6b16c9650
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2277
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-29 14:59:53 +00:00
Maciej Wawryk
6f702d4ea7 scripts/perf: Fix type error in run_nvmf.py
TypeError: unsupported operand type(s) for /: 'int' and 'range'

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8266c80cc886a51b43d9719b8bf845ffd0730b03
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2259
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-29 14:59:40 +00:00
yupeng
240b243af5 bdev: add doc for bdev_auto_examine
Explain the bdev_auto_examine parameter of the bdev_set_options rpc.
Add bdev_auto_examine parameter for the bdev_set_options
in the python rpc client.

Signed-off-by: Peng Yu <yupeng0921@gmail.com>
Change-Id: I268cd5c01129180467967c2ebfffa7c3ba7e5b1a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2580
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-29 14:58:16 +00:00
Michal Berger
6e08a3146c pkgdep: Use major version of the pip bin
pip3 can come for different python releases (3.6, 3.7, etc.) and
depending on which version is installed some maj.min bins may simply
not exist.

Change-Id: Iefc26902161e2e3578d07e99a9e742b132ef11f0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2565
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-27 07:36:39 +00:00
Seth Howell
0506fd21ff scripts/check_format: add a check for proper naming conventions.
This test prevents anyone from adding a function that begins with
spdk to the lib or module directories without also adding that file
to the corresponding map file and a header in spdk/include or
spdk_internal/include.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ife7db13d4bb8fec7570c8492ac3326222a850c36
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2423
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-27 07:35:02 +00:00
Seth Howell
4bd48a3d97 scripts/check_format: remove extra output from shfmt check.
All of the other checks simply print OK if the test passes
this check prints out a lot of text (especially when run at the
tip of long patch series) that can make it difficult to tell if
anything is actually wrong.

Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ie681d200aa13f208e4afa1ab419cbad63ef6606c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2403
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-21 09:19:42 +00:00
Maciej Wawryk
e96d3280a0 scripts/pkgdep: Add ninja and meson packages to minimal install option
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I36cba747313349f9dbe9d98e6e6297e1a449af3f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2517
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-20 14:14:29 +00:00
Michal Berger
3425fc1fb8 check_format: Don't attempt to review perms of the symlinks
Change-Id: I3cd20ad9f1cadb5ca5e6de78d9b44d5df95f51d8
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2502
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-19 09:43:37 +00:00
Dayu Liu
5cdab5e2e6 doc: fix some typos and align indent when setup.sh --help
Signed-off-by: Dayu Liu <liu.dayu@zte.com.cn>
Change-Id: Id0a2a3a577adfd5026b662d97d74c9ec6aebeb8e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2396
Community-CI: Mellanox Build Bot
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-18 10:14:56 +00:00
Darek Stojaczyk
e5f021c10e scripts/check_format: auto-detect shfmt binary
A newer version of shfmt was released, but the script still
looked explicitly for the older one.

Make it execute the first binary '^shfmt.*' in PATH whose --version
returns at least v3.1.0.

Change-Id: If621abfa4d995a9dc985e6bab8c33aa36c327fae
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2428
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-18 10:14:52 +00:00
Tomasz Zawadzki
2e55b97d96 setup.sh: execute checking ulimit as a user
When using vfio a warning for memlock limit is printed.
TARGET_USER is one set by caller when executing the script,
or set to user that called the script (even when using sudo).

Yet ulimit was printed for the one executing the script,
most likely root.

This patch verifies ulimit for a particular TARGET_USER.
Along with more explicit information which user the
information pertains to.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I20ad62109bc316295208e21f959ecfc263307e1c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1964
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-13 07:46:51 +00:00
Maciej Szwed
b0bac20ad6 rpc: Add possibility to load module with additional RPC commands
This patch adds '--plugin <module_name>' parameter
to rpc.py script which is used to provide module
name with additional RPC commands that should be
included in rpc.py script. User must export path
to rpc module with PYTHONPATH before calling rpc.py
with this new parameter.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I971429a33cf4ea943d2b7f3abd5a6cb82398d2a4

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2263
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-13 07:46:17 +00:00
Michal Berger
871214e2c1 rpc.py: Allow loading json config from different sources
This is relevant in context of rpc_cmd() which was introduced with the
following commit:

     "scripts/rpc.py: add daemon mode" 8b98cdb64a

When rpc.py runs in a server mode, its stdin is already attached to a
pipe connected to its parent Bash process. In such a setup, it's not
possible to use some of the rpc methods, e.g. load_config().

To make use of said methods possible, allow them to load the config
from different sources - a regular file or a string.

Change-Id: I6fa7d13fa1957b6449ce5d5c5a810bd58d0a5703
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2106
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-13 07:46:04 +00:00
Michal Berger
15f52aec2f setup.sh: Watch for NUMA_NO_NODE value
The -1 is valid value indicating that the node id for given device
has not been specified. Detect it and map it to "unknown".

Change-Id: Id97d9046be0e4d5ff257bca66ae3e73e906253fe
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2229
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
5853749ba0 setup.sh: Check status of all numa nodes
Use * instead of ? to match all the potential nodes not only those
with a single digit id. The actual node limit is dependent on
kernel's CONFIG_NODES_SHIFT hence in theory, the actual number of
numa nodes may be far greater than 10.

Change-Id: I1d587735b433ed23594725b9d77a44f7cc729494
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2228
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
c8c4f71efe gen_nvme: Be FreeBSD friendly
Since sysfs is not used by the FreeBSD the following error can be
seen each time gen_nvme.sh is run:

  grep: /sys/bus/pci/devices/$bdf/uevent: No such file or directory

Avoid that by looking up the pci address in a way specific to
FreeBSD.

Change-Id: If81e71cece52c2f27dcf68f7a7eba3dd7d8ce10f
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2227
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
d51345c0b9 setup.sh: Add basic function for handling status option on FreeBSD
Change-Id: I64bcc9f06be7e5cca18195bab58f87a4ba74c903
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2226
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Michal Berger
18c0288730 setup.sh: exit from the script if not supported platform is detected
Also, store kernel name in the separate var and use it throughout the
entire script.

Change-Id: Iaa1c4c4aa52fb4aa708fb476478a81e9c6067b58
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2340
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-05-12 08:12:09 +00:00
Michal Berger
5a436f0c61 scripts/common: Cache vendor|device ids for reverse lookup
Change-Id: I69959547d1c71cf3b65f8eb0b786197650a6ece0
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2225
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-12 08:12:09 +00:00
Karol Latecki
4cfc2e2641 scripts/pkgdep: install yum-utils for CentOS
This will probably be not installed by default in CentOS
installations. If we're going to use "config-manager" then
it requires yum-utils.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I7d310e376430b065742b671d983d11dd87bb5fac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2221
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-11 12:35:00 +00:00
Karol Latecki
4d7936227a scripts/pkgdep: add missing "assume yes" to install command
Change-Id: I2a1a22e9d665a5b806b42db504062780f4f5ce53
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2220
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-11 12:35:00 +00:00
Michal Berger
d596ba87b2 check_format: Introduce shfmt tooling for .sh style enforcement
Add new dev tool for enforcing proper formatting of the Bash code
across the entire repo. This is done in order of defining a common
set of good practices to follow when writing .sh|Bash code.

As powerful as shfmt may be, it allows only for some specific rules
to be enforced, hence it still needs to work side by side with
shellcheck syntax-wise. If it comes to style, following rules are
being enforced:

  * indent_style = tab - Lines must be indented with tabs. The exception
			 from this rule is the use of heredocs with
			 <<BASH redirect operator. Spaces can be used to
			 format the line only if it's already preceded
			 with a tab.

  * binary_next_line = true - Lines can start with logical operators. E.g:

			      if [[ -v foo ]] \
			      	&& [[ -v bar ]]; then
				 ...
			      fi
  * switch_case_indent = true - case|esac patterns are indented with tabs.

  * space_redirects = true - redirect operators are followed with a space.
			     E.g: > foo over >foo.

In addition, shfmt will enforce its own Bash-style for different parts
of the code as well. Examples and more details can be found here:

  https://github.com/mvdan/sh

Change-Id: I6e5c8d79e6dba9c6471010f3d0f563dd34e62fd6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1418
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
Michal Berger
844c8ec383 check_format: Reformat the Bash code in compliance with shfmt
Change-Id: I93e7b9d355870b0528a0ac3382fba1a10a558d45
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1718
Community-CI: Mellanox Build Bot
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-05-07 20:52:21 +00:00
paul luse
3044bd27d0 lib/accel: remove RPC for setting the module
This was added before the usage of having a SW engine and 2 HW
engines was fully thought out. The current rules are:

* if no HW engine specific enable RPC is sent, use SW
* if a HW engine specific enable RPC is sent, use it
* If a 2nd HW engine specific enable RPC is sent, ignore

In this scheme there's no need for an RPC that lets the user
choose which engine to use because they already do so when
they enable an engine.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I006ffb3b417f1e93bb061b29535d157ba66f03b4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2033
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-05-07 10:41:44 +00:00
Michal Berger
ae7300425b autobuild: Use mktemp() for spdk workspace
With issues similar to https://github.com/spdk/spdk/issues/1293 it
would be better to use temporary directories for spdk workspace
instead of depending on the hardcoded path (/tmp/spdk). This will
mitigate potential scenario where permissions of the target dir
are suddenly changed in-between autobuild runs.

Change-Id: If44a2dcce712a185287186f33e7361223dc6d451
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1863
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-05-07 10:41:08 +00:00
Tomasz Zawadzki
519d720902 scripts/rpc: do not list positional arguments for the subparser
Main subparser for rpc lists all positional arguments. Like so:
positional arguments:
  {framework_start_init,start_subsystem_init,framework_wait_init,....

This again is duplicated with better help message.

This patch just removes this list, leaving the most detailed one.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I5e6b9d0dd6ccf68bfe1cd5959ed509032738c037
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2126
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-05-07 08:26:54 +00:00
Tomasz Zawadzki
0d105d4f1f scripts/rpc: do not list commands for the main parser
The main parser will show usaged like so:
usage: rpc.py [-h] [-s SERVER_ADDR] [-p PORT] [-t TIMEOUT] [-v]
              [--verbose {DEBUG,INFO,ERROR}] [--dry_run] [--server]
              {framework_start_init,start_subsystem_init,............

The above contains very long list of all RPCs.
This is duplicates help from the subparsers, which contain
more descriptive help message.

This patch replaces the usage with:
usage: rpc.py [options]

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ibf400987103f5571f79c57d8a9588d97f463d773
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2125
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-05-07 08:26:54 +00:00
Tomasz Zawadzki
17c2f4ee8f script/rpc: do not send rpc when no arguments provided
Any time script is called an RPC is called and socket
connection is attempted.

This should not be required when no arguments were provided,
such as calling help to explore possible options.

Example of the error:

[tzawadzk@PowerEdgeFour spdk]$ sudo ./scripts/rpc.py
Traceback (most recent call last):
  File "/home/tzawadzk/spdk/scripts/rpc/client.py", line 40, in __init__
    self.sock.connect(addr)
ConnectionRefusedError: [Errno 111] Connection refused

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "./scripts/rpc.py", line 2438, in <module>
    args.client = rpc.client.JSONRPCClient(args.server_addr, args.port, args.timeout, log_level=getattr(logging, args.verbose.upper()))
  File "/home/tzawadzk/spdk/scripts/rpc/client.py", line 56, in __init__
    "Error details: %s" % (addr, ex))
rpc.client.JSONRPCException: Error while connecting to /var/tmp/spdk.sock
Error details: [Errno 111] Connection refused

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3734e1603fa7151fd53b99b8af4ea79bf82fe0f3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2124
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-05-07 08:26:54 +00:00
Michal Berger
36e573fc6b scripts/common: Introduce cache for the pci devices
Expose a cache of pci devices in form of an assoc array that could be
looked up during the runtime of a script like setup.sh.

In case of setup.sh, caching speeds up execution quite visibly:

config run, no caching:
real    0m4.488s
user    0m1.440s
sys     0m1.260s

config run, caching in use:
real    0m2.876s
user    0m0.365s
sys     0m0.420s

Note that for initial config runs, binding controllers to proper
drivers is the actual bottleneck.

status run, no caching:
real    0m1.877s
user    0m1.252s
sys     0m0.984s

status run, caching in use:
real    0m0.371s
user    0m0.242s
sys     0m0.204s

reset run, no caching:
real    0m2.559s
user    0m1.409s
sys     0m1.322s

reset run, caching in use:
real    0m0.960s
user    0m0.432s
sys     0m0.419s

Additionally, in case common tools, e.g. lspci, are missing, fallback to
sysfs to pick all needed devices from the pci bus. Targeted for Linux
systems only.

Change-Id: Ib69ef724b9f09eca0cbb9b88f1c363edc1efd5dc
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1845
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-07 08:23:52 +00:00
Maciej Szwed
2bd001620f scripts: Add SSL certificate option in rpc_http_proxy
This is required for SSL verification when running on https.
Added argparse to handle multiple optional arguments.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Ib8731bafaaf38264ed86ed84ca7eeca5ff4935b3

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2122
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-05-05 08:04:00 +00:00
Vitaliy Mysak
20bc88d284 scripts/vagrant: fix exporting array in bash script
In create_vhost_vm we export SDPK_VAGRANT_DISTRO
variable to vagrant subprocess that is called
during the script.
But if SPDK_VAGRANT_DISTRO is an array variable,
then Vagrant will not see it
and always default to ubuntu16.
This small patch fixes that.

Change-Id: I45e9ed4f847e4fa10b90650a613f9cfe17ee38e3
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1676
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-28 13:47:46 +00:00
Ziye Yang
94345a0a1a nvme: Add the priority field in struct spdk_nvme_transport_id
Purpose: To set the priority of the NVMe-oF connection especially
for TCP connection.

For example, the previous example can be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420

With the change, it could be:

trtype:TCP adrfam:IPv4 traddr:10.67.110.181 trsvcid:4420 priority:2

The priority is optional. We try to change
spdk_nvme_transport_id but not in spdk_nvme_ctrlr_opts since
the opts in spdk_nvme_ctrlr_opts will reflect in every nvme ctrlr,
this is short of flexibility.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Sudheer Mogilappagari <sudheer.mogilappagari@intel.com>
Change-Id: I1ba364c714a95f2dbeab2b3fcc832b0222b48a15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1875
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-24 15:53:34 +00:00
paul luse
42eb233e48 scripts/rpc: addd RPC support to enable idxd
The new RPC also includes and option for which
idxd config is to be used. Config options will be
explained in detail in a later patch.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I1322bab4d6ca9cc45ec3a56d7267dc7dc09dbea3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1726
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-23 15:48:32 +00:00
Liang Yan
6a29c6a906 bdev/rbd: add ceph rbd resize function.
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Change-Id: Id7f24f1f5e0789ee6d4c07ae7f59cc2d98983e98
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1766
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-23 07:28:10 +00:00
Maciej Wawryk
6ec892634e scripts/perf: Enable percentile 999 measurements in fio config
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I32c534196108c4795bcd6cd8392e3a1b0a5dd6e9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1842
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
2020-04-17 07:31:55 +00:00
paul luse
3429f97a89 scripts/setup: add support for IDXD (aka DSA)
Add bin/unbind support between the kernel idxd driver
and VFIO. UIO not tested but tested this extensively
on a system with valid idxd kernel driver and VFIO.

Future patches in this series add the actual idxd
low level library and bdev accel plug-in module. See
those patches for more details on idxd.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I09dd00b1aedcf88577335bde79d31cfd50ac4cde
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-04-16 08:10:33 +00:00
Maciej Wawryk
36b7098a84 scripts/pkgdep: Fix failing symlink creation during python instalation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ida6bfaf9ed21c5349a3107e033391f8d30a9ff93
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1497
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-04-16 08:08:40 +00:00
Karol Latecki
485d615d7e scripts/vagrant: disable default synced folder
Disable default synced folder. We have our own set of
directories to synchronize so this is not needed.

Change-Id: Ie52e38af94fc2c403e3d3f095071833b9df09c9c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1158
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:08:02 +00:00
Karol Latecki
3a06abc9d9 scripts/vagrant: enable hugepages backing for VM memory
Change-Id: I66c1c965d9de8b4e9625c1f1850cbb28d11138ad
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1684
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-04-16 08:08:02 +00:00
Karol Latecki
7fe1b36a36 scripts/vagrant: select tool for synced folders
In order to have all the boxes fully functional we
need to select the proper tool for synced folders.

FreeBSD boxes fail to set proxy properly via vagrant-
proxyconf plugin, so they fail to install rsync and sshfs
clients. NFS is pre-installed so we can use it.

Other boxes, like generic/fedora*, have issues using NFS
so we need to use sshfs/rsync instead.

Change-Id: Ic191b5a80dbd43479943c1739971abdff39baf41
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1581
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:08:02 +00:00
Maciej Wawryk
451c9e7b69 scripts/perf: Add frequency-info after changing cpu frequency
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Icf45fbdff547fcf45c361b1d9c95b352756ad126
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-16 08:07:14 +00:00
Pawel Kaminski
fbedb897bd test/openstack: Change http to https for cirros server.
Extend build timeout, enable image volume cache and
change max spdk queue depth on local.conf.

Change-Id: I1dcacd8fbfbfabcba1d017d950dd09ce8ee68076
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/762
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-15 07:37:25 +00:00
Jin Yu
2eabc71518 vhost: add packed ring support
Update vhost blk contruct rpc, make it enable to
support packed ring feature.

Change-Id: Ia1f75e72e8441e8d82fad89073e4875f89e5b9cd
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1567
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-15 03:55:59 +00:00
Maciej Wawryk
282b1505e5 scripts/perf: Fix cpu_frequency parametr passing
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I9617cd92d20b07c32761fb67ce6a1b30e0985831
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-09 08:08:38 +00:00
Maciej Wawryk
a61af6cfc9 scripts/perf: replace dots by floor sign in percentile variable
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I34e912dd207f16b1926edb37df41128a0524bc4c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1711
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-09 08:08:17 +00:00
Maciej Wawryk
7c1e69b3fa scripts/nvmf: Fix subsystem divide and format error
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I874c3a28c453737d0ba6ab42ec0d0a9fbc4719f7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1679
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-08 06:43:16 +00:00
Ben Walker
649367850a make: Allow CC and CXX to contain spaces
On some systems, CC is set to a value with arguments already included.
Make sure not to lose those.

Change-Id: I93b354dc21f045f3cacff1f0d1214442ea893aa5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-07 08:37:57 +00:00
Maciej Wawryk
a2eeec65f8 scripts/vagrant: synchronize artifacts from vm to host
Sync "output" directory (containing all build results,
like doc, coverage, test completions, etc.)
from guest to host.

Change-Id: I9a57a72eb610aacde011fc39bb8d39c74486cef1
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1044
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
2a75be1a56 scripts/vagrant: cleanup packaging routine
Add "halt" command before packaging, it's needed for this
process. Destroy the VM once it's packaged as we have no
further use for it.

Move provision-related commands to update.sh.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ib8041df023b80b4cd5fbad734481eb6bd4ffc21c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1154
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
5277a5324e scripts/vagrant: change vagrant box sources
Some time ago I created a patch which unified all boxes
URLs to generic/* which come from a single author/company
and seemed to be quite universal.
Unfortunately some of the boxes are not that generic as
the name suggest, so I am reverting my change and will
use boxes from vaious sources.

Ubuntu 1804 - had heavy DNS issues due to hardcoded
addresses in resolved.conf. Very hard if not impossible
to fix if working with synced folders like we do.

Centos 7 and 8 - sshfs not available by default. Also
had some other minor quirks. Official boxes available
so let's use them.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I514d3b92c6ac5844b1929fdadfd35d6079933247
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1622
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
a34c17027b scripts/vagrant: fix permissions for backend files
Owner and group were not set properly on Ubuntu systems,
as they use different names for that.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie7fc49516072ce4bcd506c4c39bbd26786004c49
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1580
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
3d3e8096b7 scripts/vagrant: replace hardcoded ocssd metadata file paths
These can't be hardcoded to a single location. It makes
running multiple VMs with OCSSD emulation impossible.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I4ce0412fc9fc4542670a9e70c08219003e41828b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1488
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
53dbdf6dce scripts/vagrant: update help message
Just moving stuff around and updating some information.
I just felt like it's needed.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ia41f99603fb60111924cd5a3daf3c44091245c89
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1157
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
e16128cece scripts/vagrant: remove vhost test configuration steps
We are not using VMs to run Vhost tests (at least at
the moment) and these steps are outdated. These steps
will need a re-write if we decide to put them back in,
so we can remove them for now.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I61d09bdc481814d8bcf6fe86ed4ccd84b36ad8d9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1155
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-03 06:31:17 +00:00
Karol Latecki
2f96a090de scripts/vagrant: remove Fedora 29
Remove Fedora 29 from the OS list. It reached EOL
some time ago and is not supported.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ie1bd704a39ab4c83d5e652741d21f6f0ac260540
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1156
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-04-03 06:31:17 +00:00
Maciej Wawryk
051d3fe016 test/vm_setup: Fix permision error in Centos7
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ic704e9ae97bb68d96401818fb52940e86eaa36a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-04-02 14:51:24 +00:00
Michal Berger
576602acc4 qat_setup: Fix logic check against service's exit status
service is a plain Bash wrapper which either executes systemctl or
rc sysvinit scripts directly, hence its exit status should follow
standard logic - 0 for success != 0 for failure.

Reverse the check to not complain about service failure when it
was actually successfully started.

Change-Id: I29dd22c884157182549ebcc2ab4097eb098d7d3b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-04-02 07:03:47 +00:00
Tomasz Zawadzki
99cff8350b test: remove SPDK_TEST_EVENT flag
Test in event.sh take 5 seconds on CI at this time.
For the tests to run both SPDK_RUN_FUNCTIONAL_TEST
and SPDK_TEST_EVENT had to be enabled.

This served as very minor time saver, so removing
this flag shouldn't affter test times more that the
5 seconds.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ie3eaa587daf28b2efbc3cfdff5ad648b7529e7f1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-31 07:57:40 +00:00
Michal Berger
6b8f9a0f81 test/bdev: Convert setup_rbd_conf() to RPC calls
Change-Id: Icfbb571d1cd2a2539cd9026f1437c75450b77512
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1022
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-31 07:55:58 +00:00
Karol Latecki
051a6902d8 rpc/client.py: replace use process_time() instead of time()
time.clock() was depreceted since Python 3.3 and was finally
removed in Python 3.8.
https://docs.python.org/3.8/whatsnew/3.8.html#api-and-feature-removals

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ffe372b61c8afc0ddbbee0064b95059fa184616
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:44 +00:00
Karol Latecki
4f24f2ca4a scripts/pkgdep: change archlinux pmdk installation dir
By default "make install" installs pmdk header files in
/usr/local/include while autotest scripts expect them
to be in /usr/include.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I906ffd8a7858a37525758c3fc8129cb22286a8f5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1368
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-23 09:37:17 +00:00
Maciej Szwed
3524dd0153 pkgdep: Add ncurses to pkgdep
ncurses library will be required by upcoming spdk_top app.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I16268c2ac772a45abdbd12d8a761d46041ce6e20

Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-20 08:41:11 +00:00
Karol Latecki
2da5c24d8e scripts/pkgdep: change keyserver for archlinux rdma-core installation
Previously used keyserver keeps rejecting the gpg key.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Ifed1d5c4db61f1b6d1e985fa048cc17801046f35
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1329
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-19 08:51:13 +00:00
Karol Latecki
9bf3be9f95 scripts/pkgdep: add archlinux dependency for pmdk
pkg-config was missing for pmdk installation.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I795c4b3a71d611292beaa650960afb639e827e78
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-03-19 08:51:13 +00:00
Maciej Wawryk
b5700d4d7c scripts/vagrant: Fix unsupported machine type error
Vagrant fails during creation virtual machine using custom qemu emulator.
Setting machine type to "pc" resolved this problem.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1822439bb2235b5881d3c331c378017928953ac4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-19 08:50:53 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Maciej Wawryk
0172f27974 perf/nvmf: Add more latency measurements
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I1ff5e6440ebef50d0488f020edfd8d5b7e274162
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1281
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:02 +00:00
Maciej Wawryk
3509fc12df Add cpus_allowed_policy param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I82a1632d4bb8742bd93f33a5a094a9cb4aa2e927
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1259
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-17 16:38:16 +00:00
Maciej Wawryk
0ebd0dce19 Add subsystem division in Kernel initiator
This will allow us to use num_cores and cpus_allowed option also
in Kernel initiator side.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad4e8bf7cd8a614ada8f443a8b1036dc2c4803e4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1173
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
0eab351528 Add cpu_frequency param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id594da091d7bcd5d23a752ccfab9bfa5b7726b49
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1113
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Maciej Wawryk
e068db83f1 Add cpus_allowed param for NVMeOF benchmark tests
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I0d113e39c1ae9ff284f408998f5bba82352a2891
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/993
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-16 08:45:50 +00:00
Liu Xiaodong
f2bfad5908 scripts/common.sh: print error msg to stderr
Error msg in common.sh should be printed into stderr
in order to avoid improper usage.

For example:
  If env has no lspci and pciconf, then function
iter_all_pci_class_code will return the error msg back,
every word in error msg will be used by caller wrongly.

Change-Id: I6a875e49527539ba82bd331c8878e972e26cbc39
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1227
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-12 09:03:33 +00:00
Darek Stojaczyk
8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Shuhei Matsumoto
49cffc1e68 rpc: Add thread_get_io_channels RPC
Add an new JSON RPC thread_get_io_channels to retrieve IO channels
of the threads. IO channel don't have its own name and so get the
name of the corresponding IO device instead.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id8fbb328ffba09e1eace35994cdbf36b24832b82
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/887
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-10 09:13:02 +00:00
Shuhei Matsumoto
0b0f0c5415 rpc: Add thread_get_pollers RPC
Add an new JSON RPC thread_get_pollers to retrieve pollers of all
the threads. By adding a helper function spdk_poller_state_str(),
output poller state as string to improve readability. Most of the
code of thread_get_stats and thread_get_pollers are common and so
unify these two RPCs as possible as we can.

Sample output of thread_get_stats RPC in doc/jsonrpc.md was wrong
because thread_get_stats doesn't output ticks. Fix this together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I003ffe569d3c0651ae65c5858eff8287f7e9031d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-06 10:29:56 +00:00
Maciej Szczepaniak
12642633bf bdev/ftl: Add ftl_path parameter to rpc parsing
Signed-off-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Change-Id: I51cc68536f162b0d920d135ba491b1657e699618
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/880
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-06 10:28:21 +00:00
Shuhei Matsumoto
95b35daa5f rpc: Add thread_set_cpumask RPC
Add an new JSON RPC thread_set_cpumask to control CPU affinity of
the thread dynamically from outside of SPDK application.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6b6b794405f612200a2c3cb27dc4fc1ad2e88bec
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/501
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-04 10:03:30 +00:00
Maciej Wawryk
23f78f22cf scripts/perf: extract SKT section to new file
PCM cpu measure give as result 1306 columns, we want check only ones which contains
SKTdata and SKTtraffic values.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I52b6781a7ff99e0d894f7f1c2b25b655fa274eb1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/946
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-02 10:33:48 +00:00
Artur Paszkiewicz
339fb7cd15 genconfig.py: allow numbers in config option names
To make it possible to add CONFIG_RAID5.

Change-Id: I1712bde5bae552811343fb1865f1224d3adb1c7e
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/854
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-20 14:40:19 +00:00
Maciej Wawryk
f7d520e864 scripts/vagrant: Fix missing line break
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iaace103027e4261e01b5e13233afc4094abf2386
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/939
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-19 09:47:56 +00:00
Maciej Wawryk
af694c3de8 scripts/pkgdep.sh: Add dependencies for Centos8
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I5921d5f7f70f2df634ccd008eb27a02496a47bee
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/603
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
e5a5eaa660 scripts/pkgdep.sh: Add flag for building docs dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: If2c64eaa688fdc067479a357cbfd407570937419
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/602
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
fe5954c67f scripts/pkgdep.sh: Add flag for RDMA dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id38a3c5066c2c71d681d61f7ea8be748ddadaaf0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
2fc1ed76af scripts/pkgdep.sh: Add flag for FUSE and NVMe-CUSE dependencies
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I2cfdd33b7d0e655d80e28958aa0f12c465592cb7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
f035937dd5 scripts/pkgdep.sh: Add flag for pmem dependencies
Add flag for pmem dependencies needed by reduce and pmdk components.
Also add missing libpmem in Ubuntu section.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I596a1e6660f8b6f92bdab81046fa3533ae78fca3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
c79a950fe4 scripts/pkgdep.sh: Fix NASM dependencies Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I776d93e156c07c618df9cb5eeb99c0ca8f78e388
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
0c50f8311e scripts/pkgdep.sh: Fix libfuse3-dev dependencies for Ubuntu
Add information about missing dependencies for Ubuntu versions smaller than 19.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Id48b462be35a761fd9fad4bdc48c1757cf5a7580
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-18 14:42:16 +00:00
Maciej Wawryk
9ebbf3acac scripts/pkgdep.sh: Add flag for developer tools installation
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I8876fd7c60198c597d931c5f498644c415c2a868
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 14:42:16 +00:00
paul luse
f070b78cd1 module/crypto: add RPC and plumbing to support QAT AES_XTS cipher
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iadd03f8da305e97e3f1d57ef4aeaece400e8d959
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/441
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-02-18 08:05:42 +00:00
paul luse
e753402aa1 lib/accel: add RPC to select which accel module should be used
Startup RPC only to avoid issues trying to switch while in use.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I7f1801905de0927f3610c4065182dd8fa88cdfdd
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/660
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
paul luse
aa7a13afc7 global: rename copy to accel
The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation for adding new capabilities
in the future. Additionally, APIs for what was previously called
the `memcpy` engine have been renamed to identify the engine as a
software accelerator.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia607aa718416146fbba1e6792b8de0f66bd8a5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-18 08:05:34 +00:00
Karol Latecki
93be26a51d Fix Markdown MD022 linter warnings - headers blank lines
MD022 Headers should be surrounded by blank lines

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-17 10:07:21 +00:00
Maciej Wawryk
344c318464 scripts/vagrant: Create prebuild box and allow using spdk boxes
Build local vagrant box and add it to local vagrant box list.
Check if spdk box exist and use it instead of generic boxes.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ib3b87c9783354680657f96b5b5adea1706712c5e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/848
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
dd9a88cc7f scripts/vagrant: Unified distribution names in Vagrantfile and create_vbox.sh
Remove unnecessary case statement in Vagrantfile

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I6aa5d0650d268851ee5af73ffc2aa54d9396fec5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
22310f6176 scripts/vagrant: Make the Box as Small as possible
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I23b92fb6ef7d1169712f81d6d842db3981b8f764
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/677
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Maciej Wawryk
58e6ef44f8 scripts/vagrant: add option to build box for spdk
New option --package-box will install all dependencies needed to build SPDK

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iedc21812b754692a5eba87127186e11ec44d805d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/538
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-17 10:06:09 +00:00
Seth Howell
b4aa3074bc setup.sh: make it easier to override with igb_uio
When overriding with igb_uio you used to have to manually probe uio
before calling setup.sh. I feel this is a common enough use case that we
should check it in setup.sh directly.

Change-Id: I8a31f90bab6960eade468816ffe960820f83d2fe
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Seth Howell
615b6849b4 setup.sh: fix DRIVER_OVERRIDE behavior.
This wasn't done properly for passing the whole path to a .ko file
previously.

Change-Id: Iabfcda95537003a68f952e4e8e6bcd141d2936f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/793
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-13 09:51:54 +00:00
Maciej Wawryk
db65d58359 Add pcm measurement to nvmf tests
PCM - Processor Counter Monitor is an application to monitor performance
and energy metrics of Intel® Core™, Xeon®, Atom™ and Xeon Phi™ processors.

Using this measure tool will help us to capture memory bandwidth consumed
on Intel processors.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Ie580d2aad6ddd98c5256df65f65e478dd78317e5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/541
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-11 08:37:42 +00:00
Tomasz Kulasek
8e3b0c159b rpc: fix allow to use bdev_nvme_apply_firmware in rpc.py
Change-Id: I53c5c69b7b954a491e29a72305bfc1b6c57aabb1
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/532
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-06 09:46:17 +00:00
Karol Latecki
3d8a0b19b0 doc: Fix Markdown MD032 linter warnings
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
2020-02-06 09:46:06 +00:00
Maciej Wawryk
4c67fe81eb scripts/pkgdep.sh: Separate minimal required dependencies
We want to split pkgdep.sh to install packeges that are required to run SPDK
and that are needed by developers.
This patch is only split one install command to divide the minimal
dependencies from the developer tools.

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: I332cbc430bd04e6ef2872ab762da99e837bf0fc2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-04 19:50:26 +00:00
Maciej Wawryk
3d2ee8d1ce scripts/vagrant: allow password authentication
Add password authentication to vagrant for future
boxes building functionality

Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Change-Id: Iad9c3c4000ac43243900ee6b5396863b9ab820ff
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-04 18:19:43 +00:00
Wojciech Malikowski
dcd3fc1fd1 bdev/ftl: "use_append" parameter for bdev_ftl_create RPC
Allow for using appends instead of writes.

Change-Id: I2f0d3bcdbb0eee034f7b0b6349de854ddbf7273d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481839
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-02-04 18:10:01 +00:00
paul luse
6ea187fcbb scripts/vagrant: update readme
Fix a bunch of typos, remove a few things and add some disclaimers
for legal cleanliness :)

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I366380f7b822c5b7a5ca92ab5a9bd3fd68296bee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483752
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 17:04:12 +00:00
paul luse
e914958951 module/compress: rename set_compress_pmd RPC to match convention
set_compress_pmd -> compress_set_pmd

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb1ecc7adfe10485c44f98ab9e31eaa6857596e7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:37:31 +00:00
Seth Howell
f257be1313 doc: replace all cloudfront refs with ci.spdk.io
It's good to have a human readable domain again.

Change-Id: If773e28a267b635a94a45b07fba9a32d62d0c248
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:19 +00:00