510 Commits

Author SHA1 Message Date
Seth Howell
d63d4d5a37 scripts: add a qat card setup script
This script should be called when testing the crypto module. It will
configure any qat cards with class code 37c8 for use with the spdk
crypto bdev module.

Change-Id: I233069fbe9ba3c7a6463f52443234f9fe2c9a1f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428549
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-22 22:33:13 +00:00
Liu Xiaodong
e66ea62412 scripts/rpc.py: add method "send_nvme_cmd"
An example to send nvme_cmd rpc request:
scripts/rpc.py send_nvme_cmd -n Nvme0 -t admin -r c2h \
-c <base64_urlsafe encoded cmdbuf> -D 4096

nvme-rpc will be processed internally by bdev_nvme.

Change-Id: I6e731b76be0f503d48154a8b34a1e81b4b454396
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/417962
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-22 22:01:14 +00:00
Pawel Wodkowski
ddd38a5e9d configure: remove CONFIG.local symlink
nvme-cli is updated to use mk/config.mk so we can now remove
CONFIG.local symlink.

Change-Id: I7d39c2559e43198b9e0bc4519edaa1cb3180de07
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/428373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-19 14:26:14 +00:00
Jim Harris
840b922551 genconfig.py: only use CONFIG.local
Everything we need is in CONFIG.local - so just use that
file.  This allows us to remove the for loop as well.

CONFIG.local is currently a symlink to mk/config.mk - that
will be changed in the patch that removes the symlink.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ia7f84eac85d322f2873f089ab62c99a69c4d9ef9

Reviewed-on: https://review.gerrithub.io/429947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-19 14:26:14 +00:00
Ben Walker
ba502c17c5 pkgdep: Prefer make -C to cd
Change-Id: I3a132a87b340232f1b3739cbff3c262ab8614a39
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/429527
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-17 15:16:15 +00:00
Ben Walker
2c1aaa7604 pkgdep: Don't built intel-ipsec-mb as root
This leaves files created by the root user in the directory and
makes future calls to make clean fail.

Change-Id: Ie33d0d33e8c01a2d17f6991284c5118b5bd545ff
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/429282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-15 19:33:45 +00:00
Pawel Kaminski
09a9130ed2 test/spdkcli: Add load and save config commands.
Change-Id: I499f54b025080ad1916acc0cf265a58c806da002
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/428494
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-15 17:32:09 +00:00
Karol Latecki
1250a41664 scripts/perf: fix nvme perf scripts
Due to change of defauly Python interpreter to Python3
we need to decode byte object from check_output()
to utf-8, otherwise there is an error.

Change-Id: I83e2d79ec8c3934c5c6d00768288fbb4c5a50914
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/428172
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-11 20:46:18 +00:00
Seth Howell
5faf653f0c pkgdep.sh: Fix error handling for ipsec submodule
If the ipsec submodule is registered to spdk, an empty intel-ipsec-mb
directory will be created. We could potentially try to run make inside
of this empty directory, so instead do a preemptive submodule update.

Change-Id: I367fdef468bf21ef91b8354155d199cea97c3daa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428404
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-11 20:45:47 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
chenlo2x
93ee8c7a7f scripts/check_format: fix an escaping issue.
Some older versions of Fedora require putting the git grep contents
in quotes before feeding to the while loop.

Change-Id: I649d5520f4a8c0a903aed2e2dda337e91b5b596e
Signed-off-by: chenlo2x <longx.o.chen@intel.com>
Reviewed-on: https://review.gerrithub.io/428332
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-09 04:45:00 +00:00
Pawel Kaminski
3d0488d28a spdkcli: Add iscsi subsystem.
Add create/delete methods from iscsi subsystem to spdkcli.

Change-Id: I950b3f29720fd5f09e4144d139e6d0e5885c69c8
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/424607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 17:48:09 +00:00
Piotr Pelplinski
c3d9530cd0 spdk: fix check_format.sh
Patch 427539: Check file permissions in the check_format script
introduces at least two errors.
- exit too early from check_format.sh
- wrong file permisions on file vbdev_crypto.c
This patch fixes them.


Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I346523bce7448e749471bd78a06e9233e0d35ef2

Reviewed-on: https://review.gerrithub.io/428188
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-05 16:02:48 +00:00
wuzhouhui
74a28c199f vhost: rpc: add an optional parameter -n/--name for get_vhost_controllers
User can use this optional parameter to get the information of specific
vhost controller.

Change-Id: I3911c6c7d4e7b75e82277d1e4690d5e40019aa06
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/425451
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-04 23:33:04 +00:00
Ben Walker
d705ab93e1 Check file permissions in the check_format script
Change-Id: I572b24b2f4f149333f2e4b04f6f4961de9c23373
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/427539
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-04 23:08:12 +00:00
Pawel Kaminski
b6e130e98f spdkcli: Add nvmf support
Add create/delete/get methods from nvmf subsystem to spdkcli.

Change-Id: Id1f4e539d14746c4d6108bb58df921c301d47e96
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/423425
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-02 22:14:43 +00:00
Karol Latecki
c7917f2252 scripts/autotest: print short info for setup.sh cleanup and status
Print shorter output about unsupported option instead of
printing full "usage" text.

Change-Id: I92a98b0bdf0b2ed9ac56da644f24777f76e7df29
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426471
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-02 22:10:32 +00:00
Seth Howell
b7ceb7508f scripts/vagrant: Add fc28 image and update docs.
Keeping consistent with the distributions we support in the build pool.

Change-Id: Iec13686507f890e9e18ecd5a1f4f238b2370f1d7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/427161
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-01 16:15:22 +00:00
Seth Howell
6695ac2797 scripts/vagrant: Enable test machine deployment for fedora
Allow the option to run vm_setup.sh during provisioning on a fedora
system. This is one step closer to fully automating the setup of a build
pool test machine.

Change-Id: Ia3965b31e0a9217d176ffe3c165b8eb6b3ccad13
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426946
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-01 16:15:22 +00:00
Seth Howell
80bf04e6ad scripts/vagrant: Make rsyncing spdk directory optional
For some use cases (e.g. creating a test pool vm), we won't want to
rsync spdk from the host, and instead will want to clone a new one from
scratch.

Change-Id: I4c27f8ffc6c04aa0901dfe5b536b7e6ba94f7693
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426945
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-01 16:15:22 +00:00
Karol Latecki
ad06d03a86 scripts/spdkcli: override configshell init
Override configshell constructor to allow command paramaters to use
whitespaces and quotes.

"This patch will be obsolete once
https://github.com/open-iscsi/configshell-fb/issues/46 is fixed."

Change-Id: Ia5f883991b031901be7dadf61cbe7ebdcf5a9cb7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/426037
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-01 15:04:05 +00:00
Karol Latecki
2aaaff5ee8 scripts/vagrant: allow adjustable nvme img size
Allow specifying size for created NVMe image file.
Default is 1024M but some tests suites (vhost_initator
for example) need more.

Change-Id: I4bf7a5241a0e9c2320cd473b1787c0be25d9a8e4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2018-09-28 18:43:42 +00:00
Karol Latecki
f2d9e5144a scripts/vagrant: automate vhost test VM creation
Enable users to automatically create VM image dependency
needed to run vhost tests.

Current state of Vagrantfile_vhost_vm is only valid
for Ubuntu 16.04 and 18.04.

Change-Id: I9cdf46c1db7fcd55cfda0dd7db7ff5570610c5ee
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-28 18:43:42 +00:00
paul luse
cafd537c7d test: exercise crypto rpc calls
Most vbdevs have complete test suites that include rpc calls
to set them up and teare them down.  With crypto there's not
a lot to do w/test other than make sure it's configured and
run IO so right now that amounts to tools that use a config
(ini) file.  Adding create/delete calls here to get basic rpc
coverage in CI.

The fixes/udpates in the vbdev module and the .py files were
fixes found in enabling testing of the rpc calls.

Change-Id: I5d7d25e1c96dcf76e0055f23d27781b858e290b0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/425875
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-27 22:07:02 +00:00
Seth Howell
0ee7ebcc70 Vagrant: Supply pkg information to the guest.
The vagrant proxy plugin doesn't appear to support FreeBSD or pkg in the
same way that it supports linux distros and their associated package
managers. This change supplies the needed proxy configuration to get
around this issue.

Change-Id: I7dfbd963c5fd01d598c1456c5d8139be81777440
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426422
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-25 17:18:30 +00:00
Ben Walker
2eacfd8786 nvmf: Generate a reasonable default subsystem serial number
If the user does not specify a serial number, just set it
to all 0.

Change-Id: Ie1aaa4c1bf5f9eaf76fcc9ed6f6bd6ded2aeb200
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425324
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
2018-09-20 18:35:27 +00:00
Pawel Wodkowski
4dc0ef6f45 scripts: fix executable attribute for includes
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.

Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-18 16:34:45 +00:00
Pawel Kaminski
ca12cb3097 scripts: add config converter
Add a new script to enable converting
old INI config files to the new JSON-RPC
config file format.
This prepares for deprecating the INI
config file file format in the future.

Change-Id: I0f2bfa9a585ce3537772b662d8e4028ab08a554d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/420457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-09-18 15:24:40 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
John Barnard
183d81d0c6 nvmf: Move target opts to transport opts (part 2)
- Add independent functions to create transport with specific opts
  and add to target while maintaining backward compatibility with
  current apps and rpc configuration that still use the add listener
  method to create a transport.
- Add new rpc function to create transport and add to target.
  + Update json reporting to include new rpc function.
  + Update python scripts to support new rpc function.
  + New nvmf test script (cr_trprt.sh) to test new rpc function.

Change-Id: I12d0a42e34c9edff757755f18a78b722d5e1523e
Signed-off-by: John Barnard <john.barnard@broadcom.com>
Reviewed-on: https://review.gerrithub.io/423590
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-17 20:42:16 +00:00
Pawel Wodkowski
b96f97cf29 scripts: use python3 in all scripts
This it to prepare for RPM package. Also lower number of dependencies
needed by SPDK tools.

Update changelog to deprecate Python 2 and explicit interpeter invoking
in scripts.

Change-Id: I2497cca721cbcbadc1c99c675f8b8b7f682d5efa
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425233
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-14 22:24:30 +00:00
Ben Walker
aedac1a878 nvmf: Add an nvmf_subsystem_create rpc
Add a new nvmf_subsystem_create RPC. The current RPC,
construct_nvmf_subsystem, is old and jumps through a lot
of hoops to maintain backward compatibility. Leave it in,
but create a new RPC as the main mechanism going forward.

Change-Id: Ia99cb5994a90579956136dac1800ed1fdce482dd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/425169
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-12 13:20:32 +00:00
Avinash M N
1f3f160581 nvmf: Add IP based load balancing
This patch adds support for IP based load balancing. All the rdma connections
coming from a particular IP will be assigned the same core on target. The load
balancing policy - roundrobin/hostip can be configured through rpc during startup only.
Roundrobin is the default policy.

Change-Id: I7bb216cb4cb2639fa401e23090143c4cf1d7bb5a
Signed-off-by: Avinash M N <avinash.m.n@wdc.com>
Reviewed-on: https://review.gerrithub.io/422190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-11 23:55:25 +00:00
Jim Harris
55014ddf53 rpc: remove file parameters from save/load_config in rpc.py
User can pipe or redirect data as needed.  As features like
bdev crypto are added, storing config data in files needs to
be carefully scrutinized - don't make it easy by providing
ways to write data to files that may not be sufficiently
protected.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I24dd32f81b965e275e1bac875f6d008d1b68007a

Reviewed-on: https://review.gerrithub.io/425165
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-11 21:11:51 +00:00
Vitaliy Mysak
59caaa979c scripts: change shebang
Changed "/bin/sh" to "/usr/bin/env bash" in scripts/pkgdep.sh
Previously script failed on my Ubuntu server, with error:
	trap: ERR: bad trap

Change-Id: I054c0388c462e2d0340b6e3d4e581a8e6cdc7097
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/424998
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-10 17:44:10 +00:00
Jim Harris
ff6299f831 build: remove DPDK_DIR
This is an artifact from before SPDK had a configure
script or a DPDK submodule.  Make configure the
only supported way for specifying the location of the
DPDK installation to use with SPDK.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5c197c46220928bb18b97c8807755967d76ea42c

Reviewed-on: https://review.gerrithub.io/424893
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-07 16:01:28 +00:00
Pawel Wodkowski
8cf4a7af05 scripts/pkgdep.sh: distinguish Fedora, CentOS and RHEL
Fixes #417

Change-Id: Ic2f00d624c9e20386aab1eb1ba0e24d12fdf56d8
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424413
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-05 22:14:41 +00:00
Pawel Wodkowski
e1c38cad29 scripts/pkgdep.sh: make libunwind-devel package optional
Change-Id: Ief1df1e08c8efdbd53865c5b0f5ae2c26973a90a
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 22:14:41 +00:00
Jim Harris
ba38785b7e bdev: remove get_bdevs_config RPC
This RPC does not work for a lot of bdev types.  For
example, NVMe namespaces and virtio scsi LUNs are not
explicitly constructed by an RPC - they are indirectly
constructed by an RPC associated with an NVMe controller
or virtio-scsi controller.

While here, remove spdk_bdev_config_json.  It was
only created to facilitate this get_bdevs_config RPC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I537166d8f91ab458bd2000859d74f7254bfc9c0a

Reviewed-on: https://review.gerrithub.io/424584
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-05 19:46:17 +00:00
Pawel Wodkowski
93fb58f29d scripts/pkgdep.sh: don't ignore errors
It is likely that bizarre errors occur if any of these commands fail
so don't continue but just exit with an error.

Change-Id: I007192f22c89f3d7f92b1f4a0cc31cb50b605dc3
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-04 18:51:57 +00:00
Pawel Wodkowski
6637dd32be scripts/pkgdep.sh: use 'uname -s' only for FreeBSD
Change-Id: Ia3c0d9f91d9bcc8bfd7254a24a1f20c48c5cd50e
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/424411
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-04 18:51:57 +00:00
Jim Harris
a2794082ca doc/vagrant: add cd to examples/bdev/hello_world
This ensures the bdev.conf file is found.

Reported-by: Amir More (in GitHub issue #420)
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I309d8f211e0a3e0e4530c5d3c225a52b6ea68c2c

Reviewed-on: https://review.gerrithub.io/423931
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-30 03:41:01 +00:00
Seth Howell
b6770c73cc pkgdep: minor fixes
CHange double brackets to single brackets for compatibility with
freebsd, also added -y to a pkgdep call to avoid the script hanging when
called programmatically.

Change-Id: I9a851005efae77b9146f14cac8079e2d448143d4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423936
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-29 23:49:12 +00:00
Shuhei Matsumoto
3a08001dd4 iscsi&scripts/rpc: Add get_iscsi_auth_groups RPC to get current configuration
Add an new RPC to get current authentication group configuration.
This patch is utilized in the next patch to support JSON config dump
for authentication group configuration.

Change-Id: I34be9e196f8e7a484bcd316da54f05d0f6ee0300
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-29 21:40:31 +00:00
Shuhei Matsumoto
ad323b8064 iscsi&scrpts/rpc: Require to specify CHAP secret file explicitly to load it
Previous patches enabled users to configure CHAP secrets dynamically
by RPCs. Subsequent patches will enable users to load CHAP secrets
from JSON config file.

Loading CHAP secret file is done by default and this will conflict to
JSON config file.

Hence the path to CHAP secret file is required to specify in the config
file or JSON RPC set_iscsi_options explicitly after this patch.

Users who have used CHAP secret file are expected to specify it explicitly
and this will be no harm for them.

Besides, CHAP secret file is not oly for discovery sessions but also for
login to iSCSI targets. However there were wrong description to make user
misunderstand. Hence remove these wrong description in this patch too.

Change-Id: Ic4093cabc0c14b87e26baef4bba6b0d292e40c06
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-29 21:40:31 +00:00
Shuhei Matsumoto
750e5d25fb iscsi&scripts/rpc: Add add/delete_secret_to/from_iscsi_auth_group RPCs
Add RPCs to add/delete a secret to/from an existing authentication group
dynamically.

Use mutex to ensure exclusive access to CHAP secrets.

Tries to use descriptive message in RPCs when error occurs.

Change-Id: I59650ae11a2fe675d03b90bbd4d2dc5b9c0160ed
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 21:13:49 +00:00
Shuhei Matsumoto
dcafd5b1ea iscsi&scripts/rpc: Add add/delete_iscsi_auth_group RPCs
This patch adds new RPCs add/delete_iscsi_auth_group to add and
delete authentication group dynamically.

Mutex is added to ensure exclusive access to CHAP secrets.

Additionally provide descriptive message in RPC when error occurs.

Change-Id: Iaddfbdd5688ca7907d2c7d859835faa056deecd1
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/421464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 21:13:49 +00:00
GangCao
d393983d74 lib/iscsi: export FirstBurstLength as user configurable parameter
According to https://tools.ietf.org/html/rfc3720, the
default value for the FirstBurstLength is 65536 bytes
while SPDK iSCSI target picks the smaller 8192 as the
default setting. This value is the communication for
the iSCSI initiator to send the unsolicited data and
instead of having a fixed setting here, expose it as a
user configurable parameter to fit the real use case,
especially for the data out iSCSI write.

Example of usage as following in the iSCSI.conf:

FirstBurstLength 8192

Change-Id: I71690c7c48aa0875f1f975c0ea935389de6d1e6d
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/421142
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-27 19:42:07 +00:00
Paul Luse
0fb1e1c524 ipsec: updates to require ipsec installation
Although a pre-req to crypto only, the latest ipsec library now
requires root to install.  This patch takes 2 approaches to making
sure the ipsec module is installed properly.  First, in configure,
if crypto is enabled it will check to see if ipsec is already installed
and if so do nothing.  If it is not it will exit and instruct the user
to manually install and re-run the configure script.

Secondly, to be proactive, this patch adds the installation, if not
already installed, to the pkg dependency script because it is already
executed as root.

It also does the same for nasm, checking the required versions in both
places.

Change-Id: I037629f21be474b88c12451733ac12ff0b1651f5
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/422746
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-27 16:33:16 +00:00
Jim Harris
92861ff5f8 scripts: remove --skip-alias from which commands
--skip-alias is not supported in all versions of which.

While here, explicitly import CalledProcessError - some
versions of Python didn't seem to mind, but it's not
true for all versions.  Also print related error
messages to stderr instead of stdout - this helps
with debugging since in some tests the stdout is
redirected to a file.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I7637adc36da6290ece28c77b3398185191347eef

Reviewed-on: https://review.gerrithub.io/423414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-27 14:58:29 +00:00