Commit Graph

343 Commits

Author SHA1 Message Date
yidong0635
d47468a151 test scripts: add rpm-build package in vm_setup.sh
This package is needed in vm_setup.sh for vpp,
without this will report "rpmbuild: command not found".

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic00a55addc940e359aae17b45e58b9a736b08fd3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/447282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-03-18 12:55:02 +00:00
paul luse
5133af0921 test scripts: add -E to sudo git clone in vm_setup.sh
Running as a non-privileged user, sudo git clone won't work
without the -E because it won't get the proxy settings, for those
using a proxy.

Change-Id: I8b471abf0066312b0eb63449d00a5647737c0136
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-03-04 19:12:10 +00:00
Jim Harris
577b667ac3 test/env: add UNIT_TEST_NO_PCI_ADDR
Similar to UNIT_TEST_NO_VTOPHYS, we need this since
a future patch will result in some of the pci_addr
functions getting linked in via a new dependency
between memory.c and init.c.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I61bd4e172ee70a5dcb99c8c6fc1fb19070a2a7cd

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/446460
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-03-04 14:00:16 +00:00
Vitaliy Mysak
34bdceabb8 ocf: update autotest to use OCF submodule
Make autotest.sh use OCF submodule as target

Change-Id: I1f6411c1b143d3c53ffa405a5bf01fdbcd4e9868
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444844
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-27 17:26:51 +00:00
Vitaliy Mysak
80ff32cfcc scripts: vm_setup.sh fix OCF github repo path
Fix wrong url for git repo for OCF

This patch is connected to issue #670

Change-Id: I030889089a4b0433517dd909246a3bc16b67c71b
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/445249
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-21 18:02:11 +00:00
Jim Harris
f74d069ec0 env_dpdk: move vtophys.c contents to memory.c
CPU profiling on workloads with intensive vtophys
operations (i.e. very small CB-DMA transfers) exposed
overhead introduce by spdk_vtophys having to call
spdk_mem_map_translate in a different compilation
unit.  Let's just move the vtophys.c contents into
memory.c so that spdk_vtophys can inline
spdk_mem_map_translate and avoid this extra overhead.

This of course breaks the memory and vtophys unit
tests, so some additional changes are needed there
to keep everything linking correctly.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I295ed5f441d3eec7abdbc9d881c49d2174ec9f48

Reviewed-on: https://review.gerrithub.io/c/444975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-20 16:41:46 +00:00
Wojciech Malikowski
0699216fac ftl: extended nightly tests
Change-Id: I8dc3e895f6b360f3ad81f157db502dd58629c5e6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442878
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-12 19:45:17 +00:00
Ben Walker
8abfb06e31 thread: Optionally allow the current time to be passed to
spdk_thread_poll()

This is an optimization if the calling function already knows the
current time.

Change-Id: I1645e08e7475ba6345a44e0f9d4b297a79f6c3c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443634
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-11 22:58:45 +00:00
Darek Stojaczyk
0b4da45dbc vm_setup.sh: add iptables dependency
We started to use iptables in patch 21bd94275
(libsock: add functional tests) but never added
the package dependency.

Change-Id: I651f2545a11f546f8b47f9759fbaed3a141f0928
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 19:23:55 +00:00
paul luse
d9d4e40dd2 bdev/compress: Add configure option and build dependencies
Includes the required DPDK dependencies for SPDK block Reduce aka
Compression.

Change-Id: Ic1ea3cbeb9373a7700f6f0c2a3194d65d6a34a41
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/429523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-11 19:23:17 +00:00
Wojciech Malikowski
a4968634f1 test/common: spdk_mempool mock configurable element size
Change-Id: Ia93b4d7ea3ce77b8c45d34eedb5201ca5ea34688
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-06 17:51:53 +00:00
Seth Howell
5232a73f38 autotest_common: add waitforblk_disconnect
This is the functional opposite of waitforblk.

Change-Id: I5e9f77c9ea05b83b8dbfbf49bb3baa2ae8cc1086
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443316
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-02-06 17:33:18 +00:00
Wojciech Malikowski
eccf549159 test/common: Added spdk_ring_count mock
Change-Id: Iefb4c76f2db5416d185556fa8c84a36291e72d3e
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443237
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-05 18:02:15 +00:00
Darek Stojaczyk
8604e568cb autotest: introduce SPDK_RUN_FUNCTIONAL_TEST
Introduced a new variable to run functional tests.
It's enabled by default, and can be manually disabled
on systems where e.g. only unit tests are run.

SPDK_RUN_FUNCTIONAL_TEST is a supplement to SPDK_UNITTEST.
The two are completely independent - both can be enabled,
disabled, or run in any combination.

The new variable is prefixed SPDK_RUN_ as it aligns nicely
with SPDK_RUN_CHECK_FORMAT, SPDK_RUN_VALGRIND, and
SPDK_RUN_ASAN, all of which control how much is tested.
SPDK_UNITTEST should eventually follow the same pattern
as well.

This gives us 2 layers of configuration:
SPDK_TEST_* <- what is tested
SPDK_RUN_* <- how it is tested

The following would run UT+ASAN for FTL and BlobFS, without
running their functional tests:

```
SPDK_RUN_FUNCTIONAL_TEST=0
SPDK_RUN_ASAN=1
SPDK_TEST_UNITTEST=1
SPDK_TEST_FTL=1
SPDK_TEST_BLOBFS=1
```

Change-Id: I9e592fa41aa2df8e246eca2bb9161b6da6832130
Signed-off-by: Seth Howell <seth.howell@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442327
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-04 19:19:36 +00:00
Wojciech Malikowski
ca558b6163 bdev_ftl: Added FTL bdev functional tests
This patch introduces functional tests for FTL bdev.

The tests cover various I/O workflows and check data integrity. Several
scripts have been added to test the FTL library:
 * generate_config.sh - prepares configuration scripts for specified
 device
 * restore.sh - tests restoring device's state from the SSD
 * fio.sh - runs tests based on fio and fio_plugin

The tests are run from autotest.sh when the SPDK_TEST_BDEV_FTL flag is
set.

Change-Id: I561d99ed35fe91eadd3756789cc99afe2da8c1db
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431330
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 17:33:10 +00:00
Darek Stojaczyk
6903c407a5 test/common: suppress waitforlisten output on BSD
We currently use:

`./rpc.py 1>&2 2>/dev/null`

And apparently our stdout redirection gets treated as
a python argument, which is not what we intended. This
caused waitforlisten function to print some unnecessary
messages.

To fix, we could either wrap python with extra brackets
`$(./rpc.py) 1>&2 2>/dev/null` or use &>/dev/null. We
go with the latter.

Fixes #616

Change-Id: I49d2380392435447b7b93259055c1d02c4b9c157
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 14:00:10 +00:00
Ziye Yang
552e21cce6 spdk: Add ISA-L support with related crc32 function
In SPDK, we will build isa-l with no shared option
and then integrate it into SPDK. And we do not need
to install isal in the system libaries.

Note: ocf build in autobuild.sh now needs to build
include/spdk/config.h before building the ocf library,
to ensure that header is available in a clean build
environment.

Change-Id: I3f0ce6932b386de17a77cf5bfdfd738b22417e2d
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Signed-off-by: paul luse <paul.e.luse@intel.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441279
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
2019-01-29 08:31:00 +00:00
Vitaliy Mysak
ec2e259402 scripts: vm_setup.sh add OCF dependencies
Add auto installation procedure for OCF dependencies

Change-Id: I75fc1b26e0eb7132a310cff62cad61bb7c3b05ee
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439359
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-29 06:10:36 +00:00
Vitaliy Mysak
2fde729fd7 OCF: tests: add integrity test
Add SPDK_TEST_CAS suite
Add basic integrity test with multithread-multidevice fio with cache
  modes WT and PT

Change-Id: If2916558894d9f7efd9628eb99f7338c90b312b1
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-28 09:41:52 +00:00
yidong0635
310df31b88 test: add installing python3-pandas in vm_setup.sh
Add installing python3-pandas.
put it into vm_setup.sh, this package is needed by autorun_post.py

Change-Id: I76c98c54f16be69879ac05d5128a049fc693e455
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/435968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-18 15:55:32 +00:00
Wojciech Malikowski
4a4e928524 test/common: Changed spdk_ring mock to be thread safe
This change is required for unit tests that use spdk_ring mock
in multithreaded scenarios.

Change-Id: I8f5e83de69a685078cbb01e55daac650f6ae1db1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440721
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-01-17 20:58:57 +00:00
Ben Walker
9cba82b955 thread: Rename spdk_free_thread to spdk_thread_exit
This name more closely resembles pthread_exit, which is a
closer analogy to how the new threading library works.

Change-Id: I68b04509f3ff8e94b8688804a7e5661155a3ecd1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
b1c79d722b thread: Rename spdk_allocate_thread to spdk_thread_create
This mirrors pthread_create, which works more closely
to the new style where SPDK libraries can spawn their
own threads.

Change-Id: Ic524c4c35bcf7c1611e4f261ebb64b98ac5a5a1b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
1068e22da5 thread: spdk_free_thread now takes a thread parameter
Instead of implicitly grabbing the thread from the thread
local variable, make it explicit.

Change-Id: I733fad06181439e12b1e71a4829b84e7b64e2468
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440595
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-17 11:24:38 +00:00
Ben Walker
6728c4f4c0 thread: Eliminate function pointers in spdk_allocate_thread
These are no longer used by anything.

Change-Id: I0db6bc88e4dc945ff4f64df2ac410e1d00a669c1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-17 11:24:38 +00:00
Ben Walker
5977aad8f7 thread: Eliminate use of pthread_self and thread_ids
SPDK threads are no longer mapped 1:1 to system threads. They are
instead identified by the memory address of the spdk_thread object.

Change-Id: I417f8376cebd2ee94f624f4436e6394b51486063
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437999
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-15 16:53:12 +00:00
Ben Walker
9bef42f284 test: Use ut_multithread framework in bdev/pmem
Change-Id: Ib9de83db429e4ae6f8e0cc988ca161cebdb27c9c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-15 16:53:12 +00:00
Wojciech Malikowski
ef2e614cef ftl: Initial implementation
This patch adds core FTL (flash translation layer) submodules.

On regular SSDs the FTL is part of firmware, whereas Open Channel
enables moving it to the host and allows for having full controll
of data placement on the device.

Main functionalities added:
 * logical to physical address map
 * read / write buffer cache
 * wear-leveling
 * bad block management

Change-Id: I5c28aa277b212734bd4b1f71ae386b3d6f8c3715
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Signed-off-by: Jakub Radtke <jakub.radtke@intel.com>
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/431322
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-11 09:15:39 +00:00
Darek Stojaczyk
8c4ed83b49 vtophys: add length parameter to the vtophys function
This follows the same trend as the mem_map APIs.

Currently, most of the spdk_vtophys() callers manually
detect physically noncontiguous buffers to split them
into multiple physically contiguous chunks. This patch
is a first step towards encapsulating most of that logic
in a single place - in spdk_vtophys() itself.

This patch doesn't change any functionality on its own,
it only extends the API.

Change-Id: I16faa9dea270c370f2a814cd399f59055b5ccc3d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-09 08:24:59 +00:00
Karol Latecki
2a9667066a test/vm_setup.sh: Add sshpass
sshpass used in test/nvme/hotplug.sh test but not present
in vm_setup.sh dependencies.

Change-Id: Ic9d7f137f7694e5318f6f412b1e7a73536a82f8c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438788
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-04 00:52:28 +00:00
yidong0635
6907c36f2c test/common: add stop.sh in ceph setup testting
Add stop.sh before start.sh,to avoid reporting resource busy
it could lead to start failed.

Change-Id: I1756adaad31937a7a4b95fdc488f2334360d4ac2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/437616
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-19 17:04:28 +00:00
Ben Walker
536beb1306 test: Use regular assert in allocate_threads
This allows us to call it during test set up instead of
only within a unit test.

Change-Id: I2daae8d630729b5e0712057766a3cba8812109e9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/437479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Ben Walker
7b94053827 thread: Rename spdk_thread_msg to spdk_msg_fn
This is the callback type used for message passing. The
old name is easy to confuse with the callback to pass
a message to a thread or the upcoming callback to
spawn a thread.

Change-Id: I5fd63b57c4be2a4262a197850e6de4901be03ee7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-18 23:31:08 +00:00
Ben Walker
605e530a4e thread: Use TLS to accelerate thread look up
Change-Id: I8136db265c9cb8d61de4845ce6eaff2351b5b597
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/435939
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-14 18:54:03 +00:00
Karol Latecki
0fc2933906 test/vm_setup.sh: update qemu to version 3.0.0
Use Qemu v.3.0.0 from spdk/qemu.

Change-Id: I44ecdac5f591a6096ec8450dd70044a7ba6628e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/436476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-10 22:02:39 +00:00
Seth Howell
1500792f52 test: modify coredump maximums to 5G
Having unlimited size core dumps is not a good idea on a lot of
machines. Modify it to 5GB.

Change-Id: I1d52bfa9f2450e2d8f824c3b86aa2ad5fe4579c3
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-10 10:03:04 +00:00
Darek Stojaczyk
76a7517caa pci: remove virtio-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: I2ec68f20ba209f02ee5c2de4b6fe5330a4bc0853
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Darek Stojaczyk
41130653b7 pci: remove ioat-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: Ia92bd8f4525712bd27ade16ead67435c5e0fbe7a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Darek Stojaczyk
576f08f732 pci: remove nvme-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: I7558590e41e5c580a130a6aba7ae4f7dcff58da8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Pawel Wodkowski
db1236ef10 autotest_common: waitforlisten - add 20s timeout and FreeBSD support
In case the process we are waiting for fail to start
listening on RPC socket the test script will stuck till the build times
out. Fix this by adding 20s timeout and 0.5s sleep in the while
loop.

For systems where there is no ip command and netstat output is missing
'Status' column for Unix sockets (like FreeBSD) call the get_rpc_methods
RPC command to check if process is listening.


Change-Id: Ia8b06af7875b65a7fd8be65cf55e92881f6f95db
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/433102
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-05 00:35:35 +00:00
Pawel Wodkowski
f751ea1723 autotest_common.sh: enable trace flag only if it was anabled before
We are too verbose in many places. Some scripts don't want to be traced
all the way, instad they enable tracing only for some parts and
autotest_common.sh just do 'set -x' in many places.

This patch save 'x' flag on function enter and restore it at exit.

Change-Id: I39b3d3dd3f711e1131e476f9d322d9e1b097ad12
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/432604
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-05 00:35:35 +00:00
yidong0635
4b88ed5687 test: add a needed package in VM setup list.
filesystem need support btrfs.

Change-Id: Ie82d87dfd98ffcbeb674313e1ef4c5097763ad40
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/435425
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-12-04 15:46:56 +00:00
Pawel Wodkowski
bf9dbae56b tests: wait for each start and stop nbd disk RPC
This workaround is needed to stop EBUSY errors from NBD ioctls(). This
can be fixed in NBD bdev code but till that point lets merge this
workaround.

Change-Id: Ic1546963538beda4d4409cef93062103e2a23e34
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/434040
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 07:28:31 +00:00
Seth Howell
be5d98000f test: adjust scripts for 01.org qat driver
Change-Id: I1cf9d27d05f09c199ef35066f2e62480025d3feb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/433185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-15 20:08:16 +00:00
Ben Walker
3c98150821 thread: Update unit test to use new threading logic
No longer override the send_msg implementation.

This updates ut_multithread.c, which has minor ripple
effects into bdev_ut.c. But that unit test is otherwise
unchanged.

Change-Id: I2fd30a1010bdff0a810d376d985ab1b8a2b22fb9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/424262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-15 18:17:34 +00:00
Jim Harris
8c50e8c106 test: handle align=0 in test_env spdk_malloc
DPDK treats align=0 as effectively align=8.  But the
test_env spdk_malloc uses posix_memalign and passes
the align as-is.  posix_memalign requires a non-zero
align parameter however.  So change the test_env
implementation to match DPDK.

This fixes libreduce unit tests, since it does some
allocations with default alignment.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If2fbad0bcc3bc92b50b50f21d7e893e0ba05e325

Reviewed-on: https://review.gerrithub.io/433083
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-14 09:58:18 +00:00
Ed rodriguez
9ba446a25c test: Add support to configure ipsec git repo
Also make ipsec configuration and install optional
  in pkgdep.sh.

Change-Id: I0c973398cebce0f50c704b21f6397e8d996ba9f9
Signed-off-by: Ed rodriguez <edwinr@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/432424
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-12 22:27:35 +00:00
Piotr Pelplinski
dadd2a6dc0 bdev: add unit tests for double buffering in bdev modules
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: Idcfb328a26fc85f3c144da9c27086a71c00d0d46
Reviewed-on: https://review.gerrithub.io/427830
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
2018-11-08 23:11:17 +00:00
Jim Harris
970228038f reduce: add configure option
--with-reduce required to build reduce.  This depends on
libpmem being installed.

We still need to work out details in pkgdep.sh and
vm_setup.sh.  Some distributions like Ubuntu still
require configuring extra package repositories to
get libpmem packages.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4e056ce1da9a1fecb4458f8f5e7ff5d61c422533

Reviewed-on: https://review.gerrithub.io/430646
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-08 00:01:47 +00:00
Tomasz Kulasek
21bd942751 libsock: add functional tests
Change-Id: I3c86c1b4a9c8ae3f18264f33b15d647e4081a12c
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/421911
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-07 17:08:27 +00:00
Pawel Wodkowski
ea0744f435 autotest: match exact address in waitforlisten function
Current behaviour will match anything that contain address. Change this
to match whole address only.

Change-Id: I6343fb91ed624cba41adbc33d711fb48c2f167a1
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/431530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-02 17:55:25 +00:00
Seth Howell
38d2cb200d vm_setup.sh: small fixes
Refactor the printout of autorun-spdk.conf and fix the path for
pkgdep.sh (the previous path was based on an old assumption)

Change-Id: Id95e0f80f835dd1b055a94a6bfdbffcce029b8af
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430843
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-31 21:55:49 +00:00
Seth Howell
33df76dc93 test/bdev: change crypto device conf based on environment
Also, remove bdev_svc memory limit to prevent it from hanging while
handling discover_bdevs calls.

Change-Id: I2868839f40931b964d49a90a86dfb391e2247e30
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/429535
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-24 23:29:28 +00:00
Seth Howell
2772d86c23 test: remove -g from discover_bdevs call.
Forcing bdev_svc to use a single hugetlbfs file causes the crypto bdev
to fail to properly allocate qpairs for qat devices. The qpairs come
back from the dpdk function queue_dma_zone_reserve misaligned.

Change-Id: Ib0c95856031ca89f6528ea6d39ef68e03e2f9009
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/430372
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-22 22:33:13 +00:00
Seth Howell
d63d4d5a37 scripts: add a qat card setup script
This script should be called when testing the crypto module. It will
configure any qat cards with class code 37c8 for use with the spdk
crypto bdev module.

Change-Id: I233069fbe9ba3c7a6463f52443234f9fe2c9a1f7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428549
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-22 22:33:13 +00:00
Darek Stojaczyk
d68ee5b431 autotest/freebsd: fix contigmem.ko path when using external dpdk
Change-Id: I9997c848ecb2514fbaf0ae9f4a0eb80d5e1522bc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-17 15:21:05 +00:00
Darek Stojaczyk
d94846c7f0 autotest: remove usages of deprecated --with-raid
The RAID module is no longer marked experimental and
is now built by default. Using '--with-raid' was giving
us an additional deprecation message:

> --with-raid option ignored and is now deprecated.
> RAID module is always enabled.

Change-Id: If9cc65a4900fc93d07fa88a3ebe3498001916285
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/429179
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-12 22:47:50 +00:00
Ben Walker
d705ab93e1 Check file permissions in the check_format script
Change-Id: I572b24b2f4f149333f2e4b04f6f4961de9c23373
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/427539
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-04 23:08:12 +00:00
Karol Latecki
de2cd83828 test/vm_setup.sh: add missing assume yes option
Dnf install for perl-generators had "-y" option missing

Change-Id: Ib12edcc11902f73824d7ed27d89d00b1631b51f2
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/426077
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-10-03 16:16:03 +00:00
Pawel Wodkowski
ae2b2f64ea log: add a way to turn on and off stack unwinding in logs
As a side effect, if SPDK_LOG_DISABLED is used in spdk_log() as log
level then no message will be printed.

Change-Id: I2d57b60a5a310a9ef2a1187a81088d0acf828742
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425105
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-02 22:09:27 +00:00
Seth Howell
30974c72ed test/common: change branch of nvme-cli
Update the branch to point at our code rebased on the most recent
mainline release of nvme-cli. This branch includes fixes for
compatibility with gcc 8.

Change-Id: Ie1bdb046d1e24e832bda585dd9841619451ec5f5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426985
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2018-10-02 22:08:12 +00:00
Seth Howell
f1b3df73b6 test/common: Configure repo locations inline in vm_setup
This allows a userto run the script without specifying a configuration
file. Removing this dependency opens us up to things like calling
vm_setup directly from a vagrantfile without having to first copy the
spdk source over.

Change-Id: I72074a445f8befc714c03cab57a2da539350c092
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-01 16:15:22 +00:00
Karol Latecki
f2d9e5144a scripts/vagrant: automate vhost test VM creation
Enable users to automatically create VM image dependency
needed to run vhost tests.

Current state of Vagrantfile_vhost_vm is only valid
for Ubuntu 16.04 and 18.04.

Change-Id: I9cdf46c1db7fcd55cfda0dd7db7ff5570610c5ee
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-28 18:43:42 +00:00
Seth Howell
a0a92be2df test/common: parallelize vm_setup.sh
We can significantly speed up the vm_setup.sh process by installing
dependencies in parallel.

Change-Id: I424efdf6da26a7f1394fbdca1cb6129e7329fc16
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426981
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-09-28 18:29:29 +00:00
Seth Howell
5ae7353072 test/common: remove directory switching from vm_setup.sh
By keeping the main shell in the same directory the whole time, we will
be able to parallelize a lot of the work of cloning and making
dependencies. This should significantly reduce make time to run this
script on a target machine.

Change-Id: I5efbd9c8f50b95411cbe7be69905b08df557344e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426980
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-28 18:29:29 +00:00
Seth Howell
0c04c729ba vm_setup.sh: move all installations under a single command
This significantly decreases the amount of time it takes to run this
script.

Change-Id: I5424cd7c9f3caf49a529916d528c0cc3a7af8838
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426947
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-28 18:29:29 +00:00
Seth Howell
99850ca7d5 vm_setup: add nvme-cli repo
The custom nvme-cli code is used in both the NVMe and NVMe-oF tests.

Change-Id: I51aab2050e1921aac899970e7abd3f1cd64836ba
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425642
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-20 21:55:18 +00:00
Pawel Wodkowski
4dc0ef6f45 scripts: fix executable attribute for includes
rpmbuild pointed out that this is wrong to have executable files without
shebang. Taking the opportunity and fix this for other files too.

Change-Id: Ib21f436672150edc0aff511bff2eb6839870cf79
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/425382
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-18 16:34:45 +00:00
Pawel Kaminski
ca12cb3097 scripts: add config converter
Add a new script to enable converting
old INI config files to the new JSON-RPC
config file format.
This prepares for deprecating the INI
config file file format in the future.

Change-Id: I0f2bfa9a585ce3537772b662d8e4028ab08a554d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/420457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-09-18 15:24:40 +00:00
Paul Luse
51606ed402 bdev: Add crypto virtual bdev module
Initial support for softare AESNI_MB DPDK driver only.

Have tested (both aesni and QAT seprately and concurrrently) on underlying NVMe devices
with bdevio and a bdevperf script that runs IOs from 512B to 128K each with Q depths from
1 to 512 in powers of 2 for 30 seconds each run.

QAT can be included in the code (but not makefile) and marked as experimental
until we are ready to test in CI.  It works well on 2 systems but is a big PITA to get
the hardware setup and configured for use with DPDK (IMHO).

Change-Id: If518c3df8e74e00efa18afdf194824c5e69778fc
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/403107
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-17 21:23:14 +00:00
Seth Howell
92a5fb593c vm_setup: add sshfs
sshfs is used to mount the shared filesystem used for communication
between the pool and agents.

Change-Id: I7c2a216ac57e394915f99bccf39d62d2ee17e5fb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/425513
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-09-15 00:29:53 +00:00
Karol Latecki
af32aa1b4e test/nvmf: enable tracepoints for nvmf_tgt
Enable tracepoints and move shm file to
the output directory in case of crash.
Cleanup all shm files after successful run.

Change-Id: Ie46c86ddda71015038624f9b2b5fd4a3e1d7f61f
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/425078
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-12 16:58:58 +00:00
Karol Latecki
f0a2cc8247 test/common: check if pid is alive before kill
Need to check if process still exists before killing it.
Otherwise if process crashed and no longer exists kill will
return with "no such process" error and prevent other steps
in trap from executing.

Change-Id: I4402d06a4fafa09dc13fba4ef9ebeb2b2fc686bb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/425353
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-09-12 16:58:58 +00:00
Seth Howell
a3595cbec1 autobuild: add shared object build to tests
This was not being tested on every patch previously.

Change-Id: I0a1756f3709da6608f66a2160962ed9b0fb38c7b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424890
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-09-10 17:50:06 +00:00
Seth Howell
8b45e46698 test/common: update dpdk directory information
The default directory for dpdk on centOS7, fedora28, and ubuntu 16.04 is
/usr/share/dpdk/x86_64-default-linuxapp-gcc. update autotest_common.sh
to reflect this. Also, add a build pool flag to control whether or not
to ignore that direcory.

Change-Id: I618b902ed04dd4a9f63c5bd2014a9fbb7fcbf91b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/424596
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-09-05 19:48:22 +00:00
Jim Harris
86d77e2eb6 bdev: account for missed qos timeslice timeouts
There could be cases (especially in virtualized and/or test
environments) where we could accumulate significant skew in
the timeslice frequency.  Rather than depend on the application
framework to try to guarantee the rate of timeslice poller
callbacks, keep track internally of the last time the poller
was invoked.  If/when we accumulate and detect skew equivalent
to one or more timeslices, increase the allowed IO and bandwidth
of the next timeslice to accomodate.

Since bdev poller now calls spdk_get_ticks() to do accounting,
this patch also fixes up the increment_time() unit test function
and the test env layer to properly increment the fake TSC.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Iba301ddc0fb3d02042106a8bf6e4a6a9a84dc263

Reviewed-on: https://review.gerrithub.io/423580
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2018-09-02 23:59:27 +00:00
zkhatami88
eb6a2cb8f5 nvme:replacing spdk_dma_malloc with spdk_z/malloc
Change-Id: I755d36495e7d8e55c26c4103a9520102223798ef
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/409960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-08-31 17:42:29 +00:00
Jim Harris
b5d1b4f445 test: don't print backtrace when errexit is off
Some test scripts may temporarily disable errexit -
for example, trying a command several times until
it succeeds.  In these cases we don't want to
print a backtrace every time that command fails.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I481f50b046eb13e23f7048672281ec7d62a573a2

Reviewed-on: https://review.gerrithub.io/423921
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-30 04:47:39 +00:00
Seth Howell
aca4ba4dd1 test/vm_setup: fix yum-deprectad for fc28 with vpp
Change-Id: I6608725c031daff6f6d59ced0c327f19b6eea979
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/423755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-29 18:46:02 +00:00
Pawel Niedzwiecki
341fff6397 test/iscsi: Add installation of socat in vm_setup.sh configuration
script

socat is used in iscsi_tgt's libsock test

Change-Id: I027930a99e58ff9862a540d5eed3b1a50de28a93
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/422255
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-15 18:09:46 +00:00
Chen Wang
1f0bff73df test: refactor the run_test function to add detailed information
1.Refactor the run_test function which used to add detailed information
  during run test suites and test cases.
2.Refactor the lvol feature test scripts to make sure their log is the same.
3.Users can use "run_test suite command" to run test suites
  and use "run_test case command" to run test cases.
4.Update the vhost and lvol test as example.

Change-Id: I7b6387019a861bd1c4f89b9a7712e53150aea8fa
Signed-off-by: Chen Wang <chenx.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/403610
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-08-15 17:24:01 +00:00
WangHaiLiang
8680bea106 test/common: remove "rbd pool init"
Fix the issue #384.
Avoid false information:unknown option 'pool init rbd'

Change-Id: I6aee52d1b3a1b4ca250963922d01d7f66151832f
Signed-off-by: WangHaiLiang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/421482
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-08-10 03:10:23 +00:00
Ben Walker
f56b230063 test: Simplify the mock library
Unify the regular mocks and the pointer mocks. Simplify
several of the #defines.

Change-Id: Ica8c69dbb70a685a55b5961b73fd7872f451c305
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/418884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-08-09 17:18:23 +00:00
Karol Latecki
a1acca9274 test/common: fix typo in vm_setup.sh
sgdisk is a part of package gdisk, not a package itself

Change-Id: I214f9720aaeda88006c40e915960c2e1d5cd87b3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421496
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-08-08 15:54:02 +00:00
Karol Latecki
372b0d4931 test/common: add elfutils to vm_setup
Some functions in autotest rely on eu-readelf binary but only
elfutils-devel package was installed in vm_setup.sh script.

Change-Id: Id51ed672a0ef309121e1dddc2a1c8a360f3940c5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-06 16:29:19 +00:00
Karol Latecki
b606baabbc test/common: fix for eu-readelf path over 80 chars
Readelf psargs field holds file path up to 80 chars.
Above 80 chars path is truncated and --wide option does not
enable displaying longer path.

Added checking if the path was not truncated by testing if
file path from psargs exists. If it does not - check rest of
eu-readelf output for a matching path and use it instead.

Change-Id: Ia89b41115803e2463e2030e739121ce1fa8039e0
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421100
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-08-06 16:29:19 +00:00
Pawel Niedzwiecki
a278b14296 test/qemu: Update qemu version to spdk-2.12
Change-Id: I2784166f6b92440ab4979d4ccfd588f91252bbaa
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/420878
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-31 19:59:10 +00:00
Jim Harris
ecb44872b6 test: add some basic raid tests
RAID is experimental for 18.07, pending iov support
which is needed for vhost.  But we need to at least
build RAID in our tests, and run some basic
bdev tests to ensure it doesn't quickly become stale.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I68bb45df6d00e31dc96ed8bc902b61a5ef699ba2

Reviewed-on: https://review.gerrithub.io/420678
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-30 14:59:45 +00:00
Seth Howell
8749a9a591 vm_setup.py: add sgdisk
Used in the ceph tests.

Change-Id: Iea7bda61e792542aae2dff09b0ac321bd4073cc1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/420499
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-26 22:40:22 +00:00
Paul Luse
911c042df8 test: add mock and new function to test_env.c
Need to mock spdk_mempool_get() to test code path when it fails.
Also added spdk_mempool_get_bulk() to test env.

Change-Id: I7f64230c9841215a404149e9a48ad4bf8a63822c
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/420110
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-26 22:08:42 +00:00
Pawel Wodkowski
848daf274b log: add backtrace option
Add '--enable-log-bt=lvl' option to show simple backtrace in following
form:

thread.c: 346:spdk_io_device_register: *ERROR*: io_device 0xf2ef80
already registered
*ERROR*: === BACKTRACE START ===
*ERROR*:   1: spdk_io_device_register() at 0x6d64df
*ERROR*:   2:  spdk_copy_engine_initialize() at 0x71059c
*ERROR*:   3:   spdk_copy_engine_subsystem_initialize() at 0x572ed0
*ERROR*:   4:    spdk_subsystem_init_next() at 0x6ca756
*ERROR*:   5:     spdk_subsystem_verify() at 0x6caba7
*ERROR*:   6:      _spdk_event_queue_run_batch() at 0x6c1ffa
*ERROR*:   7:       _spdk_reactor_run() at 0x6c5349
*ERROR*:   8:        spdk_reactors_start() at 0x6c784f
*ERROR*:   9:         spdk_app_start() at 0x6bf18e
*ERROR*: === BACKTRACE END ===

This adds additional libunwind dependency so don't enable by default.

Change-Id: Ice93d7571a000d8a57d2fedda7670c9a0b6ff7b7
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-on: https://review.gerrithub.io/419726
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-25 15:48:25 +00:00
Shuhei Matsumoto
d891b398a3 test/nbdjson: Fix solid/intermittent CI failure
In part_dev_by_gpt(), stop_nbd_disk is missing and it looks that
this has caused solid/intermittent CI failure in the nbdjson test.

Change-Id: I46b14fa3b7c91547b88f49b28d0cba252b00e8a2
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/420123
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-23 19:31:41 +00:00
Daniel Verkamp
fbb481c2c6 test: add SPDK_TEST_NVME_CLI autotest flag
As requested in GitHub issue #361, we should have a way to control
whether the SPDK-patched nvme-cli tests are executed.

Also add a report_test_completion call for the NVMe-oF nvme-cli test
so we can easily track whether it executed (the local PCIe nvme-cli
test already had a completion).

Change-Id: I70c26bcf2489ccfa563214a9723603aa705be7ab
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/418866
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: John Meneghini <johnm@netapp.com>
Reviewed-by: <ed.rodriguez@netapp.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 23:18:27 +00:00
Ed Rodriguez
2e0df7813e test: vm_setup.sh localization
Change-Id: Ief3a1fbde5d8eabefa87c39feefe3d496e3a5369
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/418236
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
John Meneghini
806f43b26d test: improvements to vm_setup.sh and pkgdep.sh
Change-Id: Ie433edd7c72f84b62c17bacebd31d61ffa6f30ab
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Signed-off-by: John Meneghini <johnm@netapp.com>
Reviewed-on: https://review.gerrithub.io/416052
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-10 22:47:16 +00:00
Paul Luse
58d8a4564b test: add ability to mock spdk_dma_malloc()
Used in crypto unit tests.

Change-Id: I7d7e31e9030c238cb539b9fcf0936e588c1a4211
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/418320
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-10 21:31:39 +00:00
Seth Howell
5b072d3739 test/config: add some comments to vm_setup
Just clarifying why and where some packages are used.

Change-Id: I36f17856a76ed1eb3d88054da2692beee9e569f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/418230
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-07-06 22:15:24 +00:00
Seth Howell
b3d61e5d01 test/config: add command line args to vm_setup.sh
Change-Id: Iae7a2206d3d64834da03226ebeba5e91c635703d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/416885
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-07-02 23:09:23 +00:00
John Meneghini
f04277f0bf test: remove more sys_sgsw dependencies from test scripts
- To override the default /home/sys_sgsw dependency
   add DEPENDENCY_DIR to ~/autorun-spdk.conf

  E.g.: DEPENDENCY_DIR=/home/vagrant

 - To override the default HUGEPAGES use
   add HUGEMEM to ~/autorun-spdk.conf

  E.g.: HUGEMEM=1024

Change-Id: Ib8db9d7d053ae319fe4c725159742875468d47f0
Signed-off-by: John Meneghini <johnm@netapp.com>
Signed-off-by: Ed Rodriguez <ed.rodriguez@netapp.com>
Reviewed-on: https://review.gerrithub.io/415907
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-07-02 21:32:39 +00:00
Dariusz Stojaczyk
8681eecd9a env: introduce spdk_memzone_reserve_aligned
This is an spdk_memzone_reserve variant with additional
alignment parameter. Now that memzones must be used for
physically contiguous memory, it will become extremely useful.

Change-Id: Ie48d682217e0e2f5c859a1603bb8a81fd2a7d7df
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/416978
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-07-02 19:13:23 +00:00
Ben Walker
2fac05e919 test: Fix inaccurate mock of poller behavior
This exposes that one of the tests was not expecting
the correct behavior, so also fix that.

Change-Id: Idb73b3ea74950b2e6f959a40e5740375cb76b8c7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417364
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Ben Walker
f443bd40e4 test: Add spdk_mempool_put_bulk to test_env.c
Change-Id: Id5d22fb9c9e70c432b120676cbbf040b0c8e6c49
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417363
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Ben Walker
ef8dfe1111 test: Mock spdk_ring operations in test_env.c
Change-Id: I1ef5cc1c58d536ccd6774c31cb633af494103e24
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/417361
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-07-02 19:12:45 +00:00
Pawel Kaminski
93fca0e79f test/json: Base bdevs test for bdev subsystem.
- load spdk_tgt with bdevs
   - check if json config is properly saved and loaded
   - check if all bdevs are properly loaded
   - check if configuration of spdk_tgt is properly set

Change-Id: I89226bc9c05880e73523fb189116e7433a513244
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/406326
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-06-29 16:27:57 +00:00
Seth Howell
a33a7b44a3 test/config: minor error fixes to vm_setup.sh
The name of the rpm source for open-iscsi changes between fedora
releases. Also, there was a typo on the tsocks check.

Change-Id: I0f9810c51c35c3efd5616fad303d08b02d9e2218
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/416882
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-28 17:38:48 +00:00
Jim Harris
9d04d0efd5 env_dpdk: pick base-virtaddr that ASAN won't override
ASAN has some memory address regions that it will not
allow applications to use for mappings (including mmap).
So when specifying --base-virtaddr to DPDK for
shared/mutli-process mappings, pick an address that
ASAN will allow the application to map.

Ref: https://github.com/google/sanitizers/wiki/AddressSanitizerAlgorithm

We will still disable ASLR while using the test stub for now.
Maybe we can eliminate this too in the future, but for now just
modify the autotest_common.sh comment since we've figured out
how to keep ASAN from messing with the mmap hint.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ie24fd35bd22aa3ceab6271e8936775b157e5c330
Reviewed-on: https://review.gerrithub.io/416420
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
2018-06-21 22:49:40 +00:00
Jim Harris
696d9eaed0 autotest: reduce HUGEMEM allocation to 2GB on FreeBSD
The FreeBSD contigmem driver has problems allocating
contiguous memory after a large number of unloads/reloads.
Since we run a very limited subset of SPDK tests on
FreeBSD, reduce the HUGEMEM allocation from 8GB to 2GB
to help alleviate the problem and reduce the frequency
of test failures due to contigmem panics.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I777153a6ad4f89311bb4e2f50bf69ee09e628eea

Reviewed-on: https://review.gerrithub.io/416214
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-20 22:48:30 +00:00
Seth Howell
714776c028 test/config: Call pkgdep.sh from vm_setup
vm_setup.sh now calls pkgdep.sh to ensure that the basic packages needed
to compile SPDK are installed. Pkgdep remains a separate script because
it is supposed to contain the minimal set of packages for building SPDK.
while vm_setup is used to configure a complete testing environment.

Change-Id: I4038f522c66ad5b2c55ed73b1170f796b2c94e27
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/415895
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-20 00:18:40 +00:00
Daniel Verkamp
42b6dd9399 pkgdep: add nasm in preparation for intel-ipsec-mb
A nasm or yasm compatible assembler will be required to build the
intel-ipsec-mb library needed for the DPDK crypto framework.

Change-Id: I3e1822069452208aa796caf84ddd9995fcac2cd6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/415065
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-06-13 23:32:31 +00:00
Jim Harris
e30535fecf thread: add spdk_thread_get_count()
This will return the number of currently allocated threads.

Modify the bdev_io caching code to use this new API since these
caches are really per-thread, not per-core.  SPDK does not support
dynamic threading yet, but once it does, we will want callers to be
using functions from the thread API - not counting the number of cores
allocated to the application.

spdk_env_get_core_count may still be useful as a helper function, so
it is still kept and not deprecated.  For example, app.c uses it to
print the number of cores allocated to the application.  bdevperf should
eventually be modified to use spdk_thread_get_count, but holding off on
that for now until spdk_event_allocate() uses threads instead of a reactor
lcore to specify where the event should be executed.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5a30e3e825e6821da87d3927a2443768dfd740f4

Reviewed-on: https://review.gerrithub.io/414709
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-13 00:38:41 +00:00
Ben Walker
a83f91c29a thread: Replace #include of io_channel.h with thread.h
Change-Id: I6babd4cf990bf19b510db88bdfb0ca81e29d9252
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/414700
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Madhu Pai <mpai@netapp.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-06-12 15:24:07 +00:00
Jim Harris
f007d46c5a test: disable RBD and PMDK tests if not installed
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id3152f911a9a1fb4b3ac9d8be56570a665c079f1
Reviewed-on: https://review.gerrithub.io/414281
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-06-11 23:25:55 +00:00
Daniel Verkamp
700477897a test/vm_setup.sh: update nvml-tools -> pmempool
This package has been renamed to reflect the new upstream project name.

Change-Id: I71f6bfaf36fb1fe7dc4cb050e3d7faad1097994e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/411781
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-22 21:04:10 +00:00
Seth Howell
27d7bac9a0 test/common: Make common pmap call more generic
This is to support older versions of pmap (specifically procps v 3.2.8
which ships with centOS 6) which don't support the -p and -X options.

Change-Id: Ic8d4b94985c165e8bf224204af48b5d856aa2403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/411755
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-05-21 17:09:23 +00:00
Seth Howell
86ee572b79 autobuild: On FreeBSD, install the contigmem driver
The contigmem driver is built with dpdk and frequently updated. We
should copy this driver each time we build dpdk in order to maintain
support for FreeBSD

kldload checks the sysctl variable kern.module_path when deciding which
modules to load. Since the values stored in this variable are not
determined by our program, I chose to copy the module to both common
directories for kernel modules in FreeBSD.

Change-Id: I2d439a9cbac5bebb79e6ee48bec47409bb215be9
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/409250
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-05-03 19:49:53 +00:00
Daniel Verkamp
d08917d89e ioat: remove ioat kperf kernel driver test harness
This driver was added to allow benchmarking of the SPDK user-mode I/OAT
driver vs. the Linux kernel I/OAT driver; however, this isn't a
particularly interesting test, since the kernel I/OAT driver is totally
inaccessible from user-mode code (it is only exposed to the in-kernel
dmaengine framework).

Maintaining an out-of-tree kernel driver for the sole purpose of
benchmarking is out of scope for the SPDK project, so remove the kperf
driver and test harness.

This can always be retrieved from git history if needed later.

Change-Id: I0ced6e8a88de2cf09a6c0970dfef0ae8f357f193
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/408900
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-30 17:11:53 +00:00
Tomasz Zawadzki
5bafc240e5 test/iscsi: utilize network namespaces in iSCSI tests
Network namespaces are used to assure that kernel
is not routing packets within host stack,
but they go through veth interfaces.

This patch serves as a base for future VPP test changes,
where namespaces are used as well.

Change-Id: Ic7b82b0a0837bca2e16774fde244348a691fe056
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/405641
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-26 17:10:36 +00:00
Seth Howell
9acb995ede test/common: Don't override autorun-spdk.conf
Change-Id: I1b75ffaccdf909ce1ce11f4464efdd539a7754a7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/408410
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-24 21:53:51 +00:00
Seth Howell
fc0dc65adc test/common: Fix flamegraph typo in vm_setup
Change-Id: I27b9e118fe760bc5e7425fad3731f54b618fb347
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/408409
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-20 17:49:36 -04:00
Tomasz Zawadzki
0629b01d21 test/rbd: add configurable monitoring IP address for ceph
Ceph monitor IP address was always 127.0.0.1, with this change
it can be configured at setup time.

Since each test might prefer to specify different addresses
or ones that do not exist at start of autotest.sh. rbd_setup was
moved to begining of each test respectively.

Change-Id: I5626f7ea979d0db921208355ba23314cf48e971f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/407910
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-17 13:55:44 -04:00
Jim Harris
18f3a22b20 test: disable ASLR for multi-process tests
Multi-process tests should work fine using the DPDK
-base-virtaddr parameter.  But we run tests with ASAN
enabled and ASAN instrumentation results in mmap
address hints getting ignored.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I089d31726c7b8f5ed0ccdc2deb19acc5431260f1

Reviewed-on: https://review.gerrithub.io/407843
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-17 13:26:24 -04:00
Jim Harris
dba0f6e0a8 test: run pmap on stub process after it starts
This will be a debugging aid to root cause some intermittent
test failures seen due to ASLR in the test pool.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I23a856ab53bdf378b5da51f1cb8d9f2cdf6efce1

Reviewed-on: https://review.gerrithub.io/407842
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-04-17 13:26:24 -04:00
Seth Howell
e3263286d3 test/nvmf: confirm kernel finds NVMe-oF namespaces
In Fedora 27 the nvme program returns before the kernel has loaded the
namespces associated with the subsystem. This patch ensures that those
namespaces can be enumerated by the kernel before we run any tests on
them.

Change-Id: I77505ed5cd5b1118a841650851fa5ecaf76f5619
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/407829
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-16 19:25:37 -04:00
Seth Howell
2393edd538 vm_setup: add new test flags to autorun-spdk.conf
Change-Id: I66cb037cf392230129a8fbcc5e455c1cc80c8903
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/406996
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-09 17:58:54 -04:00
Seth Howell
08939a7714 vm_setup.sh: add pre-existing directory checks.
This will allow us to run vm setup multiple times on a vm without it
failing due to pre-existing directories.

Change-Id: Ied17dd40daaa4b5f9a1e2448940d413c7db7cc96
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/405508
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-09 17:58:54 -04:00
Tomasz Zawadzki
ac277af97d test/config: add install VPP in vm_setup.sh
This patch adds compilation and installation of VPP.

Note that it removes one of VPP config files, that is
responsible for setting up hugepages. It is already done
with setup.sh script.
Parameters kernel.shmmax and vm.max_map_count were set
to low count and causing issues with hugepage total sizes
above 1GB.

Change-Id: Ic6c31f4192c654672e36c4131e34eb5b8aaac022
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/404144
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-04-09 17:45:31 -04:00
Jim Harris
218dcd840a bdev/pmem: change all NVML strings to PMDK
NVML (Non Volatile Memory Library) changed its name to
PMDK (Persistent Memory Development Kit), so make the
necessary changes to the SPDK repository.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id07a87eafb4e9a3099603030c58c0e927bafa608

Reviewed-on: https://review.gerrithub.io/406256
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Jim Harris
6e1611da18 test: use ss -lx instead of netstat by default
netstat is deprecated on CentOS/RHEL in favor of
iproute utilities.  At least on one of the systems
in our test pool, netstat isn't installed - we could
install it but let's instead try to use ss if it's
available and still fall back to netstat when it's
not.

While here, remove waitforlisten_tcp instead of
changing it to use ss.  We are not using waitforlisten_tcp
anywhere and do not recommend enabling RPC over TCP except
for test purposes.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I4ee3597536797f501750915a4a219b55004bd454

Reviewed-on: https://review.gerrithub.io/406464
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-07 00:07:57 -04:00
Dariusz Stojaczyk
8e6b1fbd33 test/virtio: use "single file segments"
This enables virtio tests to run with 2MB hugepages

Change-Id: I0f9e2de0cb1f68fb967b929d8a27c6e410cb675a
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/404318
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-04-03 18:57:29 -04:00
Daniel Verkamp
20e69cf6b1 test/vm_setup.sh: add nvml-tools for pmempool
Change-Id: I964c45f652c6a24a05300d63f9c417e0f26e97e9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/405919
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2018-04-03 14:14:29 -04:00
Seth Howell
397521bdb9 autotest_common.sh: handle ceph luminous in rbd_setup
unlike previous implementations of ceph, luminous (which ships with
fedora 27) does not create a default pool for us. In case we are on a
system using luminous, we need to create that pool ourselves.

Change-Id: I183dc4fff6b92f7b270bd5a5d05947f0d99e9d04
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/405703
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
2018-04-02 16:58:18 -04:00
Seth Howell
899eb5f19a test: move ut_multithread.c to common/lib
Change-Id: Ied7da08503b52470dbb8bb77eee1a711f41e7f28
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404977
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-29 00:31:24 -04:00
Seth Howell
c9fb57a298 test: move test_env.c under test/common/lib
This file is accessed by files in multiple directories under test.

Change-Id: I634481fb58eab5c097aaece5289f88e531954fcb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404976
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-29 00:31:24 -04:00
Seth Howell
b337d878ca test: move config directory under common directory
Change-Id: I9db13ab60348000f1bfeb39888fbaea6535d26ae
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/404953
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 19:45:09 -04:00
Liang Yan
2e0610a075 autotest: Add vtune test(porting from dts to autotest)
Add compiling SPDK with vtune test case.

Change-Id: I7ece8ef87e703eee5b1a8c326fd88224c9a02895
Signed-off-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-on: https://review.gerrithub.io/392424
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-23 03:48:37 -04:00
Ziye Yang
15ce326251 test/iscsi: Add iSCSI initiator test
This patch leverages bdevperf, which configures
an iscsi initiator bdev to test iSCSI target.

Change-Id: I94c9299ef1f286a13678495d270326f4616c6241
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/404068
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-21 20:35:42 -04:00
Jim Harris
2af15344ab bdev: add iSCSI initiator bdev module
This uses libiscsi to implement an iSCSI initiator bdev
module for SPDK.  Still a lots of work to do on this - posting
it in case anyone is interested in working on this further.

A number of todo items are listed in a README in the lib/bdev/iscsi
directory.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Change-Id: I060e33de0cd6796246789bf0e1bb4f2df59d8f71
Reviewed-on: https://review.gerrithub.io/390313
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-21 20:35:42 -04:00
Pawel Niedzwiecki
b96cf2ef2a test/common: Separate failing nightly tests
Introduce RUN_NIGHTLY_FAILING flag for nightly tests
that are always or periodically failing.
A new cron job will be added to run test with this flag.

RUN_NIGHTLY_FAILING is set for:
test/vhost/readonly/readonly.sh
test/nvmf/multiconnection/multiconnection.sh
a fio job in test/lib/bdev/blockdev.sh

Change-Id: I32640c0d4b916156c2ee996a9847c7c9e64941f8
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/403985
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2018-03-21 12:52:10 -04:00
Pawel Niedzwiecki
ad22830896 test/common: exit with error in waitforlisten() if app crashed
In some cases, if vhost crashes on startup, test would continue
normally instead of failing

Change-Id: If2f82b5e3dad07403051353d6fcf5e83d5d002c0
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/403064
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
2018-03-07 11:02:35 -05:00
Seth Howell
d1f9da828a autotest.sh: create cleanup function
This function consolidates cleanup behaviors that the pool should follow
in the event of a failure.

Change-Id: I8623f065146292b29a38eb199dcbeac5787723e7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401857
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-02 13:28:24 -05:00
Daniel Verkamp
d51ea8deb4 scripts/ceph: remove subshell and cd
The Ceph scripts can now be run from any directory; they don't depend on
their $PWD being the script location anymore.

Change-Id: Iff883a3e9b8ba2d0a6193e2f5c047eb9d7d423fd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/401657
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-03-02 12:56:22 -05:00
Seth Howell
a562812dcf autotest: add test completion tracking
Change-Id: I1ca0578a010db2ff8535505bfd981cd1c368e403
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/392240
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-03-01 17:13:41 -05:00
Seth Howell
c9e5d2a6d0 autotest_common.sh: move to test/common
This is the first step in reorganizing the spdk test heirarchy.

Change-Id: I83467653af2da2a53251649092bed6902d6cb86e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/401707
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-02-27 20:37:27 -05:00