Commit Graph

4173 Commits

Author SHA1 Message Date
Tomasz Zawadzki
a1591e23c4 ut/blob: verify success of spdk_bs_create_snapshot() on power failure
This change verifies that not only it is possible to open
the snapshot, but also the spdk_bs_create_snapshot() reported
success.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ifb7fcc91802c838710f1f9be41090057268a5900
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1262
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-23 09:37:27 +00:00
Tomasz Zawadzki
2eada474ba ut/blob: recreate bs on each create_snapshot power fail UT
blob_create_snapshot_power_failure() test always attempted
snapshot creation until the snapshot did open.
Which meant that on bs reload after dirty shutdown,
the original blob might be in unrecoverable state.

UT did not verify create call beyond that point.
Next patch will stop after snapshot is possible to be opened,
AND the spdk_bs_create_snapshot() returned success.

This patch makes it easier to recreate same starting
conditions after snapshot did not open.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I1901b29a8319b2203a855e7879821ed3850f4917
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1261
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-23 09:37:27 +00:00
Tomasz Zawadzki
0f5157377f lib/blob: merge EP of a clone when deleting a snapshot
In general it is not possible to delete snapshot when
there are clones on top of it.
There is special case when there is just a single clone
on top that snapshot.

In such case the clone is 'merged' with snapshot.
Unallocated clusters in clone, are filled with the ones
in snapshot (if allocated there).

Similar behavior should have occurred for extent pages.

This patch adds the implementation for moving EP from
snapshot to clone along with UT.

The UT exposes the issue by allowing delete_blob
to proceed beyond just unrecoverable snapshot blob.

Fixes #1291

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib2824c5737021f8e8d9b533a4cd245c12e6fe9fa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1163
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-23 09:37:27 +00:00
Changpeng Liu
5f66e5b160 test/fuse: make the --with-fuse option be consistent with configure
Change-Id: I47dce4dfe01a7badb493978257577c1a8fc0be93
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1404
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-23 09:36:46 +00:00
Michal Berger
004b52fe6c test/bdev: Move part_dev_by_gpt() into blockdev.sh
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I9a888f80148d5fd2eec5feed299b0c81bb78b984
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1282
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-20 13:59:17 +00:00
Michal Berger
1db267cf0d test/bdev: Convert setup_nvme_conf() to RPC calls
Change-Id: I49415a159a5849232637f281fd9360636691d339
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1018
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-20 13:59:17 +00:00
Michal Berger
ae2d8097c3 test/bdev: convert setup_bdev_conf() to RPC calls
Change-Id: I479bb9df75468b13f739c49c60b38ab0861bf0af
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1017
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-20 13:59:17 +00:00
Michal Berger
54e713036e test/bdev: Cover default setup_bdev_conf() run in its own case
Change-Id: Ibd24c65ac8921083c5999721b720dadfa3c8de5b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-20 13:59:17 +00:00
Michal Berger
a940721a29 test/bdev: Use separate function for cleaning lingering files
Change-Id: Ib44dbf1a4ec8e44f26cb8219e53b3ed1ae03d48b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1095
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-20 13:59:17 +00:00
Ziye Yang
465f4c1254 sock/uring: Add the async network I/O support for socket
This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).

PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.

Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-20 08:41:28 +00:00
Shuhei Matsumoto
ded6d2c7f2 lib/util: Add spdk_sn32_lt/gt() to compare two numbers based on serial number arithmetic
Previously the macro SN32_LT and SN32_GT had been added to iSCSI
library. But such comparisons may be used in other libraries.
So add two inline helper functions spdk_sn32_lt() and spdk_sn32_gt()
to include/spdk/util.h.

Add unit test for these functions. These functions are located in
header file but math.c is the place if they are located in source file.
Hence add unit test as the one for math.c.

The next patch replaces the macro SN32_LT and SN32_GT by
spdk_sn32_lt() and spdk_sn32_gt() in iSCSI library, respectively.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3e4d80fea98ad4ae1516e27b9c9e8ec6f37e7a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1346
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-19 08:51:31 +00:00
Darek Stojaczyk
7098c65339 test/common: inline fio_config_add_job
The function is almost 20 lines long and it essentialy
appends two lines two a specified file. We don't need it.

Change-Id: I565de8e54aa71b04f15745456bec7b2b3b8fc178
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-19 08:50:59 +00:00
Darek Stojaczyk
e03861f138 memory.h: move to public headers
There's no reason not to publish those. Especially if
they're needed in other public headers.

Change-Id: I7dfc6922fcc0dfc46822ad8a16a375f997b98e84
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-19 08:50:45 +00:00
Pawel Kaminski
970601af86 test/bdev: Replace old style config with json config
Overwrite ini config with json and use it in such a form throughout
the entire test run.

This patch is the beginning of the series targeting conversion of
ini config to json in blockbdev tests.

Change-Id: Ic64f24b2ea9df85ad1ba3d038a78384e6f2badf9
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-19 08:39:01 +00:00
Karol Latecki
db43b387ba test: do not use iter_pci_class_code in tests
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.

This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.

Fixes #1235

Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-18 08:03:21 +00:00
Darek Stojaczyk
bad71510cb test/iscsi/fuzz: remove unnecessary delete_tmp_files()
This function removes leftover fio files, but fio is
not run in this test. Remove it.

Change-Id: I1522a018c9cc3093bd1bcb1c2361dd7b31deacbe
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-18 08:03:13 +00:00
Evgeniy Kochetov
19453e5b6c ut/nvme: Test Set Num Queues controller intialization step
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: I590420fb7d976240a5a924ea7c0204c29d072990
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1272
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-18 08:02:48 +00:00
Evgeniy Kochetov
2ca711694e nvme/ctrlr: Remove Get Num Queues initialization step
NVMe specification in ch.7.6 "Controller Initialization" suggests to
use only Set Features "Number of queues" command and says nothing
about Get Features. All required information is available after Set
Num Queues step.

Fixes #1270

Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ide38ba9c7f063f1d6b13bfce4232c588cc906784
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-18 08:02:48 +00:00
Seth Howell
4fe25e7233 vm_setup: add abigail for fedora machines.
The fedora machines running unit tests will require this package to run
the ABI compliance checking tests.

Change-Id: I9f9307c04e235cb9b6f78c4908609a529a86ccba
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1137
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-18 08:02:30 +00:00
Seth Howell
29044c8d52 test/make: don't print missing symbols in libs.
We don't fail the test based on missing symbols so printing out the
information about them during tests is not super useful. Restricting the
output to what caused the test to fail will allow for simpler debugging.

Change-Id: I7b251dfe39c940074d7c8d887941adb635fd3965
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-18 08:02:30 +00:00
Seth Howell
19a97c11a2 test: confirm ABI compatibility between releases.
This test checks to confirm that the major version for the SO
has been incremented if symbols have been removed or changed since
the last release or the minor version if symbols have been added.

Change-Id: I2bbef858c4cb18afbb854c90782495c45605c63d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-18 08:02:30 +00:00
Pawel Kaminski
b5de4adfd7 test/lvol: rewrite delete_snapshot_with_snapshot to bash
Change-Id: I568066322369d192953885617ed11304a36ad4d2
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/698
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-03-17 16:37:02 +00:00
Pawel Kaminski
dbf5793576 test/lvol: rewrite delete_snapshot to bash
Change-Id: I9480cc329b76d4c4d89aa1ecc4bb2042487c7aab
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/697
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-03-17 16:37:02 +00:00
Pawel Kaminski
97d7aa90b9 test/lvol: rewrite set_read_only to bash
Change-Id: I6540067b2ca8ffc0255e1222e3203922432d0b34
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/696
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
2020-03-17 16:37:02 +00:00
Pawel Kaminski
45088e0c60 test/lvol: rewrite clone_decouple_parent and clone_decouple_parent_rw to bash
Change-Id: I6c21e2b3ab9cf91d8e5ab725c630173a558a5c04
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-17 16:37:02 +00:00
Pawel Kaminski
4c8066a32b test/lvol: rewrite clone_inflate to bash
Change-Id: If479b610c6349e0394122bd6a87d38037d199bca
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/688
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-17 16:37:02 +00:00
Michal Berger
053c15b32b fio: Remove hardcoded path to fio repo
The hardcoded path is replaced with $CONFIG_FIO_SOURCE_DIR as defined
during the compile time.

Additionally, all checks which determine if fio is available are now
based on $CONFIG_FIO_PLUGIN=y knob instead of the presence of the fio
repo.

Change-Id: Ie469747f3863c9561f53d32e8c3f29778afaaf74
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-17 14:44:20 +00:00
Michal Berger
6aa9db94b3 test/common: Include build's environment in the test suite
Change-Id: I0f45c80991c22ca72fb29fdf9af6bc50fcd7ae99
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 14:44:20 +00:00
Shuhei Matsumoto
2fa51eeb46 lib/nvmf: Make spdk_nvmf_poll_group_destroy() asynchronous
The next patch will create poll group threads dynamically for
NVMe-oF target, and will need to wait for completion of poll group and
I/O channel destroy. This is a preparation for the next patch.

Add callback function and its argument to spdk_nvmf_poll_group_destroy(),
and to struct spdk_nvmf_poll_group, respectively.

The callback has not only cb_arg but also status as its parameters even
if the next patch always sets the status to zero. The reason is to follow
spdk_nvmf_tgt_destroy's callback and to process any case that the status
is nonzero in future.

spdk_nvmf_poll_group_destroy() sets the passed callback to the passed
poll group.

Then spdk_nvmf_tgt_destroy_poll_group() calls the held callback in the
end.

This change will ensure all pollers are being unregistered and
all I/O channels are being released.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb854066a5259a6029d55b88de358e3346c63f18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:49:00 +00:00
Tomasz Zawadzki
863d73d34b test: limit files scanned when searching for test completions
When gathering test completions original grep went over all
files in repository recursively. In some cases it took
way longer than required.

This patch limits searches only to rootdir and test directories.

References #1068

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ice0bba25f2fad62516226a7b045d12b6614bead5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:26:06 +00:00
paul luse
fae0356697 test/bdevperf: fix issue with overlapping I/Os
A previous fix for miscompare had a slight issue with the scenario
where we chose an offset and then discover that it's already taken
because we've wrapped to this location before the previous IO
completed. In such case, we intended to bump the offset to just go
to the next block however we only bumped the local variable so the
next IO submitted will overwrite the previous.

The fix is simply to use the previously incremented target->offset
and increment it again for the next IO.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I26661eb0631ac50cc4f86f42f1d1789d297e204d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1250
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-17 08:23:33 +00:00
Tomasz Zawadzki
b5ef8c45b5 ut/blob: recreate bs and blobs for each iteration of power fail UT
blob_delete_snapshot_power_failure() test always attempted
snapshot deletion until the snapshot did not open.
Which meant that on bs reload after dirty shutdown,
the snapshot was in unrecoverable state.

UT did not verify delete call beyond that point.
Next patch will stop after snapshot is unrecoverable,
AND the spdk_bs_delete_blob() returned success.

This patch makes it easier to recreate same starting
conditions after snapshot did not load.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e14e8dadaaef4aa3287ebd376465253466b2362
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-03-17 08:23:25 +00:00
Ben Walker
ea65bf612d Revert "nvme/tcp: Change hdr in nvme_tcp_pdu to pointer"
This reverts commit ea5ad0b286.

This code is moving from the nvmf target to the posix sock
layer in this series.

Change-Id: I333bdf325848e726ab82a9e6916e1bbdcd34009c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/446
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-17 08:23:07 +00:00
yidong0635
01e85f471c test/nvmf: Add more cases in perf testing.
Add qd=1 test in per-patch test.
Add qd=32 in nightly test.

That can find some issues in corner case.

Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic7e33812facaa031af52ba0217d310bf0040473c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1240
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-16 08:45:36 +00:00
Tomasz Kulasek
6f98b0946a test/iscsi_tgt: fix wait for iscsi devices in rdb test
We should wait until login fully performs before fio tests
to avoid intermittent failures.


Change-Id: I55862f5e9c7b9fbdf225036caed92c0ac935f1b1
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1251
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-16 08:45:28 +00:00
Tomasz Kulasek
7b7e97604b test/vpp: fix error handling in vppctl non-interactive mode
On Fedora 30 we have noticed VPP 19.04 related issues:

  1) Error values returned by vppctl in non-interactive mode
     are not relevant to the success/fail of command.
     Vppctl ALWAYS returns 0, so "-e" bash option is unable
     to detect any errors.
  2) We have intermittent pipefail errors (error 141) returned
     by vppctl on disconnect from vpp, even though commands are
     executed succesfully.

Change-Id: Ie22ea24f7e81017089b899111724d338eeb81113
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1214
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2020-03-16 08:45:07 +00:00
Hailiang Wang
bb5567496f test/iscsi: add application for target fuzz testing.
Enables us to test randomized data against the iSCSI target interface.

Change-Id: I56bd5bcd936b92ba152d4d5678d7124b3165c03c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-13 08:53:46 +00:00
Konrad Sztyber
02385a64b9 bdev/nvme: asynchronous module_finish
Now that a namespace can be depulated asynchronously now, the NVMe bdev
module should also be finalized asynchronously, after all namespaces and
controllers are deallocated.

Change-Id: Ic082fec8e31e9bd5ee1c698cd8dfca9f248776d3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-13 08:53:39 +00:00
Konrad Sztyber
9d0f9b330d bdev/nvme: asynchronous namespace depopulation
This patch adds the ability for a namespace to be depopulated
asynchronously.  Currently both regular NVMe namespaces, as well as the
OCSSD ones are depopulated synchronously, but it'll be changed in the
upcoming patches.

The nvme_bdev_ctrlr.ref is now not only tracking the number of bdevs
created on that controller, but also the number of populated namespaces.

Change-Id: I7b112d9b0d41739f3dc7d427e9da340843128c54
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
2020-03-13 08:53:39 +00:00
Shuhei Matsumoto
ea863bb0b3 lib/event: Count number of threads per reactor
Add thread_count to struct spdk_reactor to count number of threads
per reactor. This number will be used in the next patch to know
if all threads are idle or not for each reactor to support CPU
power saving.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f7cc5a6b78d85e9f8d0b539c60058c13e282759
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-12 09:04:02 +00:00
Wojciech Malikowski
e0294b002d test/ftl: Prevent from trying to kill same process twice
ftl fio tests should not kill spdk_tgt process on exit.

Change-Id: I541e84002202a5f01bd5a2dffe31a60e02da30e9
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-12 09:03:47 +00:00
Changpeng Liu
c61b70c3d5 nvme/opal: rename spdk_opal_init_dev() and spdk_opal_close() with construct/destruct suffix
Since the OPAL here is just for NVMe device, so we don't need to use dev_handler as common
handler, just rename it to spdk_nvme_ctrlr.  And we don't exit the initialization if
OPAL construnction had a failure.  Also move the timeout initialization to construct().

Change-Id: I11f0aea961eaa3da0c6253eb03d0227f7e7e5f11
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1101
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-11 11:36:08 +00:00
Karol Latecki
996248fd86 test/iscsi_tgt: do not silently skip rbd.sh tests
Remove if/exit steps from rbd.sh and move the condition
to parent script iscsi_tgt.sh.

Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I86c9c659e0f8b4f544296e2ef8a861b25fd4f424
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-03-11 11:35:33 +00:00
Darek Stojaczyk
817e91b7ce test/lvol: use rpc daemon throughout all lvol tests
$ time ./test/lvol/lvol2.sh

Without rpc daemon (before):
real    0m52.195s
user    1m29.316s
sys     0m7.660s

With rpc daemon (now):
real    0m12.452s
user    0m13.505s
sys     0m3.798s

Note we only have about a half of lvol tests ported to bash atm,
so this time difference would only grow.

Change-Id: I28ec0b92f19e0c7fd48392a72f32535c1106b8be
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1058
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Darek Stojaczyk
8b98cdb64a scripts/rpc.py: add daemon mode
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.

We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.

The background rpc.py is started with a new --server flag
that's described as:

> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.

As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.

```
time {
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]

        malloc=$($rpc bdev_malloc_create 8 512)
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "1" ]

        $rpc bdev_passthru_create -b "$malloc" -p Passthru0
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "2" ]

        $rpc bdev_passthru_delete Passthru0
        $rpc bdev_malloc_delete $malloc
        bdevs=$($rpc bdev_get_bdevs)
        [ "$(jq length <<< "$bdevs")" == "0" ]
}
```

Regular rpc.py:
```
real    0m1.477s
user    0m1.289s
sys     0m0.139s
```

rpc_cmd():
```
real    0m0.085s
user    0m0.025s
sys     0m0.006s
```

autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.

Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.

(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)

Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-03-11 11:35:03 +00:00
Tomasz Zawadzki
aba9841212 ut/blob: add setup/clean for blob creation
Added new setup/cleanup functions for a suite of
unit tests that use a particular blob through out the
tests.

This simplifies the contents of particular UTs,
and brings forward focus of each UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib7d69b90fcbe8e6a0ba303475fcbaeb90fa110a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/945
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
104d4c0a57 ut/blob: add blob deletion at the end of UT
Some of the unit tests were only closing a blob,
without deleting it at the end of unit tests.
This is valid behaviour, but to later simplify
the setup/cleanup - a delete blob was added.

This change was made to unit tests where,
it was not focus of particular UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e338ba6fcb63af26658e8968971a6d56f70807b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1203
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
cc991aad61 ut/blob: simplify blob closing and deletion
Operation of closing and deleting a blob is common
throughout the tests.

This patch creates helper function ut_blob_close_and_delete(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is closed and deleted, when it is not the subject of
particular unit test (focus is on other functionality).

It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic56594da8cd4c442b62890d62961b44771059679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
5df4f50f76 ut/blob: simplify blob creation and opening
Operation of creating a blob (with or w/o opts),
then opening it through out the tests is quite common.

This patch creates helper function ut_blob_create_and_open(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is created and opened, when it is not the subject of
particular unit test (focus is on other functionality).

It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia541b855d154503f824bfa3909bcfeac649c5853
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1201
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-03-11 11:34:55 +00:00
Tomasz Zawadzki
95ebed3c9e ut/blob: always use local blobid
Always call API on blobid that was received from blob
create, instead of relying on global one.

Where necessary added declaration and assignment.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3529b4d5e86c1301bb64268029a57873f9f600cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-03-11 11:34:55 +00:00