This patch is used to add the async network I/O support in
sock layer. If code is configured with I/O uring, --with-uring,
we can use io uring in Linux (version >=5.4-rc3).
PS: We also make VPP's default priority > uring, because
for the iSCSI or sock test linked with VPP. It tests VPP with
a given address (which is not a special VPP can only open address),
so using uring can also listen those address succefully. And if we make
uring with priority > VPP, actually, VPP will not tested in those cases.
Additionally, the current CI pool is not ready for test, we need
wait for the CI system ready. And I test on my local platform, it works.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie8ee3c8ddf8d2a7264f2b382376733e002816dcc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/952
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Previously the macro SN32_LT and SN32_GT had been added to iSCSI
library. But such comparisons may be used in other libraries.
So add two inline helper functions spdk_sn32_lt() and spdk_sn32_gt()
to include/spdk/util.h.
Add unit test for these functions. These functions are located in
header file but math.c is the place if they are located in source file.
Hence add unit test as the one for math.c.
The next patch replaces the macro SN32_LT and SN32_GT by
spdk_sn32_lt() and spdk_sn32_gt() in iSCSI library, respectively.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id3e4d80fea98ad4ae1516e27b9c9e8ec6f37e7a8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1346
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The function is almost 20 lines long and it essentialy
appends two lines two a specified file. We don't need it.
Change-Id: I565de8e54aa71b04f15745456bec7b2b3b8fc178
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1288
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There's no reason not to publish those. Especially if
they're needed in other public headers.
Change-Id: I7dfc6922fcc0dfc46822ad8a16a375f997b98e84
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1041
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Overwrite ini config with json and use it in such a form throughout
the entire test run.
This patch is the beginning of the series targeting conversion of
ini config to json in blockbdev tests.
Change-Id: Ic64f24b2ea9df85ad1ba3d038a78384e6f2badf9
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/530
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Do not use iter_pci_class_code function in tests to
iterate over NVMe drives. This function can return
drives which at the moment of execution can not be
whitelisted for use.
This can result in test errors (such as simply
bdev_nvme_attach_controller RPC command failing) or
even using and deleting data from NVMe drive which
was not meant to be used in tests.
Fixes#1235
Change-Id: I82b9935fc88605b636c2096be6c71d4880a567c8
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1309
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This function removes leftover fio files, but fio is
not run in this test. Remove it.
Change-Id: I1522a018c9cc3093bd1bcb1c2361dd7b31deacbe
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1292
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
NVMe specification in ch.7.6 "Controller Initialization" suggests to
use only Set Features "Number of queues" command and says nothing
about Get Features. All required information is available after Set
Num Queues step.
Fixes#1270
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Change-Id: Ide38ba9c7f063f1d6b13bfce4232c588cc906784
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1271
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The fedora machines running unit tests will require this package to run
the ABI compliance checking tests.
Change-Id: I9f9307c04e235cb9b6f78c4908609a529a86ccba
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1137
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
We don't fail the test based on missing symbols so printing out the
information about them during tests is not super useful. Restricting the
output to what caused the test to fail will allow for simpler debugging.
Change-Id: I7b251dfe39c940074d7c8d887941adb635fd3965
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1094
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This test checks to confirm that the major version for the SO
has been incremented if symbols have been removed or changed since
the last release or the minor version if symbols have been added.
Change-Id: I2bbef858c4cb18afbb854c90782495c45605c63d
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1068
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
The hardcoded path is replaced with $CONFIG_FIO_SOURCE_DIR as defined
during the compile time.
Additionally, all checks which determine if fio is available are now
based on $CONFIG_FIO_PLUGIN=y knob instead of the presence of the fio
repo.
Change-Id: Ie469747f3863c9561f53d32e8c3f29778afaaf74
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1108
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
The next patch will create poll group threads dynamically for
NVMe-oF target, and will need to wait for completion of poll group and
I/O channel destroy. This is a preparation for the next patch.
Add callback function and its argument to spdk_nvmf_poll_group_destroy(),
and to struct spdk_nvmf_poll_group, respectively.
The callback has not only cb_arg but also status as its parameters even
if the next patch always sets the status to zero. The reason is to follow
spdk_nvmf_tgt_destroy's callback and to process any case that the status
is nonzero in future.
spdk_nvmf_poll_group_destroy() sets the passed callback to the passed
poll group.
Then spdk_nvmf_tgt_destroy_poll_group() calls the held callback in the
end.
This change will ensure all pollers are being unregistered and
all I/O channels are being released.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb854066a5259a6029d55b88de358e3346c63f18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/495
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When gathering test completions original grep went over all
files in repository recursively. In some cases it took
way longer than required.
This patch limits searches only to rootdir and test directories.
References #1068
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ice0bba25f2fad62516226a7b045d12b6614bead5
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1089
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
A previous fix for miscompare had a slight issue with the scenario
where we chose an offset and then discover that it's already taken
because we've wrapped to this location before the previous IO
completed. In such case, we intended to bump the offset to just go
to the next block however we only bumped the local variable so the
next IO submitted will overwrite the previous.
The fix is simply to use the previously incremented target->offset
and increment it again for the next IO.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I26661eb0631ac50cc4f86f42f1d1789d297e204d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1250
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
blob_delete_snapshot_power_failure() test always attempted
snapshot deletion until the snapshot did not open.
Which meant that on bs reload after dirty shutdown,
the snapshot was in unrecoverable state.
UT did not verify delete call beyond that point.
Next patch will stop after snapshot is unrecoverable,
AND the spdk_bs_delete_blob() returned success.
This patch makes it easier to recreate same starting
conditions after snapshot did not load.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e14e8dadaaef4aa3287ebd376465253466b2362
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This reverts commit ea5ad0b286.
This code is moving from the nvmf target to the posix sock
layer in this series.
Change-Id: I333bdf325848e726ab82a9e6916e1bbdcd34009c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/446
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add qd=1 test in per-patch test.
Add qd=32 in nightly test.
That can find some issues in corner case.
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Change-Id: Ic7e33812facaa031af52ba0217d310bf0040473c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1240
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We should wait until login fully performs before fio tests
to avoid intermittent failures.
Change-Id: I55862f5e9c7b9fbdf225036caed92c0ac935f1b1
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1251
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
On Fedora 30 we have noticed VPP 19.04 related issues:
1) Error values returned by vppctl in non-interactive mode
are not relevant to the success/fail of command.
Vppctl ALWAYS returns 0, so "-e" bash option is unable
to detect any errors.
2) We have intermittent pipefail errors (error 141) returned
by vppctl on disconnect from vpp, even though commands are
executed succesfully.
Change-Id: Ie22ea24f7e81017089b899111724d338eeb81113
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1214
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Enables us to test randomized data against the iSCSI target interface.
Change-Id: I56bd5bcd936b92ba152d4d5678d7124b3165c03c
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Now that a namespace can be depulated asynchronously now, the NVMe bdev
module should also be finalized asynchronously, after all namespaces and
controllers are deallocated.
Change-Id: Ic082fec8e31e9bd5ee1c698cd8dfca9f248776d3
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1198
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This patch adds the ability for a namespace to be depopulated
asynchronously. Currently both regular NVMe namespaces, as well as the
OCSSD ones are depopulated synchronously, but it'll be changed in the
upcoming patches.
The nvme_bdev_ctrlr.ref is now not only tracking the number of bdevs
created on that controller, but also the number of populated namespaces.
Change-Id: I7b112d9b0d41739f3dc7d427e9da340843128c54
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Add thread_count to struct spdk_reactor to count number of threads
per reactor. This number will be used in the next patch to know
if all threads are idle or not for each reactor to support CPU
power saving.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4f7cc5a6b78d85e9f8d0b539c60058c13e282759
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1169
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
ftl fio tests should not kill spdk_tgt process on exit.
Change-Id: I541e84002202a5f01bd5a2dffe31a60e02da30e9
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Since the OPAL here is just for NVMe device, so we don't need to use dev_handler as common
handler, just rename it to spdk_nvme_ctrlr. And we don't exit the initialization if
OPAL construnction had a failure. Also move the timeout initialization to construct().
Change-Id: I11f0aea961eaa3da0c6253eb03d0227f7e7e5f11
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1101
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Remove if/exit steps from rbd.sh and move the condition
to parent script iscsi_tgt.sh.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I86c9c659e0f8b4f544296e2ef8a861b25fd4f424
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1081
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
$ time ./test/lvol/lvol2.sh
Without rpc daemon (before):
real 0m52.195s
user 1m29.316s
sys 0m7.660s
With rpc daemon (now):
real 0m12.452s
user 0m13.505s
sys 0m3.798s
Note we only have about a half of lvol tests ported to bash atm,
so this time difference would only grow.
Change-Id: I28ec0b92f19e0c7fd48392a72f32535c1106b8be
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1058
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add rpc_cmd() bash command that sends rpc command to an
rpc.py instance permanently running in background.
This makes sending RPC commands even 17 times faster.
We make use of bash coprocesses - a builtin bash feature
that allow starting background processes with stdin and
stdout connected to pipes. rpc.py will block trying to
read stdin, effectively being always "ready" to read
an RPC command.
The background rpc.py is started with a new --server flag
that's described as:
> Start listening on stdin, parse each line as a regular
> rpc.py execution and create a separate connection for each command.
> Each command's output ends with either **STATUS=0 if the
> command succeeded or **STATUS=1 if it failed.
> --server is meant to be used in conjunction with bash
> coproc, where stdin and stdout are named pipes and can be
> used as a faster way to send RPC commands.
As a part of this patch I'm attaching a sample test
that runs the following rpc commands first with the regular
rpc.py, then the new rpc_cmd() function.
```
time {
bdevs=$($rpc bdev_get_bdevs)
[ "$(jq length <<< "$bdevs")" == "0" ]
malloc=$($rpc bdev_malloc_create 8 512)
bdevs=$($rpc bdev_get_bdevs)
[ "$(jq length <<< "$bdevs")" == "1" ]
$rpc bdev_passthru_create -b "$malloc" -p Passthru0
bdevs=$($rpc bdev_get_bdevs)
[ "$(jq length <<< "$bdevs")" == "2" ]
$rpc bdev_passthru_delete Passthru0
$rpc bdev_malloc_delete $malloc
bdevs=$($rpc bdev_get_bdevs)
[ "$(jq length <<< "$bdevs")" == "0" ]
}
```
Regular rpc.py:
```
real 0m1.477s
user 0m1.289s
sys 0m0.139s
```
rpc_cmd():
```
real 0m0.085s
user 0m0.025s
sys 0m0.006s
```
autotest_common.sh will now spawn an rpc.py daemon if
it's not running yet, and it will offer rpc_cmd() function
to quickly send RPC commands. If the command is invalid or
SPDK returns with error, the bash function will return
a non-zero code and may trigger ERR trap just like a regular
rpc.py instance.
Pipes have major advantage over e.g. unix domain sockets - the pipes
will be automatically closed once the owner process exits.
This means we can create a named pipe in autotest_common.sh,
open it, then start rpc.py in background and never worry
about it again - it will be closed automatically once the
test exits. It doesn't even matter if the test is executed
manually in isolation, or as a part of the entire autotest.
(check_so_deps.sh needs to be modified not to wait for *all*
background processes to finish, but just the ones it started)
Change-Id: If0ded961b7fef3af3837b44532300dee8b5b4663
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/621
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added new setup/cleanup functions for a suite of
unit tests that use a particular blob through out the
tests.
This simplifies the contents of particular UTs,
and brings forward focus of each UT.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib7d69b90fcbe8e6a0ba303475fcbaeb90fa110a7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/945
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Some of the unit tests were only closing a blob,
without deleting it at the end of unit tests.
This is valid behaviour, but to later simplify
the setup/cleanup - a delete blob was added.
This change was made to unit tests where,
it was not focus of particular UT.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I7e338ba6fcb63af26658e8968971a6d56f70807b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1203
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Operation of closing and deleting a blob is common
throughout the tests.
This patch creates helper function ut_blob_close_and_delete(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is closed and deleted, when it is not the subject of
particular unit test (focus is on other functionality).
It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic56594da8cd4c442b62890d62961b44771059679
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Operation of creating a blob (with or w/o opts),
then opening it through out the tests is quite common.
This patch creates helper function ut_blob_create_and_open(),
to do exactly that with the right asserts.
Throughout the UT this function replaces all instances where
blob is created and opened, when it is not the subject of
particular unit test (focus is on other functionality).
It will allow to later use it in setup/cleanup functions
for UT suite - similar to bs init.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia541b855d154503f824bfa3909bcfeac649c5853
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1201
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Always call API on blobid that was received from blob
create, instead of relying on global one.
Where necessary added declaration and assignment.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I3529b4d5e86c1301bb64268029a57873f9f600cb
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/944
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Always call API on pointer that was received from blob
open, instead of relying on global one.
Where necessary added declaration and assignment.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I15531111201d071c471f5be79ce18ff508631677
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/943
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
A lot of unit tests require only creation of a blobstore
then unload it at the end.
This code can be made common and invoked automatically by
CUnit with CU_add_suite_with_setup_and_teardown().
Those functions are called before and after each test in suite.
This should simplify the unit tests by removing the common
code.
Following patches will add further suites with different
setup/cleanup to reduce common code even more.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I984cde2381f4461e188e06050355024a88ebfca9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Next patch in series adds setup/cleanup functions
that use g_bs for its operation.
Particular unit tests should not leave the g_bs
unassigned, if cleanup is supposed to unload it.
It applies to helper functions to reload/dirty_load.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I6e022b6c762981c2c70ebf31f4981a17f54f3591
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Some of the unit tests initalized bs opts,
event if they were left unmodified.
There is no need to perform this operation,
as spdk_bs_init() can accept NULL as bs_opts parameter
to just initialize default values.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iacea1fa9d53796b8c359b611092cc63c6bce4094
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1181
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch decreases the compile time when particular test
category is not executed by tests.
Added skipping the 'examples' dir during verification
of scanbuild scanned files. Only when examples were not
compiled.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I41e7b9d18913fd02e2b6cbe44f933ab2e0371d28
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1172
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Add the error print if there is still remaining_size in
order to provide more meaningful debug info.
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7b15c9c9a630ea7ecb2d3191b73c9c99f7febf31
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1189
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
There was no sync between the start of a w/r/v operation at
a specific block with the previous completion. This resulted
in data miscompares either because the initial Q depth was
sized such that a disk wrap need to occur to complete it or in
the event that an IO takes longer to complete than when bdevperf
loops back around to that offset.
Fixes#1208
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ifa55da54246735e7b603fafd34718965b0f27b94
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1180
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
To provide some more clues when this happens.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I098aaea906a1057e63f07c40a1ebb346a283983d
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1070
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
At startup, use number of created io_target groups to detect completion
and move to the next step. By returning completion message to the
master thread, we can avoid using any atomic operation.
At shutdown, we can use spdk_for_each_channel() conveniently. Put
voluntary spdk_thread_exit() calls into the callback to
spdk_put_io_channel().
To maintain the original behavior, number of threads created is
equal to the number of cores that SPDK app uses.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I92ca4b1c1e8da0f45b47367bc2387701f4a9742f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/890
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Output performance dump per SPDK thread by using
spdk_for_each_channel(). This change is safe even when shutdown
case because spdk_for_each_channel() is serialized if it is
called on the same thread.
Keep lcore information because it is still valuable to know which
lcore each thread ran on for.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I996a4ca2c787d04672743b09a9415145cd8d0171
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/645
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
subsystems
This is optional and most transports will not implement it.
Change-Id: I51e0f1289b0e61a8bdb9a719e0a2aae51ecb451c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jacek Kalwas <jacek.kalwas@intel.com>
In the next patch a new transport call will be added to notify
transports of subsystem->listener associations. The operations the
transports may need to perform there are likely asynchronous, so make
this top level call asynchronous here in preparation.
Change-Id: I7674f56dc3ec0d127ce1026f980d436b4269cb56
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/628
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
If unit tests are not required, add option to disable them.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I387ec043fd47d3033726a51ab673752a521d45c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1171
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This was recently made asynchronous to support virtualized transports.
However, we're moving to add a new call to associated a listener with a
subsystem to transports and the operation that needed to be asynchronous
will actually be performed there. For simplicity, make this synchronous
again.
Change-Id: Ie98136a19c58f0f9bba0d140476de3bbb38e12d7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/881
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch makes bdevperf_construct_targets_tasks() asynchronous
by using spdk_for_each_channel().
The next patch will make performance_dump() asynchronous even when
shutdown case by using spdk_for_each_channel().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I1ebf39fe89a944d977d6f1808cf8607e8f477758
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/644
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
This is a preparation to make bdevperf_construct_targets_tasks()
asynchronous by using spdk_for_each_channel() in the next patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I76c827550b2920abcba14abd7f15a3111ad10d39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/643
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Squash changes for _bdevperf_construct_targets() and
bdevperf_free_targets() into a single patch to shrink the
patch series to reduce the burden of reviewers.
The pupose of the whole patch series is to move io_target from
core based to I/O channel based, and create SPDK thread per I/O channel.
It was not possible to create SPDK thread per core and then associate
target group with I/O channel as long as I tried.
So this patch moves io_target from core based to I/O channel based.
The later patch will create SPDK thread per I/O channel.
Each core has default reactor thread for now and so we can use
spdk_for_each_channel() even when we do not create and destroy SPDK
thread per target group yet.
The following is the detailed explanation:
_bdevperf_construct_targets():
Add a context for _bdevperf_construct_targets() to use
spdk_for_each_channel().
If g_every_core_for_each_bdev is false, set the target group to the
context and create target only on the group which matches the passed
group. If g_every_core_for_each_bdev is true, create target on all
groups.
Only the master thread can increment g_target_count. Hence hold
created number of targets temporary on the context and add it at
completion.
As a result of these changes, spdk_bdev_open() is called on the
thread which runs I/O to the bdev.
Hence bdevperf_target_gone() doesn't use message passing, and
bdevperf_complete() calls spdk_bdev_close() before sending
message to the master thread. Additionally, unregister pollers
directly in bdevperf_target_gone().
These changes also fix the potential issue that spdk_bdev_close()
would be called on the wrong thread if spdk_bdev_get_io_channel()
fails in bdevperf_submit_on_group().
bdevperf_free_targets():
Free each target on the thread which created it by using
spdk_for_each_channel().
This will make possible for us to use spdk_for_each_channel() for
performance dump even for shutdown case because
spdk_for_each_channel() is serialized if it is called on the same
thread.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I4fcdb1024adf4704d3c59215da5669dfdc6cca1b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/641
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch adds better check if the xattr was removed.
It will identify potential bugs when first md sync
takes precedence over the second md sync.
As per suggestion in
https://review.spdk.io/gerrit/c/spdk/spdk/+/774/6/test/unit/lib/blob/blob.c/blob_ut.c#7655
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I463f2455614ef11cc1512dc2fb1972ba1f024337
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1109
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Operation of dirty shutdown of blobstore and reloading
it occurs often enough in UT to provide a common function
to perform it.
Added ut_bs_dirty_load() to facilitate this.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0079dfabd64eaec6495db02fe200be1d6116f0c7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Operation of unloading and loading blobstore
occurs often enough in UT to provide a common function
to perform it.
Added ut_bs_reload() to facilitate this.
Couple occurences in this patch actually fix
test cases where mistakenly opts were not passed.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib46ed395cd134feaa540e00a334ae861872b3ef4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1091
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Couple unit tests were still refering to g_bs pointer,
rather than the one local to particular UT.
This patch fixes this.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If343eb561a48c4547d499139d76e1db6b2d7f3bf
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1090
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If the hugepage we use is 2MB, the numble of hugetlbfs will exceeds
the limit of 8. we must pass "-g" to vhost_fuzz app, which will pass
"--single-file-segments" to DPDK to create a single non-physically-
continuous hugetlbfs file for all its memory.
Here we pass "-g" to vhost_fuzz app which make the script work under
both 2MB and 1GB hugepage.
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I0287584c54f03a379957d6885c23e21a1fd10c4b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1148
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently we will free the cache buffers in the file R/W
data path, however we can start a thread to do the cache
buffer reclaim regulary, so that we don't need to free
the cache buffer in the Rocksdb thread context. By
doing this, we can also remove the global cache buffer
lock in following patches.
Change-Id: Icb0fbc49baecd1e9d86a5fcfe400758dfc3c53a2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/941
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
GCC9 complains:
./db/version_edit.h:134:71: error: implicitly-declared "constexpr
rocksdb::FileDescriptor::FileDescriptor(const
rocksdb::FileDescriptor&)" is deprecated [-Werror=deprecated-copy]
From what I see this can be fixed by explicitly
defining some constructors and assignment operators,
even setting them to `= default;`. I didn't dig into
this further, just ignore the warning for now.
Change-Id: Ia0ee0cc5fc1dce36f7098959d383b08855a825df
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1082
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Remove if/exit steps from calsoft.sh and move the condition
to parent script iscsi_tgt.sh. This still to run iscsi_tgt.sh
on systems which don't have Calsoft installed but still allows
CI to detect missed tests thanks to autorun_post.py.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I0eb186fe80b377f0c023774adf8ff321af7dfa81
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1075
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Add function to print warning with optional message
in case we'd like to skip a test.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: Iacb1031aac0e17ce80327ca826c51fb2b99a6be0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1080
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
No need to exclude the autotest_common.sh twice when
creating test list.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ice7be31a9d7d284dd411ebfd3854e075b37c0275
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1088
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
To avoid the starnge formatting, typedef has been used. But this comment
is hard to get the meaning. So stop breaking after return type for this case.
The strange formating is
struct spdk_scsi_dev *
spdk_scsi_dev_construct(const char *name, const char *bdev_name_list[],
int *lun_id_list, int num_luns, uint8_t protocol_id,
void (*hotremove_cb)(const struct spdk_scsi_lun *, void *),
void *hotremove_ctx)
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I20c30b225b3b96fa9207b4a89e3210c8a97fb1d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
It was removed for other/negative.sh test.
There is no need to keep it more for one scripts with test
as vhost configuration can set by series of rpcs.
Change-Id: I86937a5f38641f5ce48c00836e050623c90bdd27
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Need one more malloc bdev.
Change-Id: I9884829e1683bfa0e9653f2edbb020d002b7b060
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1055
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Rewriting test cases:
- tasting_positive
- tasting_lvol_store_positive
- tasting_positive_with_different_lvol_store_cluster_size
These three python test cases were merged into one test,
as there was a lot of similarity between them.
Change-Id: Ie68568e2b6b9e574917b6b350e96b0f42c999c95
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/683
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add raid5 module and unit tests. Use './configure' with option
'--with-raid5' to enable it.
Change-Id: I9f07da8c3567fa65499444899c899adaa2e29550
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/855
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
If we did not configure with "--enable-debug". it will
trigger the compliation failure since this function is
not defined.
Change-Id: Ia1795da736be55d4dd1418a2e74edefff7797f11
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1097
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Michael Haeuptle <michaelhaeuptle@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In case something goes awfully wrong with the test which calls this
particular function, e.g:
https://ci.spdk.io/results/autotest-per-patch/builds/4787
the CI's autotest runs may be blocked for max of 15 minutes before
failing the build. Instead, introduce internal timeout for the
routine itself (default of 30s) and fail the test sooner when it
expires.
Change-Id: I2c792c1781987e80e34c1fa19ec3eef43c25e38a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1026
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
We do not want to skip tests, especially that the
'if' statement is inside the test script.
So even though the test wasn't actually ran it still
is reported as executed because "spdk_nvme_cli.sh" returned
with rc=0.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I89c553aebcc1346f8f584dd86eeae1ca146e7340
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/980
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Michal Berger <michalx.berger@intel.com>
When running unittest_thread in fedora29 with memcheck, there is
error "Conditional jump or move depends on uninitialised value(s)"
in for_each_channel_unreg test.
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: I38abf4c70ff513e39232b83357d571dded8f02c3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1072
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
In case the random string beggins with `-` (still a valid ascii char
that nvme supports) the rpc.py arg parser will treat it as an
additional argument instead of actual subsystem value.
To mitigate, escape the `-` in case it's the char that starts the
string.
Fixes: 1240
Change-Id: I07bd8134fb9fc16234e72e9ebd27e2b6b90d6b62
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Replaced single global write buffer with the per-io_channel write
buffers. This means that the "rwb" module and all of its references
were removed and replaced with the recently added interfaces.
Change-Id: Idc899d3a4d63a8a2bede1ac26549ed06e9a2e784
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/909
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Added queue responsible for keeping track of full, ready to be written
batches. A batch might be put on this queue in case it'd already been
filled, but could not be written out due to lack of resources or it was
written out, but failed and needs to be resent.
Change-Id: Iba49cd359425300d21b8100f13f17189e99d4c7c
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Two functions were added:
- ftl_get_addr_from_entry() returning an address pointing at a given
entry
- and it's inverse, ftl_get_entry_from_addr(), returning a write buffer
entry for a given address
Change-Id: Iea1e2e59f240f50183fdb549280a0f0b722efbef
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/906
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This patch adds the ftl_batch strucutre, which describes a single batch
of data to be written to the disk. It's comprised of multiple write
buffer entries (the actual number depends on the write unit size of the
underyling device).
Additionally, a function responsible for filling out a batch was added.
It iterates over available IO channels and dequeues submitted write
buffer entries until a batch is completed. The IO channel queue is
shifted, so that each time the function is called a subsequent IO
channel is used first, which guarantees that all channels are treated
fairly.
Change-Id: Ie61d8c6cb51d5c5175540c975447bc27590c5cb4
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/905
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Added functions responsible for acquiring/releasing write buffer entries
from the per-io_channel caches. They're also in charge of limiting user
writes by reducing the number of available entries to the user.
Change-Id: I23adfe36065139036f9b72a016e34fbf6eb29230
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/904
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
The IO channel pointers are now stored inside an array of the device
they belong to. Once write buffer entries are tied to IO channels,
it'll provide a method for dereferencing an entry from its address.
Change-Id: Iaf401525eb0f5af8dc6047a1dc8bae11b56761d7
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This is the first patch of a series that replaces single global write
buffer with per-io_channel buffers. This change is intended to improve
performance for multithreaded workloads.
This patch changes the way the ftl_io_channels are allocated by only
keeping an ftl_io_channel pointer inside spdk_io_channel's context. It
allows for delaying IO channel destruction, which in turn allows the FTL
to iterate over all exisiting IO channels without locking.
Change-Id: I5e0cab8043a2b5f747e971dd3d65ed2546c8cf26
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Follow the idea of bdev_get_iostat in module/bdev/rpc/bdev_rpc.c.
Add a global variable g_bdev_count and then set it to 1 as sentinel
value in bdev_construct_targets().
Subsequent patches will make _bdevperf_construct_targets() asynchronous.
Then bdevperf_construct_targets() will be truely asynchronous.
This patch is too small to be an independent patch but keep to
hold previous review result.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I49d0b6e6d61df65d4eb31c402f98cc4300482e20
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/639
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add a flag resched to check if reschedule operation is requested
to struct spdk_lw_thread. Add _reactor_resquest_thread_reschedule()
to set the resched flag, and add it to the case SPDK_THREAD_OP_RESCHED
in spdk_reactor_thread_op(), and return true in the case
SPDK_THREAD_OP_RESCHED in spdk_reactor_thread_op_supported().
Then _spdk_reactor_run() checks if the resched flag is true for each
thread. If true, set the resched flag to false, and remove the
thread and call _reactor_schedule_thread(). Add continue to avoid
use-after-free issue for both reschedule and terminate cases.
This idea follows voluntary thread termination and will remove our
worries for all complicated rare cases.
Add unit test case to verify this update.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I656872d32dbb469ae70f771cd0419a77236bfe18
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/500
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is necessary to avoid leaving the machine in a bad state between
test runs.
Change-Id: I2c68d2dc88f0f9f4cab099564d25a3e21eb5671c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1027
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Since ftl supports only one core thread ftl_thread
structure is not needed anymore.
Change-Id: I66eda9249d74b3eff0c4473dcd0aba4a6135e296
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
This reverts commit 29e9fdc857.
Nightly tests after merging the reverted patch show that it does not
pass in current state. It affected the per-patch, since the new
tests weren't added to skipped_tests list.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If8a58cbe9ecd0e58f20c0a9ee844bc9a8ee046a3
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1045
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
With this change user can defined test cases that he wants to run.
It could enable to run hotremove test cases per-patch and nightly.
Change-Id: I6876f39886a347ada4dd548a2f45b99a37207c3e
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It comes up on failed fio runs and bloats the log.
Change-Id: I8dd0196bfc98b3ee888aca0def83f0961fbc1197
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1010
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The goal of this test is to make sure that proper parts of the lib
fail as they should. These particular tests focus on:
- spdk_rpc_nvmf_create_subsystem(): model and serial number checks
Change-Id: I504279d08b192d6dfe37a8a601eda44084b596d4
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/693
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Add tests for the following features:
- SPDK_NVME_FEAT_TEMPERATURE_THRESHOLD (4),
- SPDK_NVME_FEAT_ERROR_RECOVERY (5),
For Temperature Threshold feature the validity of the THSEL and TMPSEL
is covered as described in the NVMe spec (1.4):
"Figure 279: Temperature Threshold – Command Dword 11"
For the Error Recovery feature the validity of the DULBE is covered
as decribed in the NVME spec (1.4):
"Figure 280: Error Recovery – Command Dword 11"
Random value is selected for the lsb in the cdw11 (0x42).
Change-Id: Ia57ab4d79439ec315ddc9bbfdad8400aa926062a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/692
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
SPDK_AUTOTEST_DEBUG_APPS - if set to 1, all applications will be
executed with --log-flags=all
This should increase, to some extent, coverage of each application.
Change-Id: I1fc82a837abb7e94bda55f47056a120bd9be40d7
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/691
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Since there's more than one fuzz app covering different areas, rename
FUZZ_UP to VHOST_FUZZ_UP to make the type of the app clear right off
the bat.
Change-Id: I7730f1e14895d9221ec76dea0f53b5054a97be1c
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/978
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Instead of declaring each app in the actual test, source their default
declarations from a single location. Additionally, use them as arrays
now instead for easier cmdline management.
Change-Id: I0ea39403f070dc57987065ebb800e91c35de643a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/690
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Currently, this particular package is not available in fedora 31's
repos, thus don't exit the setup if its installation fails.
Change-Id: I56852ed172c95b23c92e25b1cec391b4e909f2bd
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/936
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Since '' are placed within "" they are treated as literal part of the
argument passed to configure. Due to that, configure fails to run cc
since all -W flags are treated as one.
Change-Id: I69e182f308628cc5b63bd363d3a47cf87336a59a
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/1024
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Remove the prefix "spdk" from spdk_reactor_schedule_thread(), add
spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().
For SPDK_THREAD_OP_NEW, spdk_reactor_thread_op() calls
_reactor_schedule_thread() and spdk_reactor_thread_op_supported()
returns true.
Then replace spdk_thread_lib_init() by spdk_thread_lib_init_ext()
with spdk_reactor_thread_op() and spdk_reactor_thread_op_supported().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I232a3b2c6bcaf4d86b0dd3cefacd3e47eadda6d8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add enum spdk_thread_op and two function typedefs spdk_thread_op_fn
and spdk_thread_op_supported_fn.
The first operation type of enum spdk_thread_op is SPDK_THREAD_OP_NEW,
and it is used as an alternative to spdk_new_thread_fn.
Add global variables, g_thread_op_fn and g_thread_op_supported_fn, and
then add spdk_thread_lib_init_ext() to initialize these.
spdk_thread_lib_init() requires both of thread_op_fn and
thread_op_supported_fn are specified or not specified.
spdk_thread_create() calls g_thread_op_fn() with SPDK_THREAD_OP_NEW
if g_new_thread_fn is NULL, g_thread_op_supported_fn is not NULL,
and g_thread_op_supported_fn(SPDK_THREAD_OP_NEW) returns true.
Update unit test to test these addition.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I56db903f62437f6ff3198248ffc5dede396c22bc
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/967
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add an unique ID for each created SPDK thread. Use a single 64 bits
variable, g_thread_id, and guard its update by the global mutex
g_devlist_mutex. For our safety, further thread creation is not
allowed if g_thread_id rolls over, and request user to restart SPDK
application.
Besides, as a minor update, move the debug log down and add ID to it
in spdk_thread_create(), and ID is added to thread_get_stats RPC and
framework_get_reactors RPC.
The thread ID will be used to set the cpumask of the running thread
to the specified value in the subsequent patches.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic09f11d4c7175c3b89acba6a42e76063acd0d1a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/498
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
There is a warning triggered when holding ref to const obj and passing
to these getters.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I2c7b4ea0d325d84d66923fc524273ea44a3a311b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/997
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
With the JSON configure, the base device will be opened on the
same thread (RPC thread) to handle the JSON operation. Later the
virtual device upon the base device can be opened on another
thread. At the time of virtual device destruction, the base
device is also closed at the thread where opening the virtual
device, it is actually different than the original thread where
this base device is opened through the JSON configure.
Add a thread here to record the exact thread where the base
device is opened and then later route it back to handle the base
device close operation.
Change-Id: Ib95e8d190bd87158ae1ecc6698da95ccc4ba9579
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/992
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
In the autotest, when calling kill_stub() function, there is error log
like this: "Device 0000:83:00.0 is still attached at shutdown!", so it's
better to detach the controller when exit the stub process.
But after call spdk_nvme_detach() in the stub process, there is another issue:
1. NVMe stub running as the primary process, and it will send 4 AERs.
2. Using NVMe reset tool as the secondary process.
When doing NVMe reset from the secondary process, it will abort all the
outstanding requests, so for the 4 AERs from the primary process, the 4
requests will be added to the active_proc->active_reqs list.
When calling spdk_nvme_detach() to detach a controller, there is a
assertion in the nvme_ctrlr_free_processes() at last to check the
active requests list of this active process data structure.
We can add a check before destructing the controller to poll the
completion queue, so that the active requests list can be flushed.
Change-Id: I0c473e935333a28d16f4c9fb443341fc47c5c24f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/977
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
There are synchronous security send/receive APIs defined in nvme.h,
however, we still need the asynchronous APIs so that we can make the
OPAL library can be used in asynchronous way. As the asynchronous APIs
are already defined in nvme_ctrlr_cmd.c, so just export them to public
APIs.
Change-Id: I5646f342a4bf70faad37daa956476f05a1327bcc
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/675
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Some distro installations don't provide the time package by default.
Since Bash is capable, use its time instead.
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I8ca31e233c23d9c89be6afba29affe45f3f9882c
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/961
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
If this call fails, for whatever reason, $ROCKSDB_CONF won't be
removed.
Change-Id: I6fff996b04415fc760d3593c0f628d2631ce96ba
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/960
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Since both stdout and stderr of the db_bench run is redirected to a
file, xtrace won't write out anything of value in case said run fails.
Example:
https://ci.spdk.io/public_build/autotest-per-patch_4071.html
To avoid looking into the file directly, try to locate last .txt that
was written out and dump its contents to stderr to increase the
visibility of what might have caused the failure.
Change-Id: Ic4873626e979130f2722a4f2d5266d93bc2bde76
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/959
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
When running unittest_nvmf in fedora29 with memcheck, there are
many errors about "Conditional jump or move depends on uninitialised
value(s)". The failed tests are:
in ctrlr_ut:
test_reservation_notification_log_page
fused_compare_and_write
in ctrlr_bdev_ut:
spdk_nvmf_bdev_ctrlr_compare_and_write_cmd
Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Ib1e6a744e86876c15ee53206909364e853574dd1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/965
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
while the size of namespace is changed,
the resize event will be notified.
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Evgeniy Kochetov <evgeniik@mellanox.com>
Signed-off-by: Allen Zhu <allenz@mellanox.com>
Change-Id: I5d85f17df898dc21c0ae1eb9f529dcb624a457ac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/849
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Always call API on pointer that was received from blobstore
initialization or load, instead of relying on global one.
Where necessary added declaration, assignment and asserts.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic4198ac984c13a8b77428deaa32ce9864e6750c1
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/942
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
CUnit provides a helper macro CU_ADD_TEST() that
simplifies using CU_add_test() function.
Test name no longer needs to be provided,
instead test is named after test function name.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I9f5d9e0d8e72ab36b2a5c04e33c336e6c0a1fa6a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/931
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Failure to prepare CUnit for tests is always a fatal error,
and application exits.
Most test managment functions in CUnit set some CU_ErrorCode,
that can be retrived by return code from a function or CU_get_error().
Thus usual structure of UT in SPDK checked every output from every
test managment function.
There is a helper function CU_set_error_action() that when set to
CUEA_ABORT, will abort the application run with proper error return code.
Along with nice error message.
For example if one were to add same UT twice:
"Aborting due to error #32: Test having this name already in suite."
Application return code set to 32.
Using CU_set_error_action(CUEA_ABORT), removes the need for error checking
on test managment functions in CUnit.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ida7de5e040b714509e79957d47a006ee518a42b8
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/930
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Expands blob_simultaneous_operations to cover case
when second md sync is perfomed without waiting for the first
one to complete.
The blob state is forced in UT, but in normal usage could be
changed by modifying xattr, allocating thin clusters or other.
Related to #1170
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I880d289cdf957d4513c080efdcb7a5d8c6b3b447
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/775
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This new UT is checking validity of md sync being run
along another md sync with changed metadata.
At this time added case where first md sync is being
interrupted by removal of xattr with second md sync.
This interruption is perfomed at increasing number of
poller executions, until the number is enough to
complete first md sync.
There are two expected states of used_md_pages array,
either with xattr or without. The state is verified
after md syncs.
This UT will be expaned in similar manner by other
operations (than xattr) that casue changes in persisted metadata.
Related to #960
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I251dca92ffc3080d8dc503a7f1ff342aa59adef9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Making use of recently added utility to poll exact number of times.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I23a9dc702c899285626522ace8ec77a9ba0f7757
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/773
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Right now functions poll_thread() and poll_threads(), polled
always until there were no further messages to process.
This allows to put next operation to execute only after
previous one completed.
Function poll_thread_times() allows to poll certain thread
for exact number of executions. If 0 is passed to max_polls,
then this function executes single message and all pollers,
until no messages are left.
With this change it is possible to only run a thread
selected number of iterations. Allowing to add another operation
without first one finishing.
It will become useful in blobstore UT, where two operations
will be executed with slight delay between them.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4a6537695389f30130ea2fd2fd43d7b2cb2dea39
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/772
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
$rpc_server is hardcoded inside the function and can't
be empty, so remove the check for that.
Change-Id: Ic819089eb113b14a7c2407cd5ce882bcdee044bb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
To make room for other raid module unit tests.
Change-Id: Icf8b8ff0c9051ccb4c7ebd784807370f8be1a8fb
Signed-off-by: Artur Paszkiewicz <artur.paszkiewicz@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/853
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This is a duplicate of test_hotremove_lvol_store_base,
previously known as test case 301.
Change-Id: I2e49652a0d94fc32f8193f724a254a50484844cc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Firstly, change unmap suppor and I/O size checks to do once per
bdev. It is enough to check if unmap is supported and check if
I/O size is multiple of data block size once per bdev. Hence move
these checks from bdevperf_construct_target() to
_bdevperf_construct_targets().
Secondly, use calloc() to remove unnecessary zeroings in
bdevperf_construct_target().
Thirdly, factor out getting next io_target_group from
bdevperf_construct_target(). This is a preparation to make
bdevperf_construct_target() asynchronous by using
spdk_for_each_channel().
These may be better to break-up but these are already included in
the long patch series. So to reduce the burden of reviewers,
squash these into a single patch.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2cb94c8aa2b6b1a1b9e9a04da533e1fd093f8f29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/640
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Factor out constructing multiple targets for a single bdev from
bdevperf_construct_targets() into an new helper function
_bdevperf_construct_targets().
Then change the return type of bdevperf_construct_targets() to void
and continue creating targets for subsequent bdevs even if failing
for any bdev. Additionally, remove a couple of comments in the source
code to avoid misunderstanding.
These changes will make us easier to use spdk_for_each_channel()
in bdevperf_construct_targets().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I3363931749b863c8de619939939d401f14e43d15
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/638
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Creating targets, creating tasks, and then running I/O is the actual
ordering of start-up. So adjust ordering of function calls to make
easier us to follow.
Change the return type of bdevperf_construct_targets_tasks() from
int to void, and call bdevperf_test() at the end of
bdevperf_construct_targets_tasks(). Then replace bdevperf_test() by
bdevperf_construct_targets_tasks().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Id1396ff4ba199778300f5ddaf79db5f8f2e0ccb6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/642
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Move bdevperf_test() into bdevperf_construct_targets() from
_bdevperf_init_thread_done() and rpc_perform_tests(). Following the
last patch, this consolidation will make the code a little cleaner.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ided4653941f45ff6ac8edbea504560cc8a9fb5a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/637
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Make sure that $rootdir and $testdir are accessible in the sourced
environment. This is to follow the convention seen throughout the
entire repo and make sure that nothing that depends on these paths
fails in case they are missing.
Change-Id: Ie20754fc7aca7aab3636cb022499e9ff2f2dd30b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/777
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Allow ftl_io initialization with a iovec array instead of a single data
pointer. It'll make it possible to use the vector version of the IO
commands.
Change-Id: I34c1e398cea681f59321605e081366785aa29128
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/894
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Maciej Szczepaniak <maciej.szczepaniak@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
It is memory optimisation as transport id is 'heavy'. As a side effect
simpler handling of listen and stop_listen on transport specific layer.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I4e9d0e0c5eee2d570ec4ac9079270c32d5afb8db
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/626
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
We've now supported ZCOPY io type and Malloc bdev will use
ZCOPY for the read/write operation. Add the support for the
test case of Malloc with ZCOPY operation.
Change-Id: I291fc9ef82b1eae800c97c5a6b7a0c810c357fe0
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/540
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The copy engine library, modules and public APIs have been renamed.
Use of the word `copy` has been replaced with the word `accel`
short for accelerator in preparation for adding new capabilities
in the future. Additionally, APIs for what was previously called
the `memcpy` engine have been renamed to identify the engine as a
software accelerator.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ia607aa718416146fbba1e6792b8de0f66bd8a5de
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This will be useful in the upcoming spdk_nvme_poll_group api.
Change-Id: Id83340a2ce9887817312f5aac38db4de8c588974
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/577
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
MD026 Trailing punctuation in header
This rule is triggered on any header that has a
punctuation character as the last character in the line
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I3ab4894092ef6b5f920d89b74e43c2e46e9581c0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/657
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
MD022 Headers should be surrounded by blank lines
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I768324b00fc684c254aff6a85b93d9aed7a0cee5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/656
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Dry run processing in iscsi_parse_portal() now only checks if
portal string is not NULL. Portals are managed not by fixed size
array but linked list now. Hence dry run processing is not needed
itself now.
So remove the dry_run parameter from iscsi_parse_portal() and
the dry run loop in iscsi_parse_portal_grp().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ib7cb444b51581b8ee603388aad34bc58d1cc9023
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/493
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The next patch will create a SPDK thread for each poll group at
startup. Hence iSCSI configuration management will be done by
different thread from these poll group threads.
Hence send message explicitly to add connection to poll group even
at startup. We can do this for the current master branch.
Remove some code related with SPDK thread framework from unit tests
for iSCSI portal group because thread management is moved to
connection.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I40cacdb2066f65866f7ef83cf3b3e4e8b8cd322e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/489
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This allows to avoid calculation of ioccsz bytes on each request
and removes access to "cold" ctrlr structures in data path.
Add UT to check validness of calculation
Change-Id: I55ceff99eb924156155e69a20f587a4f92b83f0b
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This patch introduces tests for basic RPC commands not used in
other tests:
bdev_nvme_apply_firmware
Change-Id: I3a7aff1720108548b03d581fa7664d1ff5498be5
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/533
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
When running sock_ut in fedora29 with memcheck, there is an error
"Syscall param sendmsg(msg.msg_iov[0]) points to uninitialised
byte(s)".
Signed-off-by: richael zhuang <richael.zhuang@arm.com>
Change-Id: Iaddb85ed2da2f45e200569c936f05174bc89c99b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/874
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
If set, SPDK will continue loading the JSON config even if
some commands caused an error. This can be useful when loading
RPC config from spdk_tgt into e.g. bdevperf, which supports
only a subset of RPC commands and would usually fail with
"Method not found" message.
Resolves#840
Change-Id: I070fea862fd99e5882d870e11e6a28dc9d0c8ba6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/620
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This option is used to make hello_sock can use the
designated sock implementations. We need this patch
since we will provide another uring implementation.
So better to pass the name. Otherwise if the users have
many different implementations, VPP implementation
could not be the highest priority for test.
Change-Id: Ibb3862e9e6588743ec9c01074904ed4f7c9c06a5
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Sample output from rxe_cfg:
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
eno1 yes tg3 1500 192.168.100.1 rxe0 1024 (3)
eno2 no tg3 1500 192.168.100.2 rxe1 1024 (3)
eno3 no tg3 1500 192.168.100.3 rxe2 1024 (3)
Sample output from other systems rxe_cfg:
Name Link Driver Speed NMTU IPv4_addr RDEV RMTU
ens3 yes e1000 rxe0 1024 (3)
ens4 yes e1000 rxe1 1024 (3)
Looking at the above output, amount of columns after greping particular line
varies between systems and configurations.
Meanwhile "rxe_cfg status" is an equivalent command, but allows to add
argument that will be matched to RDEV name.
With this patch we no longer depend on awk to select columns, but instead
printout all soft roce enabled devices and grep for particular dev name.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id4483c58f0fdcef2f38abdf56257658643a3069f
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/619
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
It is a leftover from previous structure of this test.
Since then usage of SUBSYS_NR was removed and this part
was left unused.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia1f6f8046f82370e793182b47920b4bf4d3231a0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
The comment in the header file has already said that all associated
I/O channels must be released before calling spdk_thread_exit().
This patch actually checks if it is satisfied in spdk_thread_exit().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I56ac50b561c6ca91d3dc2d60c21c8d91d38f081b
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
Change spdk_thread_exit() to return -EBUSY if the thread has any
registered poller. We enforce all pollers including paused poller
are unresitered before the thread is marked as exited.
By this change, a bug was found in reactor_perf test tool. Fix it
by adding spdk_poller_unregister() and add the g_ prefix to avoid
future potential errors.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If7f40357c9a6f4101b3998ea0da3cc46cc435031
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
Change spdk_thread_exit() to return -EINVAL if the thread is already
marked as exited. This will be helpful to detect wrong call sequence
of voluntary thread termination.
Besides, update reactor shutdown and unit test framework shutdown
to incorporate this change accordingly.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2296c61e273bf4d9580656dcbc2da0e8a8f3bcf7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/671
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
By the last patch, the asynchronous release of I/O channel will
complete even after spdk_thread_exit() because pending messages will
be reaped.
Then this patch stops new allocation of I/O channel after
spdk_thread_exit().
Hence we will be able to release all I/O channels for exiting
thread within finite time.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I48a45bcba7c4b2c62d8c9d398ac35a584b533627
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/821
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This is a preparation to support voluntary thread termination by
calling spdk_thread_exit().
Previously, the exiting thread had discarded all pending mesasges.
We change this to stop accepting any new message in spdk_thread_send_msg()
and reap pending messages in _spdk_msg_queue_run_batch().
Add unit test case for the new behavior. Adding g_ prefix to global
variables for clarification is done together.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ida78e7bb1b86357602aea6938dd514897b67edd6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/482
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
The test case for_each_channel_remove() did not have any
assertion and the count was not incremented correctly.
This patch fixes several issues in for_each_channel_remove()
by counting spdk_get/put_io_channel() and spdk_for_each_channel()
correctly and achieves what we wanted to do in this test case.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iceffd924f8887452bd7dad48e30d121874ab0b05
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/483
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
remove revs to a gerrithub review and a test plan doc
that no longer exist
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I05f4466181e27f60e1e717c42649726ad46f0cac
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
if given script, which executes under the debug tracer, fails before
xtrace_restore() is called, the BASH_ARGC[] will miss all the arguments
which were passed down till that very point. Similar case can be
experienced after tracer is disabled as then all the arguments hold by
BASH_ARG{C,V}[] become unavailable (i.e. until tracer is enabled again).
Since there's no actual benefit from toggling the extdebug (in fact, it
could break DEBUG|RETURN traps if ever used), enable it once when
autotest_common.sh is sourced and keep it enabled throughout entire
execution of given script.
Change-Id: I01001ead1570967a2e550d993f85f12b9f62553e
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Subsequent patches will call bdevperf_test() as callback to
bdevperf_construct_targets(). Changing return type of bdevperf_test()
to void and including its error handling will make the code cleaner.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I30fe38cfb73fbc593d079d5b66a540c0ce96fe19
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
_end_target() is an exception, and move just above bdevperf_complete().
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2197a3b7ceb36ab29f0b69e31f3babd4e996f193
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/635
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Put rpc_perform_tests() next to bdevperf_run(). This will improve
readability a little.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I8816a88be13e794f39ded3ff12e9a76e40c8282a
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/514
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Remove two global variables and use variables per target instead.
g_min_alignment:
Bdevperf creates task on a target after creating the target, and
we has removed any limitation about alignment described in the comment.
Remove g_min_alignment and use each bdev's alignment to call
spdk_zmalloc() instead.
g_buf_size:
We had set the size of task->buf by not g_buf_size but g_io_size
by mistake. We have not used any global buffer pool and can use
buffer size per io_target instead.
Delete g_buf_size and add buf_size to struct io_target. Then
initialize target->buf_size in bdevperf_construct_target() and use it
instead of g_buf_size including the fix in
bdevperf_construct_task_on_target().
Besides, as a minor cleanup, remove duplicated initialization of
global variables in this patch. Global variables are already
initialized at their definition. Remove duplicated initialization
from main() function. It is ensured that global variables are
automatically zeroed but write initialized value expilcitly for
compatibility and clarification.
These will be helpful to parallelize targets and tasks management among
multiple threads.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iba23dec3e1da8810da7523da09bae858eb4484a2
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/512
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The on_error_exit() function was being called via the local ERR trap
to cleanup after the tests in case any failure occurred during the
execution. However, this was masking the backtrace trap so the
on_error_exit() had to also call print_backtrace() to compensate -
this was resulting in an additional function being unnecessarily
placed onto the stack and included in the actual backtrace.
Avoid the above by refactoring on_error_exit() into simple cleanup()
function called upon exit()ing from the script. The rationale is that
cleanup has to happen regardless if the script failed or not and
elements like $conf_file are always created from the very scratch by
the script itself anyway prior running the tests.
Change-Id: I6bb8f4155525037624f0bd5aab288f8c502141f6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/456
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
igb_uio doesn't necessarily have to be loaded into the kernel as SDPK
still may be built with support for it but only particular set of
tests is actually using it. If such a condition is met then rmmod will
fail the entire test run since it always exits with != 0 if given
module is not found.
Avoid this by checking first if igb_uio is actually in use.
Change-Id: Ib97488797c657f810b588a0b427e578807ebe6c6
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/625
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
"MD032 Lists should be surrounded by blank lines"
Fix this markdown linter error by inserting newlines or
adjusting text to list points using spaces.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I09e1f021b8e95e0c6c58c393d7ecc11ce61c3132
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/434
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
If stub terminates right after execution, e.g. due to lack of system
resources (requested number of cpu cores to run on, right amount of
memory, etc.) start_stub() would end up blocking forever since there
wouldn't be any entity around that would mknod spdk_stub0 for it to
break the loop.
Avoid the above scenario by checking if $stubpid is still visible
under procfs and return if it goes missing.
To make sure kill_stub() is still called to clean up after start_stub()
declare proper trap prior the call to start_stub().
Additionally, avoid potential stderr noise in case kill|wait are told
to act on a PID that's not visible in the ns anymore.
Change-Id: Ief41200c57957f84b4f96a54baabc8da1f27dd43
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482653
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Currently, kill_stub() doesn't gracefully handle this particular sysctl
since it overwrites it with assumed default that doesn't necessarily
have to be a part of kernel's config on a given system.
Don't presume what the setting should be, instead, read and save the
current value and try to restore it whenever kill_stub() is called.
Change-Id: I1f1ee85c29d5e2ec2f442a54f700e3bc45ee2437
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482652
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
fdump puts out buffer in following format:
"00000000 76 61 6c 00 val. "
Each buffer is displayed with hex values up to maximum, then
followed up with a value. This is done to keep starting value
with the same alignment between each consecutive line.
Value contents should end at its length instead of
adding trailing whitespace.
After this patch, output is changed to:
"00000000 76 61 6c 00 val."
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I26b327e83f296ba3865f1a337f4a70764a80e2b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483706
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Match script before this change ignored all input
when passing just a new line in ignore file.
This was because \n symbol was stripped from that
particular line (chop()) leaving empty string.
This always matched all of the input when using index().
Now when comparing the input line and ignore line,
we check that ignore line is not empty.
Besides that, to still allow ignoring new lines
we check and compare both strings directly.
Changed chop() to chomp() to prevent further breakage,
difference is that chomp() only strips new line characters.
As that was the original intention anyway.
The loop changed so we exit on first instance of ignored
line matching, rather than mention particular line
multiple times for each matched ignore.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I62d48e1130c600ffff6713d2748239cc955bbe9e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Empty line in ignore file caused all output from the match
tool to be ignored. This made the test to always pass,
regardless of changes made.
Next patch in series will be fixing the match tool,
meanwhile this patch focuses on making the blobstore
test match and ignore files right.
This patch adjusts match file in following ways:
- keep EAL and crypto output up to date
- use $(N) to match any blob id
- adjust out of date xattr value print
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia0a2312d9691487acbdd06dc08451232171cc46d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483705
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Again, smuggle it into one of the existing test case
to create more opportunities for this test to fail.
We'll now trying to remove an inexistent lvolstore
when there's a valid lvolstore present.
Change-Id: I64a72a8ffe38ba6939d0818d2c6c81e49748a33c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462468
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
This test case is about hotremoving a malloc with
an empty lvolstore on top. We rewrite it to bash,
but also introduce one more similar test case but
with a single lvol in the lvs.
Change-Id: Iea9666080fe5e73befd97ec5e0b2898d8b30ecd6
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Those test cases were basically testing hotremove, so
that's how we're going to name them now.
Change-Id: Ib79ab7e50e3022df981f7092436b0a0991d5c9b9
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
The test case covers destroying the lvolstore using its
human-friendly name instead of uuid. We destroy lvolstores
all over the basic construct tests, so we just smuggle
one removal by name into one of those existing test cases.
Change-Id: I5027382f727be226f601573367bfcd214baf474a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462190
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
We didn't really test it so far.
Change-Id: I58e57a47465e3c4bc409d497706a56c7aa5cdaa1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462465
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
We already perform the same steps as a part of different
test case in basic.sh.
Change-Id: I0e1931307b3de69df95b2a2994b87147028677e1
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462464
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
We did not test it so far.
Change-Id: I0769c31691535b5d8f9c49e07899f035208efebb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462188
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Resize tests were extremely basic so far, extend them
with a test case that actually tries to do some I/O
before and after the resize.
I would like to keep it as a separate test case,
because it uses NBD which used to be quite buggy at
some point. If the resize tests fail on this new test
case, it means the previous very basic resize tests
have passed and the lvol resize functionality itself
doesn't show any failures - this might potentially
simplify some root causing later on.
Change-Id: I7ee45243ee4fc025fc71f6eee4c946211c07a78a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462187
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Added resize.sh. There won't be too many test cases
in there just yet, but eventually we'd like to lift
some restrictions on lvol resizes and there will be
a fair amount of extra cases to cover.
Change-Id: I3d9db16003ac806241678cfd8441281d557c214c
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462185
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Use spdk_thread_send_msg() for notification of bdev hotplug
and test completion instead of spdk_event_call().
Each I/O target group is associated with SPDK thread now. Hence
use spdk_thread_send_msg() to do bdevperf_target_gone() on the
specified thread. The group is held in struct io_target by the
previous patchm and remove target->lcore safely because it is
not used anymore.
Then use spdk_thread_send_msg() to notify master thread.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I88eadbdd1f120161cf484793f14f5eb0b44d010f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478818
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Register the global bdevperf object as an I/O device by
spdk_io_device_register(). Create and destroy I/O target groups
asynchronously by using spdk_for_each_thread() and
spdk_for_each_channel().
spdk_for_each_thread() and spdk_for_each_channel() serialize the
execution among multiple threads, and any lock is not necessary.
Additionally, for bdevperf_submit_on_core() and
bdevperf_stop_io_on_core(), replace for their loop + event call by
spdk_for_each_channel() and rename them by
bdevperf_submit_on_group() and bdevperf_stop_io_on_group(),
respectively.
This advances the work to move bdevperf from core based to thread
based.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I63b512d0b8e734e862afe1f7a4053414b99204d7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/478780
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
With the recent changes which added usage of writev_async to
both TCP target and initiator, nvme_tcp_pdu::writev_offset
becomes useless since it is not updated in data path. This
field is only used in UT. Remove this field from nvme_tcp_pdu
structure, now nvme_tcp_build_iovs builds iov which fully
describes the PDU. Update UT accordingly.
Field padding_valid_bytes is not used at all, delete it too
Change-Id: I2d6040ae64d6847cb455f59f65ec5677de8e5192
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483374
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
spdk_put_io_channel() was designed to be called on the same thread
that called spdk_get_io_channel(). spdk_put_io_channel() sends a
message to its own thread, to allow the context to unwind before
releasing the resources. This had the side effect to allow an
incorrect thread to call spdk_put_io_channel(). This patch will fix
that.
Bdevperf tool had a design flaw that needed the side effect, but
it was fixed recently. We do not know if we have any other case.
Hence add assert to spdk_put_io_channel() to find other case.
We found that unit test for blobstore had called
spdk_put_io_channel() and fix it together in this patch.
Besides, correct the comment for spdk_put_io_channel() in
include/spdk/thread.h not to create any other case in future.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I6ec7bf074818abef43b23ca40bc9385adac70a75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479390
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Allow for using appends instead of writes.
Change-Id: I2f0d3bcdbb0eee034f7b0b6349de854ddbf7273d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481839
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
bdev_nvme_detach_controller call should be used instead of
delete_bdev_controller
Change-Id: Iaa8bb2580d108eaf28c64a9fa7e352991cf2608d
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481837
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
ftl_bdev_create call should be used instead construct_ftl_bdev
Change-Id: If361ca6be8c5118dd55579e8c0a219041b7f1e94
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481811
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
This internal interface allows to create nvmf ctrlr and connect io
qpairs on add listener rpc request (i.e. when subsystem is stopped
and listener is not yet on subsystem's list).
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I998cb72ed773094faacc6668cf069ba9e2a6bf50
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481409
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Had to remove one part of a unit test because the null
checking is moved to a different function.
Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I0a95d0a9a9a5708416fdc7efefb36e17b1ffe010
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/480008
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
When attempting to decode a base64 string, while there is a way to
calculate the maximum possible decode length, there isn't a way to
calculate the exact decode length without duplicating some base64
specific logic located in spdk_base64_decode. With this change, the
spdk_base64_decode function can now optionally calculate the exact
decode length without actually performing the decode by passing NULL
in as the dst argument.
Change-Id: Ice83db979f86a6fe9f39d236d3083102ca37ec68
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The actual check was ORing the execution of mount instead of sending
its stdout to grep - in case mount returned with != 0, this would
block execution of the entire script as grep would wait for input on
its stdin still attached to a terminal.
Also, quote $mount_dir to avoid potential word splitting.
Change-Id: Iea531eef0af79a3742e0a954ea6188e304d0d14a
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Enable extdebug and try to include all the arguments passed down
the function stack in the backtrace.
Change-Id: I81381c936b0f895f1ca8e31d57ef8116d737c6cd
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
In case cwd is changed during the execution of given BASH_SOURCE, i.e.,
when the dirstack is mangled by calls to cd, the actual executable may
end up missing from the path when run directly from its directory in
the ./ fashion. Example:
[root@fedora31 fuzz]# ./autofuzz.sh --module=vhost --transport=all
autofuzz.sh cds into the $rootdir hence the BASH_SOURCE[i] in form of
./autofuzz.sh won't be found there, thus during a failure, since run
under a debug tracer, nl will fail with -ENOENT while trying to read
it.
To mitigate, check if $src is available for reading, if not, log that
the backtrace is not available.
Change-Id: I68988350ba36cca8464bdfac437f662ed4c30f67
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482694
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
It's good to have a human readable domain again.
Change-Id: If773e28a267b635a94a45b07fba9a32d62d0c248
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Github issue 1165 details some issues we have with soft-roce and these
tests. Right now we are disabling them for build stability.
Change-Id: I3a9e28ff3cc1c6ac7d9aa91d93541e295514bb7b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483300
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.
With this patch UT are ran with and without
extent table.
Changed two asserts in test, since amount is dependent on
which type of serialization is used.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica58fce6a4effd014d7dd40ee26edd0fa3196d0f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
This will be used to add another run of whole UT suite
with extent pages on/off.
Next patch in series will be enabling both types of
extent serialization for all UT.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia8b4b8822edefb90ffc13cf777885f9af95e4545
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
All the non-ext version of the call is doing, is calling
ext with NULL as opts. Then default opts are used in
its place.
This change was facilitated by next on in series,
where all blob opts will be initalized in UT
with parameter use_extent_table either set to
true or false.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I62b642c1808b38a5f7c94a5900f25f4978a4ec39
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
The network operations are now asynchronous, so wait for the kernel
to stop using the NVMe partition after unmounting the filesystem.
The kernel is presumably checking for partition tables or unmapping.
Change-Id: Ibefe8e072823a230a896ecfd0adcd9d5fff2723f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482926
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
A pointer to a stack variable is passed as an argument to
nvme_completion_poll_cb function, later this variable is used
to track completion in the spdk_nvme_wait_for_completion() function.
If normal scenario a request submitted to the admin queue will be completed
within the function which submitted the request.
spdk_nvme_wait_for_completion() calls nvme_transport_qpair_process_completions
which may return an error to the caller, the caller may exit from the
function which submitted the request and the pointer to the stack variable
will no longer be valid. Thereby the request may not be completed at that time
and completed later (e.g. when the controller/qpair are destroyed)
and that will lead to call to nvme_completion_poll_cb with the pointer
to invalid stack variable.
Fix - Dynamically allocate status structure to track the completion;
Add a new field to nvme_completion_poll_status structure to track status
objects that need to be freed in a completion callback
Fixes#1125
Change-Id: Ie0cd8316e1284d42a67439b056c48ab89f23e0d0
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481530
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
After creation of blobs in both tests, only clusters indexed from
0 to 10 are supposed to be used. Index 0 for md and 1-10 for data
of single blob since it was create thick provisoned.
Cluster allocations are done in order so if there was a bug for
overflow amount of clusters claimed, first in order would be
one with index 11.
This patch adds asserts after each bs load for first data cluster
that is supposed to be used and for first data cluster that is
not.
During the tests those should remain constant.
When creating/deleting snapshots, the blobs are affected
by changing their type to/from thin_provisioned.
Added asserts to verify their state at every blob open.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I38418da55850d5b8468e578b3c42c5b817ae8045
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482661
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
g_bserrno from blob deletion or snapshot creation,
should not be checked. It is implementation
dependent whether the error (or success) from those
calls actually means that enough data was persisted
on disk.
This test case should work even if we set the threshold
high enough that no failed opperations occur.
On the other hand some parts of those calls do cleanup
in them, meanwhile there is enough metadata data on disk already.
Such as cleaning up unused clusters or pages issue
writes, but at that point the blobs already are in expected
state.
Thus removed assert for g_bserrno, as failure is not indicative
of impossibility to recover.
While here, removed the spdk_bs_unload(). This UT are for
testing power fail safety. Never should it be the case that
enough writes occured in create/delete, but blobs are not
in the expected state.
When such bug would be introduced, it could be covered up
by spdk_bs_unload() cleanly closing up the blobstore.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic69c3061f2cc1fe04bf895632cdb11efb2fe6912
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482660
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Extent Pages claim and insertion can be asynchronous
when cluster allocation happens due to writing to a new cluster.
In such case lowest free cluster and lowest free md page
is claimed, and message is passed to md_thread.
Where inserting both into the arrays and md_sycn happens.
This patch adds parameters to pass the Extent Page offset
in such case.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I46d8ace9cd5abc0bfe48174c2f2ec218145b9c75
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479849
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
If available, automatically use MSG_ZEROCOPY when sending on sockets.
Storage workloads contain sufficient data transfer sizes that this is
always a performance improvement, regardless of workload.
Change-Id: I14429d78c22ad3bc036aec13c9fce6453e899c92
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471752
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
With our target design, there's no advantage to sending
multiple R2T PDUs per nvme command. This patch starts by
setting up the math so that at most 1 R2T PDU is required
per request. This can be guaranteed because the maximum
data transfer size (MDTS) is pre-negotiated in NVMe-oF
to a reasonable size at start up.
It then proceeds to simplify all of the logic around mapping
requests to PDUs. It turns out that the mapping is now always
1:1. There are two additional cases where there is no request
object at all but a PDU is still needed - the connection response
and termination request. Put an extra PDU on the queue object
for that purpose.
This is a major simplification.
Change-Id: I8d41f9bf95e70c354ece8fb786793624bec757ea
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479905
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
We can always accept up to the maximum I/O size in an H2C,
so eliminate the #define.
Change-Id: I349dab5f9b6ec482a7c580b1396e03c8d30a250b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482278
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The resources allocated to a queue pair do not need to be directly
correlated to the queue size requested by the initiator in NVMe-oF, as
long as enough resources are present. The RDMA transport, for instance,
does complex pooling of the resources behind the scenes when using a
shared receive queue.
Simplify the resource allocation for a TCP qpair to just always allocate
the max allowed queue size right away. This is a configurable parameter,
so system administrators can adjust for their needs. The initiator may
then request a queue size less than or equal to that, which will only be
enforced by queue depth counting and not impact the actual number of
resources allocated on the target.
This change relies on the MaxC2HSize being equal to the Maximum Data
Transfer Size (MDTS) reported. That is the default configuration, but
MDTS is configurable. Changing the MDTS with this patch to a value
larger than 128k will cause the target to break. This is addressed in
the next patch in this series.
Change-Id: Ibd4723785c6a4d8d444f9b7bbfa89f98de2320f5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479733
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
This patch adds test for compare-and-write fused
command. First it runs successful call of compare
and write command and then it runs again and the
second run is expected to fail.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iaf1151414c4dea16487e8c33630cbffb0c09ae3b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482606
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Add call for spdk_nvmf_bdev_ctrlr_compare_and_write_cmd
function in spdk_nvmf_ctrlr_process_io_cmd function
when fused command is discovered.
This patch also removes redundant defines for fused flags.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I61971a56577ab32b52e1fde1e572f718a9a2d9aa
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476621
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This patch introduces new spdk_nvmf_bdev_ctrlr_compare_cmd
function which implements support for compare operation.
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Iadf402a6441a78ea0e6468f1066c6b0e10e63b9b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477782
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Use provided configuration file via --disk-map option
instead of creating bdevs and VMs in ordered sequence
(e.g. 0 to 10, etc.).
This allows to:
- specify which NVMe device we want to use
(PCI BDF identifier)
- how to name it in SPDK nvme bdev configuration
- how many splits or lvol bdevs to create on this device
- which VMs should use created bdevs
With CPU mask configuration file this allows to better
control resources when running the test (especially in
case of NUMA optimization where using sequential for/while
loops is not a good approach).
vm_count and max_disks parameters removed. These are not
needed anymore as they're controlled by config file.
Example of config file contents:
(BDF,Spdk NvmeBdev name,Split count,VM list)
0000:1b:00.0,Nvme1,2,2 3
0000:89:00.0,Nvme3,4,4 5 6 7
Change-Id: I9fc73458825d8072537aa04880765a048e034ce4
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464565
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This test essentially confirms that the new RPC properly creates a test
file and that the script can parse that file and run through a basic set
of operations and exit without an error.
Change-Id: Idf0c831020696a3a62fcef13171eedf3fcf63f5b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
This reverts commit 4700ef0fa6.
This has merge conflicts with the iSCSI async write patch
series that was merged.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5a27460a369ef5f13bf490a287603e566071be40
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482482
