New IO types are just ignored by existing modules, but when
added it changes the value of SPDK_BDEV_NUM_IO_TYPES which
then throws an abidiff error in our build.
So just suppress changes to this specific value so that
adding new IO types doesn't itself result in an SO version
bump.
Thanks Alexey Marchuk for pointing out this abidiff
feature.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib023bd5ee30a7ebcf7da0bb4b3f8dc54f5713710
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/5206
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Our check_so_deps.sh test script removes the spdk.lib_deps.mk
include from spdk.lib.mk so that we can check which symbols
the libraries depend on. But modifying the code like this
is a bit kludgy. So instead add a Makefile variable that
check_so_deps.sh can set to get the same behavior.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I5622f4c3adb2d5ccd5ae33cb4cd116716134a9b7
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: <dongx.yi@intel.com>
spdk_bdev_create_bs_dev_ext() gets not bdev pointer but bdev name
as an argument, and hence vbdev_lvs_create() will get bdev name
accordingly.
However after completing spdk_bdev_create_bs_dev_ext(),
vbdev_lvs_create() has to get bdev pointer from the created bs_dev.
Hence add a function pointer get_base_bdev to struct spdk_bs_dev
and set it to bdev_blob_get_base_bdev() at initialization.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idef0663ace85db0269442212014286669c150069
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4706
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
The internal vhost library is used when DPDK's version
is older than 19.05 and the experiemntal vhost nvme
target.
For the CONFIG_INTERNAL_VHOST_LIB option, SPDK doesn't
enable this option by default over one year and CI
doesn't cover it either, so we may remove it with
this release.
As for the vhost-nvme target, since we are developing
a new vfio-user target solution, it's OK for us to remove
it now.
Change-Id: Ib2cce1db99cd09754307c2828b3187f2d4550304
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4562
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Instead of iterating over each symbol, create a regex group including
all the symbols to lookup and pass it to grep.
Also, refactor some small pieces:
- filter event_ dependencies within import_libs_deps_mk()
- convert SPDK_LIBS to an array built up with extglob
- drop DEP_LIBS
- drop missing_syms (unused)
The overall improvement (this covers the execution of the subshell
within which confirm_deps() is called for each .so):
real 1m23.688s
user 4m16.196s
sys 8m3.110s
real 0m3.537s
user 0m13.450s
sys 0m18.486s
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: Id61c5f947cd116e0fa0107cc7c941beb2c0bdcb4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4632
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Also, move the check against IGNORED_LIBS to import_libs_deps_mk().
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Change-Id: I190262f728118d65c35481e490ca094086760cfa
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4631
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This function was recursively greping through the .mk file to map all
ref variables $(...) to values they are set to. This was including
plenty of duplicate entries which then had to be sorted|uniqed.
Instead, and to avoid recurssion, import the entire .mk file into
Bash's environment. The mapping would look like so:
JSON_LIBS := json jsonrpc rpc
DEPDIRS-event_vmd := event vmd conf $(JSON_LIBS) log thread
|
v
JSON_LIBS="json jsonrpc rpc"
JSON_LIBS() { echo $JSON_LIBS ; }
event_vmd="event vmd conf $(JSON_LIBS) log thread"
|
v
event_vmd="event vmd conf json jsonrpc rpc log thread"
Change-Id: Ibfcd52438403cd7638e01d2d5642c08822f85106
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4629
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Newer versions of readelf (e.g. from binutils v2.35) break the lines
in the following manner:
.... GLOBAL DEFAULT 14 restore_funcarra[...]
The symbol name is shortened, however, the way how it's indicated
with "[...]" may mess with any tools that look up this field with a
regex pattern - just like grep in the confirm_deps() test.
Currently, this test fails on clearlinux where the latest readelf
is shipped on board. To avoid it, make sure readelf returns full
name of the symbol with the --wide output.
Change-Id: Ief1a3dccde5481f603302ee714021dcebc20fc58
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4627
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
This will make it easier to know what specifically changed
that requires the version bump. Use --impacted-interfaces
argument to add further verbosity.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I547d29f65323683c4096b738514ed4ac85ecd4d4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
This will allows us to reuse most of the command
line in a future patch.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I039d628a0a1b6612c1e4d66395651cb0aa2255a4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/4447
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Those modules are now enabled and tested on CI.
ABI reference build was updated to include those components.
Workarounds for blobfs_bdev can now be removed too.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4ef321dcb0ce80b13e856a1ca97ed072f7b5ddc4
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3747
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
This was missed in CI, due to disabled SPDK_TEST_BLOBFS
test flag in UT job that performs check_so_deps.sh.
Error seen when it is enabled:
there was a dependency mismatch in the library blobfs_bdev
The makefile lists: 'bdev blob_bdev blobfs json jsonrpc log rpc thread util'
readelf outputs : 'bdev blob_bdev blobfs event json jsonrpc log rpc thread util'
Temporary workarounds have been added, ignoring blobfs_dev
in the abi check test. This will allow for better transition
on CI side. After this patch is merged, UT job flags on CI will be
updated to include SPDK_TEST_BLOBFS and abi reference repository
will be recompiled with this flag.
Next patch in series removes this workaround. It will be merged
after work on CI side is done.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I4753a918d5760f154d4a59349747a0b1356e9c91
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3961
Community-CI: Broadcom CI
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Similar to internal vhost library, rdma lib should be ignored
if SPDK was not compiled using --with-rdma.
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic747f96f96182c04592bec44ff64ceafd11634f0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3963
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
These suppressions are only valid for the course
of one release. Now that we have started with a
new release, remove all of the suppressions.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1a36bc49bb3a16f98de870cc06e56dbfa75d72d6
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3722
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Update reference build path to latest build.
Change path to use a generic directory name instead
of using specific SPDK version in the name. That way
in future we can update the build just after the
release, and still make ABI checks for patches which
were not rebased.
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Change-Id: I32850a871274773e334057b60b9df238ef85a999
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Mellanox Build Bot
You don't get notified when someone starts using your hooked
device, so there's not much gain from knowing when someone
stops.
Remove that callback and also move DPDK device detach under
the same lock which sets the pending_removal flag. This eliminates
a data race window when hotremove notification could arrive
after device was detached, but before it was scheduled to be
removed.
vmd and ioat nest the spdk_pci_device struct and abigail complains
even though the parent structs only have forward declarations in
public headers. Adding those two structs to the suppression list
doesn't help though. Abidiff still complains about the pci device
struct being changed, probably because ioat.h and vmd.h both include
env.h. Abidiff suppresion list should eventually be split per-lib,
but for now ignore struct spdk_pci_device changes globally.
$ abidiff [...]/libspdk_ioat.so [...]
'struct spdk_pci_device at env.h:652:1' changed:
type size changed from 1024 to 960 (in bits)
1 data member deletion:
<SNIP>
Change-Id: I9b113572c661f0e0786b6d625e16dc07fe77e778
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2939
Community-CI: Mellanox Build Bot
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
The previous revision to output parsing could
cause issues when more than one set of parenthesis
was included in the output.
fixes: 8661ded761
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I1cbfac1186fb5ee97518f4b687f38af59c0dcfb9
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/3092
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
Move all operations on "integers" to proper arithmetic evaluation
constructs. This is to avoid the following seen in the build logs:
check_so_deps.sh: line 353: [: : integer expression expected
check_so_deps.sh: line 353: [: : integer expression expected
This can be seen whenever an object file is missing maj.min numbers
from its name.
Additionally, shuffle the code around to simplify parsing of the
abidiff output.
Change-Id: I8f4aef10f11382788533aa2d92e9ffdcd6a72e4b
Signed-off-by: Michal Berger <michalx.berger@intel.com>
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2743
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Mellanox Build Bot
Add spdk_bdev_abort function as a new public API.
This goes all the way down to the bdev driver module and attempts to
abort all I/Os which has bio_cb_arg as its callback argument.
We can separate when only a single I/O has bio_cb_arg and when multiple
I/Os have bio_cb_arg, but unify both by using parent - children I/O
relationship. To avoid confusion, return matched_ios by _bdev_abort() and
store it into split_outstanding by the caller.
Exclude any I/O submitted after this abort command because the same cb_arg
may be used by all I/Os and abort may never complete.
bdev_io needs to have both bio_cb_arg and bio_to_abort because bio_cb_arg
is used to continue abort processing when it is stopped due to the capacity
of bdev_io pool, and bio_to_abort is used to pass it to the underlying
bdev module at submission. Parent I/O is not submitted directly, and is
only used in the generic bdev layer, and parent I/O's bdev_io uses bio_cb_arg.
Hence add bio_cb_arg to bdev structure and add bio_to_abort to abort structure.
In the meantime of abort operation, target I/Os may be completed. Hence
check if the target I/O still exists at completion, and set the completion
status to false only if it still exists.
Upon completion of this, i.e., this returned zero, the status
SPDK_BDEV_IO_STATUS_SUCCESS indicates all I/Os were successfully aborted,
or the status SPDK_BDEV_IO_STATUS_FAILED indicates any I/O was failed to
abort by any reason.
spdk_bdev_abort() does not support aborting abort or reset request
due to the complexity for now.
Following patches will support I/O split case.
Add unit tests together to cover the basic paths.
Besides, ABI compatibility check required us to bump up SO version of
a few libraries or modules. Bump up SO version of blob bdev module simply
because it does not have any out-of-tree consumer, and suppress bumping
up SO version of lvol library because the affected struct spdk_lvol
is not part of public APIs.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I515da688503557615d491bf0bfb36322ce37df08
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
This type was changed. We can avoid updating the major
version of libspdk_sock though because this type is in
the spdk_internal sock header.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ib0d6cc0d99f97a34f618f776ac28e15e0a750356
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2741
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
We don't want to rev this more than once in a release cycle. The idea
behind that is that nobody should be making shared objects from a commit
between two releases so we only need to rev once per release cycle.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I8162683152bf86f41061309d341a54959e1c3646
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2682
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Mellanox Build Bot
This is an important aspect of ABI versioning.
There was a change to the spdk_bdev_opts struct
which is accessible from the public API that results
in an ABI change.
There was also a change in the spdk_accel_module_if struct that
will affect new modules so its major version needs to be revved.
There was also a change in the publically accessible rmda_hooks
structure which forced an ABI change in the NVMe library.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I5cc6886fe01b4adc2836b6e15995471f0361dc29
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2663
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
This was overlooked when the module was merged.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ia7bd8710741bf5a82e7dd27925ecef1e59ed2938
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2360
Community-CI: Mellanox Build Bot
Community-CI: Broadcom CI
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Aleksey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Allowing for custom crc32 polynomials isn't really within the scope
of SPDK. SPDK libraries have shifted over to using more optimized
versions of these functions wherever possible (e.g. ISA-L).
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Ie4837010874e957481dd00896389b38c842445e0
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2424
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
The log_rpc library should have its own flag.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: I7cfe4c9136089e83bdf2730c727a6602ffbaf83e
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2215
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
There are several functions in the internal header that
are only used in the library that have kept the spdk prefix.
Add those to the suppression file since nobody in practice will
be using them.
Signed-off-by: Seth Howell <seth.howell@intel.com>
Change-Id: Iae1666c6f0bb853e62b89858037a5cded38c9b66
Reviewed-on: https://review.spdk.io/gerrit/c/spdk/spdk/+/2213
Community-CI: Mellanox Build Bot
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
