11141 Commits

Author SHA1 Message Date
Wojciech Malikowski
4d122a3559 lib/ftl: Add support for spdk_bdev_open_ext()
In order to handle media management events spdk_bdev_open_ext()
should be used instead spdk_bdev_open(). Move this call to ftl lib
to keep media management events internal to the library.

Change-Id: If4c9382cc89fc537667923f00d3dae5df0ace248
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481503
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2020-02-04 18:10:01 +00:00
paul luse
6ea187fcbb scripts/vagrant: update readme
Fix a bunch of typos, remove a few things and add some disclaimers
for legal cleanliness :)

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I366380f7b822c5b7a5ca92ab5a9bd3fd68296bee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483752
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 17:04:12 +00:00
Konrad Sztyber
ce6171d465 lib/vmd: detach devices during shutdown
Added spdk_vmd_fini(), which detaches all PCI devices acquired by the
VMD subsystem.

Fixes #1148

Change-Id: I43218ef5f9a764546b655c28688897fb91b779cb
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482852
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-02-04 16:50:25 +00:00
Jacek Kalwas
aaa9630aeb nvmf/ctrlr: introduce request exec backdoor
It allows to property set (e.g. cc) when subsystem and qpair are not
active.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I0b0d150fbdac5bdf0d20762337f0a811f4d6d243
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481494
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 15:16:12 +00:00
Jacek Kalwas
fcca8ea6a9 nvmf/ctrlr: introduce ctrlr connect backdoor
This internal interface allows to create nvmf ctrlr and connect io
qpairs on add listener rpc request (i.e. when subsystem is stopped
and listener is not yet on subsystem's list).

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I998cb72ed773094faacc6668cf069ba9e2a6bf50
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481409
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-02-04 15:16:12 +00:00
Jacek Kalwas
489815dcf5 nvmf/ctrlr: introduce utils used during ctrlr connection
Had to remove one part of a unit test because the null
checking is moved to a different function.

Signed-off-by: Jacek Kalwas <jacek.kalwas@intel.com>
Change-Id: I0a95d0a9a9a5708416fdc7efefb36e17b1ffe010
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/480008
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-04 15:16:12 +00:00
Tomasz Zawadzki
3f5728fe83 example/fio: add option to load json_config
Added new configuration option to bdev fio_plugin
called 'spdk_json_conf'. When provided the SPDK
subsystems are loaded using this JSON configuration.

Only single type of configuration can be provided at
a single time.

While here, added bdev_rpc in Makefile to support all bdev
related RPC that can now be issued via json config.

Change-Id: I5f45fe2d8331034ef4becca43bbaedebd6b20c5a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463979
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-02-04 15:15:46 +00:00
Tomasz Kulasek
1a30ba7f25 bdev/nvme: fix release reference to channel in apply firmware RPC
Fixes #1176

Change-Id: Ie694c9cd492fec5486683e6cb04dc11892e0a493
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-04 14:53:17 +00:00
Alexey Marchuk
3424def90a nvme: Fix potential use of non-initialized variable
trstring variable in spdk_nvme_trid_populate_transport is not
initialized, that can lead to snprintf() writes some garbage to
trid->trstring if the user passes SPDK_NVME_TRANSPORT_CUSTOM trtype
Add return statement and assert to CUSTOM/default switch

Change-Id: I6c6c37f9aa74d61b346f7be27fb890c7a34e9229
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483469
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-02-04 09:49:25 +00:00
paul luse
03e96be67d examples/bdev: add bdev_rpc to Makefile for hello_world
Fixes issue #1178

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2bf53e398e10f1a8c4b4418fbf8f940f5861d7bf
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483562
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 12:47:14 +00:00
paul luse
593cb5b302 module/crypto: move some defines to header file
In prep for adding QAT XTS support in upcoming patches

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I2c14101d31d19f31527423efea24c0d87079baf4
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483531
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 12:46:59 +00:00
paul luse
946853736a module/crypto: fix one missing free() in error path
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I28b6c149ea341bbacf0d884c1bb0a7287076c9c9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483530
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 12:46:59 +00:00
paul luse
1880df4daf module/crypto: fix comment block for create_crypto_disk()
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ibcf49c84bd23fb8c8de631e79fea15d0f367b204
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483528
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 12:46:59 +00:00
paul luse
b8a618db14 module/crypto: use the correct error code for rpc decode failure
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iabea7dfeaae7a57c8cbc2781882ee9e1600058a2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483527
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 12:46:59 +00:00
paul luse
17f7cf9b29 module/raid: fix typos in comments
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iafacd934a04c0f162f38b80a03e278c97ff4d5d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-03 12:46:22 +00:00
Mike Carlin
06fc4cadbe util/base64: Extend b64 decode to calculate exact len
When attempting to decode a base64 string, while there is a way to
calculate the maximum possible decode length, there isn't a way to
calculate the exact decode length without duplicating some base64
specific logic located in spdk_base64_decode. With this change, the
spdk_base64_decode function can now optionally calculate the exact
decode length without actually performing the decode by passing NULL
in as the dst argument.

Change-Id: Ice83db979f86a6fe9f39d236d3083102ca37ec68
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/479479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-02-03 11:39:49 +00:00
Michal Berger
1c06235ab3 test/blobfs: Fix typo in the mount check
The actual check was ORing the execution of mount instead of sending
its stdout to grep - in case mount returned with != 0, this would
block execution of the entire script as grep would wait for input on
its stdin still attached to a terminal.

Also, quote $mount_dir to avoid potential word splitting.

Change-Id: Iea531eef0af79a3742e0a954ea6188e304d0d14a
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483433
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-02-03 11:39:18 +00:00
Tomasz Kulasek
e69baea132 lib/nvme: fix wrong status argument in spdk_nvme_ctrlr_format
For nvme_ctrlr_cmd_format command status should be used as
nvme_completion_poll_cb callback argument instead of pointer to
local variable.

Change-Id: Id65cb395d137c4e907c1ef019b131e8822ddfe34
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483513
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-02-03 11:39:05 +00:00
Michal Berger
18b9303dfb test/common: Include BASH_ARGV[]s in the backtrace
Enable extdebug and try to include all the arguments passed down
the function stack in the backtrace.

Change-Id: I81381c936b0f895f1ca8e31d57ef8116d737c6cd
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482695
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-31 12:39:22 +00:00
Michal Berger
1baf379e69 test/common: Don't attempt to read backtrace from unavailable source
In case cwd is changed during the execution of given BASH_SOURCE, i.e.,
when the dirstack is mangled by calls to cd, the actual executable may
end up missing from the path when run directly from its directory in
the ./ fashion. Example:

  [root@fedora31 fuzz]# ./autofuzz.sh --module=vhost --transport=all

autofuzz.sh cds into the $rootdir hence the BASH_SOURCE[i] in form of
./autofuzz.sh won't be found there, thus during a failure, since run
under a debug tracer, nl will fail with -ENOENT while trying to read
it.

To mitigate, check if $src is available for reading, if not, log that
the backtrace is not available.

Change-Id: I68988350ba36cca8464bdfac437f662ed4c30f67
Signed-off-by: Michal Berger <michallinuxstuff@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482694
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2020-01-31 12:39:22 +00:00
Ben Walker
f54757cc14 test: Add some unit test output to .gitignore
Change-Id: I6e9b50a833737aa2ff6ea16e38d6dcf9b1d4fa33
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482279
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-31 12:37:42 +00:00
paul luse
e914958951 module/compress: rename set_compress_pmd RPC to match convention
set_compress_pmd -> compress_set_pmd

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Icb1ecc7adfe10485c44f98ab9e31eaa6857596e7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482597
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:37:31 +00:00
Darek Stojaczyk
15d92f6a12 examples/hello_bdev: allow running without ini config file
Remove the config file check explicitly preventing hello_bdev
from starting and also update the docs not to reference config
files at all.

Change-Id: Ieefb629ef16d75a43c2c1fad2ac442f6e37cc16d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483386
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-31 12:37:12 +00:00
Darek Stojaczyk
70597c0a14 event: mention --wait-for-rpc in spdk_app_start()
spdk_app_start() can have a completely different behavior
if called with opts->delay_subsystem_init (set via
--wait-for-rpc) - document it.

Change-Id: Id28e1e0cad4681fc9ee29359a7f0e02eae58b434
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483385
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2020-01-31 12:37:12 +00:00
Charles Machalow
cf8b06ef66 Docstrings: Fix docstring for spdk_nvme_ctrlr_alloc_io_qpair
If NULL is passed as opts to spdk_nvme_ctrlr_alloc_io_qpair,
the defaults will be obtained from spdk_nvme_ctrlr_get_default_io_qpair_opts
instead of spdk_nvme_ctrlr_alloc_io_qpair.

Fixes issue #1160

Signed-off-by: Charles Machalow <csm10495@gmail.com>
Change-Id: I88d4399bc0872e73bc1e04b3f09834afd7d56c53
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483306
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:55 +00:00
Karol Latecki
ac26fec9c6 autorun_post: skip confirming executed tests
Allow to skip confirmPerPatchTests if needed.

Change-Id: I8741d80de5cac9954e3429b951a71dc065c40bb5
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483016
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:29 +00:00
Seth Howell
f257be1313 doc: replace all cloudfront refs with ci.spdk.io
It's good to have a human readable domain again.

Change-Id: If773e28a267b635a94a45b07fba9a32d62d0c248
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483146
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-31 12:36:19 +00:00
Tomasz Zawadzki
7c6b574722 version: 20.04 pre
Change-Id: I43ac544a5b2da45e2c894c0854f63709d3d529e3
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483248
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-31 12:35:05 +00:00
Ziye Yang
08249f00b5 sock/posix: Change the return type of function _sock_check_zcopy
Purpose: The function spdk_sock_request_put may
return an error code, and close the socket, so we should change the
return type of  _sock_check_zcopy.

If the return value of  _sock_check_zcopy is not zero,
we should not handle the EPOLLIN event.

Fixes #1169

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: Ie6fbd7ebff54749da8fa48836cc631eea09c4ab8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483311
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-31 09:29:07 +00:00
Tomasz Zawadzki
29bd502046 lib/blob: add invalid flag for extent table
With recent changes to extent on-disk metadata format,
new format (Extent Pages) is not backwards compatible.
Meanwhile old format (Extent RLE) is backwards
compatible with older SPDK applications.

Summing up:
Blobstore created pre SPDK 20.01 can only use Extent RLE.
Blobstore created starting with SPDK 20.01 can use both,
Extent Pages and Extent RLE specified by use_extent_table opts.

When use_extent_table is set to true, invalid flag for it is set.
SPDK application pre 20.01, will not load such blob.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: If14ebd03f19eb581d71dcb46191e099336655189
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483220
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2020-01-31 09:28:56 +00:00
Seth Howell
e0f63b969d test/nvmf: disable bdevperf tests on soft-roce
Github issue 1165 details some issues we have with soft-roce and these
tests. Right now we are disabling them for build stability.

Change-Id: I3a9e28ff3cc1c6ac7d9aa91d93541e295514bb7b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483300
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-30 16:52:10 +00:00
Seth Howell
8b47b31a17 test/unit: clean up conditional checks and duplicate tests
Change-Id: Ibd99f40f19e73b119ff7de434a4ddf9a29ed3191
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482444
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-30 16:52:10 +00:00
Tomasz Zawadzki
353252b1b4 lib/blob: document use_extent_table
Patch adds documentation and CHANGELOG update
for newly added Extent Table/Page path.

Change-Id: I86f6c5680084a92d50bd9ca39b68d68a9908ecf8
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483247
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-30 13:19:45 +00:00
Ben Walker
d0f4a51fdc sock/posix: Block recursive calls to spdk_sock_flush
Don't allow calling spdk_sock_flush while the socket is
closed.

Change-Id: I9020a49ab8906b0f343e3f48f8b96bd38308ab17
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483148
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-30 10:22:20 +00:00
Seth Howell
967fa2d707 CHANGELOG: Alphabetize the 20.01 changelog sections
We should probably be consistent about this going forward.

Change-Id: I6893ac991a0e506edad737db72986d82d6f1734e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482911
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-29 14:16:27 +00:00
Seth Howell
64021521f7 CHANGELOG: update changelog for the 20.01 release.
Change-Id: Ie1760d1d65d8f8266c80327c853720f4299594ce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482448
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-29 14:16:27 +00:00
Seth Howell
f4a63bb8b3 env_dpdk: keep a memmap refcount of physical addresses
This allows us to avoid trying to map the same physical address to the
IOMMU in physical mode while still making sure that we don't
accidentally unmap that physical address before we are done referencing
it.

Change-Id: I947408411538b921bdc5a89ce8d5e40fd826e971
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483133
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2020-01-29 14:15:21 +00:00
Seth Howell
f0ca01e102 lib/nvmf: properly validate fuse command fields.
The fuse command value is a two byte value, but we were only checking to
see if the fuse value was equal to SPDK_NVME_CMD_FUSE_FIRST or
SPDK_NVME_CMD_FUSE_SECOND in spdk_nvmf_ctrlr_process_io_fused_cmd. If a
haywire initiator sent a command with a fused value equal to
SPDK_NVME_CMD_FUSE_MASK, that would result in us skipping all checks and
dereferencing a null pointer in
spdk_nvmf_bdev_ctrlr_compare_and_write_cmd.

To fix this, add an extra condition to validate the cuse field.

Change-Id: I1ec4169ff5637562effd694f7046c6e3389627f1
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483123
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2020-01-29 14:15:00 +00:00
Tomasz Kulasek
8b74c02390 bdev/nvme: fix crash when invoking apply_firmware RPC
When structure for output of json decoders in not initialized
spdk_json_decode_string may fail trying to free uninitialized
string.

This patch changes mallocs used to allocate context and structure
for output of decoder with calloc.

Fixes #1151

Change-Id: I180b2ec52350b4ca90e7c318b4f2d13af554ec49
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/483107
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
v20.01-rc1
2020-01-28 15:41:26 +00:00
Tomasz Zawadzki
bba0f040cf lib/blob: fix uninitialized variable for non-debug builds
This was observed after running nighly tests on previous patch.
As part of it, autopackage.sh compiles SPDK
without debug flag set. Exposing the uninitialized var here.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Iedb1641f3c0d4a21f293c81cd4fcf35c6d1c7ae5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482893
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
12d1404125 lib/blob: set default use_extent_table to true
Extent table and extent page descriptors are now
set to be default way clusters are serialized on disk.

With this patch UT are ran with and without
extent table.

Changed two asserts in test, since amount is dependent on
which type of serialization is used.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ica58fce6a4effd014d7dd40ee26edd0fa3196d0f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/481901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
954cb9cd55 ut/blob: redefine spdk_blob_opts_init()
This will be used to add another run of whole UT suite
with extent pages on/off.

Next patch in series will be enabling both types of
extent serialization for all UT.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ia8b4b8822edefb90ffc13cf777885f9af95e4545
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482170
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
6a5bb712b4 ut/blob: replace all blob creation with extended version
All the non-ext version of the call is doing, is calling
ext with NULL as opts. Then default opts are used in
its place.

This change was facilitated by next on in series,
where all blob opts will be initalized in UT
with parameter use_extent_table either set to
true or false.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I62b642c1808b38a5f7c94a5900f25f4978a4ec39
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
5d5053373f lib/blob: fix inserting extent pages
ctx->extent_page signifies if page was allocated
for insertion.

1) It is possible for a thread to claim extent page
on its own thread, and put it in ctx->extent_page.
If conflicting thread allocates another ctx->extent_page,
then it should be freed. This does not mean failure
to insert cluster. As different threads could have
been trying to allocate different clusters,
so condition on line 6716 does not cover it.
If so then it shouldn't be an issue to release
the claimed ctx->extent_page and proceed with updating the
extent page which originally won the race.
NOTE: if clusters were conflicting, then extent_page is
freed in _spdk_blob_insert_cluster_cpl().

2) At this point of _spdk_blob_insert_cluster_msg()
we already verified that there already is
extent page allocated at "*extent_page".
In such case ctx->extent_page will be 0,
and should not be used.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Id5b57c88248890eee60d2e7dbecbd984c98b561b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
1dd8c57ab8 lib/blob: fix clearing cluser array on EP load
sz is set to number of clusters that should be have been
in particular unallocated EP (remaining_clusters_in_et
up to SPDK_EXTENTS_PER_EP).

The cluster array should be set to 0 only in region
between original size (cluster_array_size) and new
total size (active.num_clusters).
It was incorrectly using sz in the memset.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ic43e89c17d53e9529e3ed0349aeb4fb7dc6593f2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482858
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
8b6df6fbab lib/blob: count clusters starting from 0 when parsing EP
Previously part of function assumed that cluster count
1)means number of clusters in EP and another 2) that it is
following the active.num_clusters (akin to extent_rle).

This was incosistent and showed when using multiple
extent pages to serialize metadata.
This patch changes it to only go with 1), so it is clear
that it means number clusters within particular EP.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I155104cabc127ed47df04434032fb01e08948e13
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482848
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
dd0a320cf9 lib/blob: Always persist at least single extent table desc
Following changes are done in this patch:
1) _spdk_blob_serialize_extent_table now persists
at least single extent page. When num_extent_pages == 0.
2) Minimum valid size of ET descriptor is even without
extent_pages. This is a case when there are no EP,
but we still want to persist num_clusters in ET.
3) Taking above points, redone the loop for serializing
extent pages.
4) Make sure to mark blob dirty if any new extent pages
were allocted.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I81dc6cf2de2722bb49927ed42f4b9f31292f78c5
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482847
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
c177a3c841 lib/blob: remove assumption of empty cluster array with multiple EP
Changed assert checking if cluster array is allocated
when loading extent pages. This is true only for
the first extent page being loaded, of course after that
the cluster array can be already allocated.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: I0f96294ede5a12ffd6bca73cbeadba8d94a35bac
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482857
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2020-01-28 09:15:23 +00:00
Tomasz Zawadzki
42109157f4 lib/blob: add starting cluster index to extent page
Size of a blob (thus size of clusters array in mutable data)
is known from extent table descriptor.
Extent pages were read sequentially in order they were
placed in extent table. This meant that cluster
array could have been filled up from beginning to end.
Yet reading extent pages in any other order,
would result in incorrect placement of clusters.

This patch adds first cluster index that is contained within
each extent page. This will allow to read/write
multiple extent pages in parallel, since
we will know where in clusters array to put the cluster idxs.

Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Change-Id: Ib6b9332111cd93f990d057dc60624152907dd87f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482701
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2020-01-28 09:15:23 +00:00
Ben Walker
c85a3d105c test/nvmf: In filesystem.sh, wait for partition to be unused
The network operations are now asynchronous, so wait for the kernel
to stop using the NVMe partition after unmounting the filesystem.
The kernel is presumably checking for partition tables or unmapping.

Change-Id: Ibefe8e072823a230a896ecfd0adcd9d5fff2723f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/482926
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2020-01-28 09:15:23 +00:00