1468 Commits

Author SHA1 Message Date
Darek Stojaczyk
4a713238b7 nvme: fix json response on failed construct_nvme_bdev RPCs
spdk_jsonrpc_send_error_response() is documented as a
shorthand for spdk_jsonrpc_begin_result() + end result,
but we used to call it already after doing
spdk_jsonrpc_begin_result(), which makes the response
completely invalid.

Fix it by deferring the initial spdk_jsonrpc_begin_result()
until after we did our error checking. While here, remove
the NULL checks from that function. After recent jsonrpc
changes it can never return NULL.

Change-Id: Ibb69803d4e6ce6516817673eec6626dcfc4f1abf
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464174
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-09 16:35:00 +00:00
Changpeng Liu
3d12e6d4e6 bdev/nvme: deprecate ResetControllerOnTimeout configuration option
'ResetControllerOnTimeout' was renamed to 'ActionOnTimeout' for a
long time, so we deprecate the support for this old configuration
option.

Change-Id: Ia44035a8749ca046c1df16f82601389b5bafeb76
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-09 00:44:50 +00:00
Changpeng Liu
7cbe1ccd56 nvme: move SPDK_NVME_DEFAULT_RETRY_COUNT out from nvme.h
SPDK_NVME_DEFAULT_RETRY_COUNT is the default value for each controller, so
we can move it out from public header file, and change the value if users
provide a new one.

"NvmeRetryCount" was deprecated for a long time, so we removed the support
for this configuration option as well.

Change-Id: I187251cc1e5342abb4fce96727d06631b7c16a01
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464489
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-09 00:44:50 +00:00
Changpeng Liu
62bb65289d nvme: change retry count can be configured via bdev nvme driver
Also eliminate 'spdk_nvme_retry_count' finally.

Change-Id: I2f3e390e4b8a49208a11b54bb82c4891cf3e1845
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464473
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-09 00:44:50 +00:00
Seth Howell
2cbe2acaef delay_bdev: move delay_type enum to header file.
Change-Id: Iaab3104b87e87d2fef4f4858e6e05d0773c5e453
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/464453
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-08-08 22:57:59 +00:00
Darek Stojaczyk
1aa34d2361 bdev/raid/rpc: try to break lines after 100 characters
The lines in this file are extremely long. astyle doesn't
really allow us to always go under 100, but we can at least
try to do our best.

Change-Id: I36f454b67460cadf92f99578512349d048d4861b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462377
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-08-05 23:12:49 +00:00
paul luse
2b483475d9 bdev: change error to warning
Ran into this while putting the compress tests in CI, an error
message will print following the completion of the app, checked
with Jim and it is actually not an error when lvols are involved
so change to warn now as this will show up in everyone's build
log for bdev testing.

Change-Id: Ib509b89acf5b6bfbeb349692ab59bbea4964557f
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463803
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-08-05 00:24:05 +00:00
paul luse
b64ff135c5 test/compress: add bdevio and bdevperf tests
Will execute short run per patch, longer nightly. This
path only covers the ISAL PMD as there are QAT upgrades
needed in CI. It does so by using the new 'perform_tests'
RPC in both bdevio and bdevperf.

Change-Id: I93afda336d7411b695b2ddd0332cce0050d075a4
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455113
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-30 07:36:03 +00:00
Changpeng Liu
90520634bd bdev/nvme: configure the number of requests allocated for one NVMe I/O queue via RPC
A single I/O may allocate more than one request, since splitting may be
necessary to conform to the device's maximum transfer size, PRP list
compatibility requirements, or driver-assisted striping.  Very big
I/O request sent from application may get error due to limited resources
in NVMe driver layer, so here we add an optional parameter to make the
parameter can be configured by users.

Fix issue #745.

Change-Id: I7824232c54865b052dcd0ec6e91484c3837fc2c4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461182
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-29 22:52:39 +00:00
Richael Zhuang
b2101c9a5c bdev: replace spdk_mb with spdk_smp_mb
For aarch64 spdk_mb is defined as "__asm volatile("dsb sy" ::: "memory")"
and spdk_smp_mb is defined as "__asm volatile("dmb ish" ::: "memory")".
"dsb sy" completes when all instructions before this instruction complete.
And "dmb ish" only keeps the order of memory access instructions before
and after it. It doesn't affect the ordering of any other instructions.
Here I think "dmb ish" is enough.

And replace spdk_mb with spdk_smp_mb here will not affect PPC64, X86_64
and i386.

Signed-off-by: Richael Zhuang <richael.zhuang@arm.com>
Change-Id: Idd65f74567c594e5a6514a4ab0665b8f3606c883
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463455
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-29 20:54:27 +00:00
paul luse
562a93e45b bdev/compress: increase mbuf pool size
Was 1024 but w/the performance optimizations we can now use up
to 4 mbufs per outstanding IO so going to 8192 only uses 1MB of
mem and gives us better head room before we have to start queueing
compression operations to wait for mbufs. Note: the mbuf pool is
shared accross all virtual compression bdevs.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I64d3973e7c7d31925b1ce5028ef960f90331c067
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463538
Reviewed-by: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-29 20:43:08 +00:00
paul luse
5de38f9543 bdev/compress: update some defines for QAT
There's a small drive-by in here also, use the max iovec define
from the reduce header instead of manually making sure its the same
here.

The change in the size of the MBUF pool is to accommodate the new
QAT driver being faster (?) than the old such that we could run out
of MBUFS with a fio test using 6 threads.  On the + side, when we did
the queueing code in the compression operation worked good.

Also related to QAT and experimenting with fio compression ratios, QAT
can fairly often return an error on compression that we were treating
as an error for printing, change it to a NOTCELOG instead of an error.

Change-Id: Id842064b7d659f7ad7f9d785ffffdc6b827f6989
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-29 04:36:59 +00:00
paul luse
9b639aa9a4 bdev/compress: function rename
Old name was misleading due to earlier changes

Change-Id: Ic04223db983e98c9703a43e6778b0094f99bc81d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463341
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-29 03:46:31 +00:00
paul luse
faf9ad6a6d bdev/compress: fail get channel calback if we are out of qpairs
qpairs can be limited by the compression polled mode driver or
system resources and we need one per thread so if we run out we
have to fail the callback.

Change-Id: Ibd07d9c65880e7de321c439119aed479ef00746c
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463339
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-29 03:46:31 +00:00
Jeffry Molanus
bc315deae3 bdev_iscsi: fix crash when connecting to a non existing iqn
When connecting to an iscsi url that effectively does not exist we free
the data structures of the very context we are processing. This results in a
use after free and subsequently in a crash.

Signed-off-by: Jeffry Molanus <jeffry.molanus@gmail.com>
Change-Id: I67cab1efb161bfa23fa1022e150661080d90b556
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462614
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Jeffry Molanus <Jeffry.molanus@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-29 03:18:33 +00:00
paul luse
ca1a3bf18e bdev/compress: use base bdev alias as base for comp_bdev name
If available, if not use the bdev unique name. This results in
a much friendlier comp_bdev naem for the user.  For example,
now it would look something like this: COMP_lvs0/lvs as opposed
to like this COMP_0b149b31-b66b-4cf7-ab39-a55b50788cd1

Change-Id: I319a141221ed8880edcec930ca5be9f256b105e0
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463246
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-26 19:47:33 +00:00
paul luse
e9211cbf45 bdev/compress: fix qpair assignment code
Was previously creating the number of qp based on the # of cores
reported via DPDK function.  Also was assigning one qp per comp_bdev.
After testing w/multiple threads in fio 2 things needed to change:

* remove use of rte core related functions as fio threads are not EAL
threads.  This required choosing an upper limit for the number of
threads that the bdev module will support
* changing the qp assignment code such that comp_bdevs share the
same device_qp if on the same thread

Fixes issue #879

Change-Id: Ib968ee04bef63013616546341974a4431d8f266a
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463107
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-26 19:47:33 +00:00
Hailiang Wang
563b98223b lib/compress: fix a compilation error for low GCC version
Some low GCC(such as v4.8.5) can't recognize this style:
static struct rte_comp_xform g_comp_xform = (struct rte_comp_xform)
Fix issue #877.

Change-Id: I021c62b7b1437d67ce87cc2e13d1b90ea6a18140
Signed-off-by: Hailiang Wang <hailiangx.e.wang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463126
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-25 15:49:24 +00:00
Karol Latecki
5f96e92c8c bdev/pmem: add more descriptive rpc error messages
Improve error messages where possible.
Also change return code values to negative in bdev_pmem.c so that
it follows the same pattern as other bdev modules.

Change-Id: I81b205a41104e15457154d1e31540210636ef9d3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:28:46 +00:00
Karol Latecki
0fc5acea8c bdev/nvme: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: Ief04cf32f7a65ef94f3d994ca76da2637ba66ca1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:27:44 +00:00
Mateusz Kozlowski
cd115ecfcd bdev/ftl: Add defrag parameters to construct_ftl_bdev rpc
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
7a867c4fdb lib/ftl: Pass ftl conf to write_config rpc function
Changed write_config to use the ftl configuration file instead of
allow_open_bands directly. This allows for easier expansion of saved
fields.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie1d480983636f26840dba9a3a6c93b2883903bdb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460519
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
a4b3481242 lib/ftl: Remove buffer alignment requirements
Ftl should no longer require 4k alignment for I/O buffers for either
internal or external requests. The 4k alignment was needed due
to a bug in QEMU when handling internal SGL buffers in LNVM commands.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Icafc14b7811c9c0ffa13789e341c453503cf2821
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-07-23 04:00:05 +00:00
yidong0635
c0d4f7145e crypto: Fix scanbuild issue on Fedora30.
Issue reports:
vbdev_crypto.c:695:12: warning: The left operand of '+' is a garbage value
                iv_ptr = rte_crypto_op_ctod_offset(crypto_ops[crypto_index], uint8_t *,

Change-Id: I1a4f89111e94a13d8daf38070aa69306783a83c5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:38:27 +00:00
yidong0635
425b89a4ed compress: Fix scanbuild error about dereference of null pointer.
Issue reports:
vbdev_compress.c:468:34: warning: Dereference of null pointer
                 src_mbufs[iov_index]->userdata = reduce_cb_arg;
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
Add assert here is enough.

Change-Id: I8dccf107a89f7af46437a1f678adccf122beb4b2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462592
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:17:25 +00:00
yidong0635
a1de2fe007 compress: Clear some simple scanbuild errors.
On Fedora30, GCC9+ reports errors:
Issues report:
vbdev_compress.c:482:17: warning: Assigned value is garbage or undefined
        comp_op->m_src = src_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:502:17: warning: Assigned value is garbage or undefined
        comp_op->m_dst = dst_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:491:3: warning: 1st function call argument is an uninitialized value
                rte_pktmbuf_attach_extbuf(dst_mbufs[iov_index],

compress_ut.c:826:2: warning: Attempt to free released memory
        free(g_mbuf_mp);
        ^~~~~~~~~~~~~~~
Should remove free(g_mbuf_mp), for pointer exchange.

Change-Id: I1a48b7c309a4e2aa0aa513f9be44ae3504d9e385
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462574
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 14:10:32 +00:00
Pawel Kaminski
de1d0f8fe9 jsonrpc: don't check returned value from spdk_jsonrpc_begin_result()
As spdk_jsonrpc_begin_result() is not allowed to return NULL we can
remove these checks. We didn't have any tests cases that goes this path
anyway.

Change-Id: I0894e76c0162591e550e70b172566b9060a6dd5f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 05:13:30 +00:00
Tianyu Yang
96b759297f lib/bdev: fix coredump when bdev initialize failed.
If subsystem bdev module initialize failed, it will call
spdk_bdev_init_complete(-1) -> (subsystem bdev)->fini
_spdk_bdev_finish_unregister_bdevs_iter -> spdk_bdev_module_finish_iter

In abovt path, spdk_bdev_module_finish_iter will repeate to call
bdev_module->module_fini function. Some bdevs will call spdk_io_device_unregister
which never call spdk_io_device_register. It will coredump when assert false in
spdk_io_device_unregister dev is null.

To fix this, let's check whether g_bdev_mgr.module_init_complete is equals true
and then call the bdev_module->module_fini.

Change-Id: Ia9a13318720d954e40eb2d666574bcb86e5f49e3
Signed-off-by: Tianyu Yang <yangtianyu2@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462382
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-19 06:29:25 +00:00
Karol Latecki
ac8b488982 bdev/rbd: add more descriptive rpc error messages
Improve error messages where possible.
Modify spdk_bdev_rbd_create() to return instead instead
of pointer for better return code handling.

Change-Id: I5fcf90794f5fe44296422c654c5f8d404f3d5eef
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461884
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-19 02:21:53 +00:00
Karol Latecki
89003d5ca7 bdev/rpc: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: I6e5a50a1422f55aff0b0bad53493edc80f345ccb
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461600
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 07:52:07 +00:00
Karol Latecki
6d442665fd bdev/rpc: remove if checks for obligatory rpc arguments
These arguments are obligatory and spdk_json_decode_object will
fail even before we reach the if block.

Change-Id: I3c17faf570aefc7d96d009a595cf3a76994977e7
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462043
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-18 07:52:07 +00:00
Karol Latecki
46ebf1b1e9 bdev/virtio: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: I68b48fd448636412a2b26241c787e7b7826c9a34
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461873
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 05:34:52 +00:00
Karol Latecki
2805cad12f bdev/compress: add more descriptive rpc error messages
Improve error messages where possible.
Modify parts of the code so that it follows the same pattern
as in previousle changed bdev modules.

Change-Id: I6672a14900326029c670bfca99a6d085093861c3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461553
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:25:44 +00:00
Karol Latecki
695ecf48bf bdev/delay: update rpc error messages
Consolidate rpc error handling with previous bdev rpc
changes.

Change-Id: I3f0ac3b3a0f09869e8ad7edb4bd4c8fd1e42f6b6
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461557
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:25:40 +00:00
Karol Latecki
b8c84b1a39 bdev/error: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: Ieaae6f2477d4a9f636cc861afbe5d80b3c74a46c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461562
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
2019-07-18 04:25:27 +00:00
Karol Latecki
955518d6fc bdev/iscsi: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: If08bad9f4fc88b51403b89c3f5bfc8894ef4300a
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461567
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:25:09 +00:00
Karol Latecki
c6f2b30918 bdev/raid: add more descriptive rpc error messages
Improve error messages where possible.
Unify returned error codes and messages with previous
changes done in other bdev modules.

Change-Id: Id35b61e8157fab9940e5811b71289978a6f23b05
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461591
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 04:24:45 +00:00
Karol Latecki
fd4d7fcb15 bdev/passthru: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: I778b4b49c1e54c0da77f6da7906df91d8aa95f9b
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461587
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-18 03:25:06 +00:00
Karol Latecki
c05355db23 bdev/lvol: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: I8f736fb6481d65b55e16e548fac0130cd152d3fa
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461575
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 03:24:26 +00:00
Karol Latecki
0965d63d3a bdev/lvol: remove optional flag for lvol_name in construct call
lvol_name is not an optional argument when creating a lvol bdev.
Also removing unnecessary if block.

Change-Id: I7d5790d648bce7f02bfed34bd714162d3ccb10ab
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461861
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 03:24:26 +00:00
Karol Latecki
75711828a9 bdev/lvol: remove if checks for obligatory rpc arguments
These arguments are obligatory and spdk_json_decode_object will
fail even before we reach the if block.

Change-Id: I295649629770ef062086308257da83a8bab44588
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-18 03:24:26 +00:00
Darek Stojaczyk
efef26e234 Revert "bdev: temporarily allow bdev descriptors to be closed from any thread"
Now that vhost closes bdev descriptors on the same thread
that opened them, we can reintroduce thread asserts into
the bdev layer.

This reverts commit 283abcb9a2445183014b5bd5bc1511e837a71de6.

Change-Id: I1acc455df0674b808ecf2fa58dffd183db6cf3c2
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459168
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-16 10:57:46 +00:00
Konrad Sztyber
78097953f7 lib/ftl: notify init/fini callbacks on proper threads
This patch makes sure we're on the thread that requested creation /
deletion of the device when calling the notification callback.

Change-Id: Ia11a8054692874f6b57d4ebe3e3cb290c58e83b6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459618
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-12 12:39:38 +00:00
Vitaliy Mysak
6b654ab900 bdev: prevent early spdk_bdev_init_complete()
In case some module has `async_init = true` and
  some other module that comes after it fails to initialize,
  then callback from asynchronously initialized module
  may call `spdk_bdev_init_complete()` first, then failed module
  will call `spdk_bdev_init_complete()` later.
  This currently results in NULL dereference because
  first call to `spdk_bdev_init_complete()` sets `g_init_cb_fn = NULL`.

This change prevents first call to `spdk_bdev_init_complete()`
  by saying that failed module is not finished with initialization.

This patch fixes #847

Change-Id: Ib6b231d5ea27896ad88d7f11b8732921077b3d4d
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461230
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-12 04:14:58 +00:00
Shuhei Matsumoto
b860c8dbce bdev/gpt: call get buffer function before forwarding read I/O to the base bdev
iSCSI target does not allocate data buffer on read, and delegate
allocation to the bdev.

When the bdev is a split vbdev, the split vbdev does not allocate
data buffer and delegate allocation to the backend bdev.

In this case, iSCSI target expects the buffer is allocated until
notifying completion to the split vbdev. However, the split vbdev
notifies completion to the backend bdev when calling the callback
of iSCSI target. The backend bdev frees the buffer immediately,
but iSCSI target still uses the buffer. If the buffer is reused
by another I/O, data corruption will occur.

For this issue, vbdev_gpt_submti_request() calls
spdk_bdev_io_get_buf() when the I/O is read, and its callback
vbdev_gpt_get_buf_cb calls _vbdev_gpt_submit_request() then.

This will ensure the buffer is allocated before forwarding I/O
to the backed bdev.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ifb2eac500276ab5012123b7d6f7eb033d87ad17c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461350
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-12 02:25:57 +00:00
Shuhei Matsumoto
25532d08f8 bdev/split: call get buffer function before forwarding read I/O to the base bdev
iSCSI target does not allocate data buffer on read, and delegate
allocation to the bdev.

When the bdev is a split vbdev, the split vbdev does not allocate
data buffer and delegate allocation to the backend bdev.

In this case, iSCSI target expects the buffer is allocated until
notifying completion to the split vbdev. However, the split vbdev
notifies completion to the backend bdev when calling the callback
of iSCSI target. The backend bdev frees the buffer immediately,
but iSCSI target still uses the buffer. If the buffer is reused
by another I/O, data corruption will occur.

For this issue, vbdev_split_submti_request() calls
spdk_bdev_io_get_buf() when the I/O is read, and its callback
vbdev_split_get_buf_cb calls _vbdev_split_submit_request() then.

This will ensure the buffer is allocated before forwarding I/O
to the backed bdev.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Icfd0663b548479ac0bf6b5b49420f144142e3300
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461348
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-12 02:25:57 +00:00
Shuhei Matsumoto
316d5c7c79 bdev/part: Remap DIF reference tag for read/write I/O
When using stacked virtual bdev (e.g. split virtual bdev), block
address space will be remapped during I/O processing and so
reference tag have to be remapped accordingly.

This patch adds an new helper function spdk_bdev_part_remap_dif
and call it before submitting write I/O or after completing read
I/O.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idfc6081893861d412c19a9edfb348a7faa7e8c5b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461106
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-11 11:14:22 +00:00
Shuhei Matsumoto
5a31186745 bdev/part: Consolidate getting remapped offset in spdk_bdev_part_submit_request
All IO types but reset have used the remapped offset to submit I/O
to the base bdev. Previously each IO type had got the remapped
offset by itself. Consolidating it into a place will improve
readability and will be helpful for the next patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I29465e92d8fb62e45cfc97c52fedaa661b2f0602
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461105
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-11 11:14:22 +00:00
Maciej Szwed
1b4c99a2ba bdev: Introduce new bdev mutex for accessing bdevs list
In future patch in new spdk_bdev_open_ext function we will call
spdk_bdev_get_by_name function and after that call and before
calling old spdk_bdev_open routine bdev can be removed.
We need to add mutex which will prevent that. Any future code
should use this mutex when accessing the bdevs list to get
a bdev and perform some operation on it.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I785a1791346aebdd394fc51ad0e7fbfbabf317c9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458457
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-11 10:22:06 +00:00
Karol Latecki
a4b0a2b6fd bdev/crypto: add more descriptive rpc error messages
Improve error messages where possible

Change-Id: I2c75cea66dbd635d89e7f27aef59a38c5533b349
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460966
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
2019-07-10 08:29:21 +00:00