This issue can be reproduced on fedora30.
Add assert here is enough to fix this kind of warning.
Error log:
rdma.c:3070:20: warning: Access to field 'data_buf_pool' results in a
dereference of a null pointer (loaded from field 'transport')
spdk_mempool_put(group->transport->data_buf_pool, buf);
^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1 warning generated.
This is to fix issue #965.
Change-Id: Ifb742ab914ee9a0381dca0bb769ba8aa564c816f
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468908
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
On master DPDK (d03d8622d), building SPDK on FreeBSD
fails due to attempt to build internal rte_vhost.
This started occuring with SPDK patch:
28099e0 "make: add dependencies for the rte_vhost build."
This patch disables VHOST_INTERNAL_LIB on FreeBSD,
along with vhost/virtio.
Errors seen:
./configure --enable-debug --enable-werror --with-fio=/usr/src/fio --without-isal --with-dpdk=/var/jenkins/workspace/Other_systems/freebsd_autotest/dpdk/build
Using default SPDK env in /var/jenkins/workspace/Other_systems/freebsd_autotest/spdk/lib/env_dpdk
Notice: DPDK's rte_vhost not found or version < 19.05, using internal, legacy rte_vhost library.
Vhost is only supported on Linux. Disabling it.
Virtio is only supported on Linux. Disabling it.
...
gmake -j4
...
CC lib/rte_vhost/socket.o
In file included from socket.c:52:
./vhost.h:41:10: fatal error: 'linux/vhost.h' file not found
#include <linux/vhost.h>
^~~~~~~~~~~~~~~
....
CC lib/rte_vhost/vhost_user.o
vhost_user.c:41:10: fatal error: 'asm/mman.h' file not found
#include <asm/mman.h>
^~~~~~~~~~~~
Change-Id: Ia2f5fd98a82412a5691bc0f2201f7259a45d2b4d
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468043
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
New VM image for nvme hotplug tests was crafted,
test can be re-enabled.
Fixes#524
Change-Id: Ie7e5e063cd94f2d1cad4b0826f1a51fb43577e6c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
When zcopy APIs are used, target->buf cannot be passed directly
to the bdev layer. Hence in addition to the last patch, copy target->buf
to the buffer allocated by spdk_bdev_zcopy_start() when g_verify or
g_reset is true. When g_verify or g_reset is true, a little performance
degradation will be acceptable.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I2caa4d0423f727954220c4ceac6b37f1b532013d
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467901
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Some older versions of gcc fail with errors when -march=native is not
supplied to the build cmd for checking DPDK rte_vhost support.
Change-Id: I88e12f6823b2a143dc8406c7c01041c00c25011c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468636
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Realted to #957
Normally we are using vbdev_ocf_qcxt allocated by spdk device (ctx_buf),
but after io_device_destroy_cb this context can be freed, so we allocate a
local copy to use for issuing requests on ocf bdev stop. Unfortunately,
in current code we are not freeing it, this patch fix that issue.
Signed-off-by: Marcin Dziegielewski <marcin.dziegielewski@intel.com>
Change-Id: I2d5c0b529f4ff79960945b9e598f21602209f839
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468983
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Added write_unit_size field to bdev structure.
It describes required number of logical blocks
for write operation. For legacy bdevs this value
will be equal to logical block size.
For bdevs working on top of Open Channel/Zoned
Namespace SSDs or RAID 5 volumes write size unit
could be greater than logical block size and
upper layer should perform write operations
with size of multiple of write unit size.
Change-Id: I55eb6687912a7d0d1157874b2778e7d6c2d44e37
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Per patch and nightly tests will run on both QAT and ISAL. If
a test is specified to run on QAT and it's not available it
will fail, same with ISAL.
Change-Id: If4bde8ed4b25363da668ab6084c67af4e95ec66d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463832
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Update transaction length wrt to medata size
Change buffers handling in the case of enabled DIF - add function nvmf_rdma_fill_buffer_with_md_interleave to split SGL into several parts with metadata blocks between them in order to perform RDMA operation with appropriate offsets
Add DIF generation before executing bdev IO operation
Add parsing of DifInsertOrStrip config parameter.
Since there is a limitation on the number of entries in SG list (16), the current approach has a limitation on the max transaction size which depends on the data block size. E.g. if data block size is 512 bytes then the maximum transaction size will be 512 * 16 = 8192 bytes.
In adiition, the size of IO buffer (IOUnitSize conf param) must be aligned to metadata size for better perfromance since metadata is treated as part of this buffer. E.g. if the initiator uses transaction size = 4096, data block size on nvme disk is 512 then IO buffer size should be aligned to (512 + 8) which is 4160. In other case an extra IO buffer will be consumed which will increase the number of entries in SGL and in iov.
Change-Id: I7ad2270fe9dcceb114ece34675eac44e5783a0d5
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/465248
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Adding a new element to this struct in an upcoming patch and want
it to look pretty and be consistent with everything else.
Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I514244ea9441081ba83bd7425101a2bd3ba532c1
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467885
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
In prep for addition of RAID 1E, these functions only apply to
RAID0 now.
Change-Id: I30c4dc9a167d0523e2da2ef2d9861230f12f1a5a
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467554
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Next patch will rename the relevant functions so it's clear they're
RAID 0. This patch simply takes those functions which include RAID
specific mapping and assign them to existing functions via func
pointers in the RAID struct.
Change-Id: I8c7724d855937a9c1ca78cdb8ec500521f23b12d
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467553
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Test all 3 IOV positions but only on src bufs as the dst buf
code under test is identifal and the amount of test code is
rather large.
Change-Id: Idbb635149a5737df9d508adc9dba69e84ec024d3
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466157
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
To make next patch in this series clearer
Change-Id: I6654a92309b4b98351e3793439ecff1b10a24863
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467287
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Weighted Round Robin can be enabled for users, and users
can allocate different priority IO queues for different
purpose. For now we will enable this feature in the
NVMe driver first, following patches will enable this
feature in bdev layer.
Change-Id: I0f799236ca04eb85ef3c9f972ed63ff2718563ba
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466852
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Parameter 'MinConnectionsPerCore' was removed in last release and marked
as deprecated, now we will deprecate 'MinConnectionsPerCore' finally.
Change-Id: I613a371e8b5352dfb84f8e4293805b792020c643
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468789
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
We previously were failing for all make targets except for clean if the
mk/config.mk file didn't exist, but we should also be failing to
perform make clean if we don't have this file. It turns out you end up
erroring out in the next line anyways. Also, if you make that include
optional, make still fails because we don't have any kind of environment
specified and spdk.common.mk depends on that variable being defined.
There may be some benefit to being able to run make clean if we don't
have a valid configuration file, but right now, there are a lot of
things that depend on the configuration file, such as which environment
file to make or clean and which directories need cleaned.
Change-Id: I2a9c1e982c01ba76b67b153ee7336402300b76b0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/466346
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Adds an option to spdkcli.py to allow connection over a tcp
via a host address and port. These options match the format
that exists in rpc.py
Signed-off-by: Mike Carlin <mikefcarlin@protonmail.com>
Change-Id: I7e9e1c376546dd765ffd6f4f4db88e193e9aa0ef
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467844
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a bdev_io pointer to struct bdevperf_task.
In bdevperf_zcopy_get_buf_complete() which is the callback to
spdk_bdev_zcopy_start(), bdev_io is saved into the current task.
Then bdevperf_submit_task() will call spdk_bdev_zcopy_end()
by using the saved bdev_io.
Besides, when spdk_bdev_zcopy_start() is called with populate=false,
increment target->current_queue_depth when it completes successfully,
and do not increment target->current_queue_depth when the corresponding
spdk_bdev_zcopy_end() is called with commit=true.
The reason is that IO processing is already started when
spdk_bdev_zcopy_start() is called.
The next patch will use zcopy APIs for write I/O with verify or
reset cases.
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I12f3b1ccac726abe345a64f06e33d65d2a3538fd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467900
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
There were two variables controlling libftl's tests. Only one of them
is actually needed (SPDK_TEST_FTL), so this patch removes the redundant
one.
Fixes#956
Change-Id: I125b9cb7d855351488b445da165be9e27f10f914
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467932
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
