Commit Graph

9250 Commits

Author SHA1 Message Date
Karol Latecki
4462653272 scripts/rpc: change construct_crypto_bdevs args to positional
Arguments for this function were added as optional arguments
and were listed as such by argparse when using rpc.py --help.

This is confusing as all of these arguments are obligatory
in order to create a crypto bdev in SPDK configuration.

Change-Id: I7c31413314f28e44ee008d3d7c28759063700b61
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:23:02 +00:00
Tomasz Zawadzki
99b25f1c98 ut/cunit: added missing semicolon on CU_ASSERT macros
This was inconsistent on all CU_ASSERT macros,
so now it is fixed.

Change-Id: I6d20f98528c4041baf02cb419ab9644a45b807c4
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 18:21:22 +00:00
Mateusz Kozlowski
cd115ecfcd bdev/ftl: Add defrag parameters to construct_ftl_bdev rpc
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
7a867c4fdb lib/ftl: Pass ftl conf to write_config rpc function
Changed write_config to use the ftl configuration file instead of
allow_open_bands directly. This allows for easier expansion of saved
fields.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie1d480983636f26840dba9a3a6c93b2883903bdb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460519
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
a039d024ff lib/ftl: Remove anonymous defrag structure in config
Merged configuration fields for defrag into the main config structure.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I315c78e2843bd6a310b484af5b688aca424336b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460518
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
5a16b8e671 lib/ftl: Segmentation fault fix after write error handling
Fixed touching NULL write pointer, which would happen after some other
I/O in the same band would receive error and issue its deletion. Any
subsequent write callbacks would then be unable to find and update the
information.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I07d9791f1b1b673a48356fae55b784c0bf6a6f7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-24 18:20:48 +00:00
paul luse
eb1ddcb664 lib/reduce: remove dead code clearing unused backing io units
At the start of time all of values of io_unit_index[] array in
the chunk maps are FF's.  When a partial chunk is written, only
some of the io_unit_index[] vals will be assigned values from the
allocated_backing_io_units bit array.  When that chunk is written
again, it is not done so in place and at that time in when
_write_write_done() is called, the io_unit_index[] values are
set to FF again and the bit array bits are cleared for the old
chunk so that it is fresh again for use by another write.

This change removes code from an early stage of the write process
where a 'fresh' chuck in allocated.  Because of points made earlier,
it is not possible for any of the io_unit_index[] values to be
assigned or have a corresponding bit set in the bit array.

The prev version of this patch cleared the bit array because the
io_unit_index[] array was being cleared however after further
investigation, and for the reasons stated earlier, the io_unit_index[]
did not have a valid value so clearing the bit array was simply wrong.

Removing this dead code clears things up and leaves it to one place
to clear the io_unit_index[], when a chunk is overwritten.  Well, 2
places including init time.

Change-Id: I5c4e5360c9c13817d2fa7d3c7d004b44537af7f4
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458520
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:12:05 +00:00
Tomasz Zawadzki
8df52a0f4a lib/nvme_tcp: assert tcp_req->req before it is dereferenced
The value of tcp_req->req was asserted after it was already
dereferenced. This patch fixes that.

Change-Id: I5eb01e88be09d41fb8e632c49d5a7ccf2315788f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462508
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 18:09:33 +00:00
Tomasz Zawadzki
f75d1540e9 lib/ftl: check namespace for NULL during initialization
spdk_nvme_ctrlr_get_ns() can return NULL, which was
not verified, but dereferenced anyway.

Change-Id: Ib0554148cc47ea3ff2afa1862ac962f284fe6d43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:09:29 +00:00
Tomasz Zawadzki
eb84a1133f lib/ftl: assert write pointer from band
This is just assert to assure the Klocwork
static analizer that wptr is not NULL.

Change-Id: I003620431afb8497a57c50af8480c34ec3fac6d6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462489
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:09:25 +00:00
Karol Latecki
db330b7441 test/nvme: do not use mounted nvme disks in perf script
For kernel-* mode do not use mounted nvme disks for tests.

This was not checked previously and could result in losing
all data on NVMes not meant to be used in test.

Change-Id: I336c95b8e79e2592897c6b354e5a20a9c86f1482
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-07-24 17:57:27 +00:00
Pawel Kaminski
aec3021924 scripts/vagrant: Add possibility to create vm image with devstack
Change-Id: I880dbe9d29ec4822193e6f570107f7fc12e56e3d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-24 17:50:48 +00:00
Maciej Szwed
b7408943d1 test/json: Fix json tests for NVMe with multiple namespaces
Currently json tests try to remove NVMe controller for
each NVMe bdev, but with NVMe that has multiple namepsaces
it actually tries to remove same controller multiple times.
This patch fixes that issue.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I555cb2a9aa78fd4cc700ddda0f5c1026329a2d3a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 11:57:46 +00:00
Changpeng Liu
3fe300609e nvmf: check HOSTNQN access right for discovery service
Initiator can use `nvme discover` command to display all
the subsystem's information, because we don't check
the allowed HOSTNQN for Discovery service, so here
adding this feature so that only return the log pages
to the allowed hosts.

Fix issue #576.

Change-Id: I51e6770bd67ea0b41caf9de3a8899923377e6255
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Changpeng Liu
234eb48bf6 nvmf: save hostnqn to controller data structure
When creating a new controller in the NVMe-oF target, hostnqn is
a must parameter, so we save the hostnqn to controller data
structure, and it can be used to verify the access right of
Discovery service.

Change-Id: I86a6f50d3209d5bbb8ac85508288173d826ea216
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462439
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Alexey Marchuk
f0b7a6e7d1 rdma: fix possible double free on qpair destruction
Update rqpair->last_wqe_reached in the context of thread that owns qpair's poll group to avoid possible double free
This patch fixes #858

Change-Id: If5422944b7928c2cc05af528fbcc4482aeef22df
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:57 +00:00
Darek Stojaczyk
34cdf61f80 ut/nvme: add missing probe_ctx.init_ctrlrs tailq initialization
Detected by scanbuild, but it's actually a programming
error. It could make the failed test segfault instead
of cleanly printing the total number of failures.

Change-Id: I12f4bc2b1ccd2034251c1840381875d51708c6fb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
bc6a51aaff ut/ftl_reloc: silence scan-build errors
Scan-build thinks we do TAILQ_REMOVE on an element that's
not in the tailq. Add fatal asserts to prove it wrong.

Change-Id: Ie5d999c2772d42e471cd4c9a11a6f3d2125cb0bc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462870
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
a7ff5ff558 env_dpdk/memory: silence scan-build errors
Scan-build complains that spdk_vtophys_notify() can segfault
on vaddr == 0. We know that and it's not a programming error.
In fact, SPDK with UIO pci driver can segfault upon registering
any non-mapped memory address. The user is just not supposed
to do that.

Assert vaddr != 0 to silence scan-build.

Change-Id: I9a14696361eca0aeea5ede9f9f2956fcbf59bdb5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462869
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
bb63fe6fc3 blobstore: don't realloc any memory under scan-build
Scan-build has a real issue with reallocs. The original
error from latest version of scan-build is rather complicated,
but it can be greatly simplified with the following change:

> diff --git a/lib/blob/blobstore.c b/lib/blob/blobstore.c
> index 7580c9dd2..6a594edf3 100644
> --- a/lib/blob/blobstore.c
> +++ b/lib/blob/blobstore.c
> @@ -1147,8 +1147,9 @@
> _spdk_blob_persist_clear_clusters_cpl(spdk_bs_sequence_t *seq, void *cb_arg, int
>         } else if (blob->active.num_clusters != blob->active.cluster_array_size) {
>                 tmp = realloc(blob->active.clusters, sizeof(uint64_t) * blob->active.num_clusters);
>                 assert(tmp != NULL);
> -               blob->active.clusters = tmp;
> -               blob->active.cluster_array_size = blob->active.num_clusters;
> +               ctx->blob->active.clusters = tmp;
> +               assert(ctx->blob->active.clusters[0] != 14213);
> +               ctx->blob->active.cluster_array_size = ctx->blob->active.num_clusters;
>         }
>
>         _spdk_blob_persist_complete(seq, ctx, bserrno);
> ```

Scan-build will then complain:

blobstore.c:1151:10: warning: Use of memory after it is freed
                assert(ctx->blob->active.clusters[0] != 14213);

Asserting blob == ctx->blob, blob->active.clusters == ctx->...,
or even tmp != blob->active.clusters doesn't work, so use the
last resort scan-build weapon - #ifdef __clang_analyzer__.

The realloc in this case is just down-sizing a buffer to
save some memory. For scan-build, just don't do it. This
finally silences all scan-build false positives.

Change-Id: Ib88ea145370f5035eedd2412e98ee61f96ad1915
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:23 +00:00
Alexey Marchuk
5282edfd7b rdma: fix double free of qpair struct in case of failed initialization
qpair structure is freed and an error code is returned to the caller in the case of failed qpair initialization in function spdk_nvmf_rdma_qpair_initialize (e.g. bad return value of rdma_create_qp).
The return code is handled by nvmf_tgt_poll_group_add function which destroys the qpair for the second time.
This patch fixes #857

Change-Id: I0773652ecccbbd634ad272106e0a93c1e591d7d2
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462011
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:55:43 +00:00
Tomasz Zawadzki
deacebd5d5 test/blockdev: remove dependency on brd kernel module
AIO bdev works on any file available from kernel.
blockdev.sh is the only palce that uses the ramdisk
in the current autotest.sh.
After removing the /dev/ram0, aio bdev still gets tested
with regular file.

Note: the QoS functionality done on AIO0 still gets
tested in usual qos.sh tests.

Change-Id: I8d219206d49626582d5848239fb0c36b25ef92a1
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:54:39 +00:00
Maciej Wawryk
ca03bc5cf7 test/vhost: check block size for vhost boot test
Resolves #695

Change-Id: I627b06bc32e2a3aa18fb92636da384f60fb5db2d
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462446
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:51:28 +00:00
Tomasz Zawadzki
06c7ef7f6a lib/ftl: check for io not being NULL before dereferencing
ftl_io_free() handles NULL io being passed, but shouldn't
dereference it before returning.

Change-Id: Ia705d4ab9f8b3569e9704a1f97e70f7579084efb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:30 +00:00
Tomasz Zawadzki
8500db5ee0 lib/vmd: check header_type after the vmd_adapter structure is verified
First check both vmd_pci_device for not being a NULL,
same as vmd_adapter structure being present.

Otherwise dev was dereferenced before checking for NULL.

Change-Id: I3719f33c1762f789b0eb63b0f2ddce8258843ded
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:04 +00:00
lorneli
ba323d44ca nvmf/rdma: log spdk_nvmf_rdma_destroy_defunct_qpair
Func spdk_nvmf_rdma_destroy_defunct_qpair is a "last chance option"
to destroy qp manually if some driver/hardware doesn't drain qp's
failed wr as expected.

There's a probability that ibv_poll_cq polls wr of the destoryed qp
after spdk_nvmf_rdma_destroy_defunct_qpair's execution. Although in
practice the risk of this situation is minimal(if not non-existent),
add a log here so that we could detect this situation easily.

Change-Id: Ifa9534397513bcea34c18fbb8168eef8f53599c1
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462441
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
lorneli
b4d3066890 nvmf/rdma: defer qp destruction until nvmf layer closes qp
Currently rqpair will be destroyed directly in ibv_poll_cq path
if it has been drained, regardless of whether there are outstanding
I/Os issued to bdev layer. So after outstanding I/Os completing,
spdk_nvmf_rdma_close_qpair will be called from nvmf layer, accessing
a destroyed qp.

This path defers qp destruction in nvmf_rdma_destroy_drained_qpair
func until nvmf layer closes qp.

Fixes 851

Change-Id: I8bcce66f8053ddb105702ac603d5d73af54bdcfc
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461237
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
Ben Walker
7dbdf87f63 make: Allow users to specify the target build architecture
make TARGET_ARCHICTECTURE=broadwell

Change-Id: I868c2f5c203dfc1a4e6cf48c39b391e174d5d33b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 17:01:17 +00:00
Seth Howell
8f12bc5b69 test/nvmf: disable the shutdown tests for the tcp transport.
This test is very unstable on TCP. Until we can get to the bottom of the
latent failures, we should disable it. This will allow us to continue
testing a substantial amount of tcp functionality while avoiding latent
failures in the test pool.

Change-Id: I3c7ec2709ad81a3ec5c1a4d56027b27a9c6ffc7e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 08:50:57 +00:00
yidong0635
3e19963297 nvme_qpair_ut: Fix scanbuild warning about using freed memory.
Issue reports:
spdk/test/common/lib/test_env.c:103:2: warning: Use of memory after it is freed
        HANDLE_RETURN_MOCK(spdk_zmalloc);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
spdk/include/spdk_internal/mock.h:106:3: note: expanded from macro 'HANDLE_RETURN_MOCK'
                return ut_ ## fn; \
                ^~~~~~~~~~~~~~~~
1 warning generated.

Here includes test_env.c. That spdk_zmalloc will call HANDLE_RETURN_MOCK ,that leads
requesting memory in functions which call spdk_zmalloc(including nvme_qpair_init/
spdk_nvme_qpair_add_cmd_error_injection) failed.

Change-Id: I6c14dfd2fc7b7078a18a46ef8ff830f85940487e
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 04:50:29 +00:00
Mateusz Kozlowski
5094302f5a test/ftl: Changed bdev_svc usage to spdk_tgt
Changed all ftl tests to use spdk_tgt instead.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ibb19b233f5cb855e1b6edb41145890e5b103493c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461213
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 04:09:32 +00:00
Seth Howell
e394fb3bed test/fuzz: Make test timeout configurable.
We should be able to change the runtime of the application based
on command line arguments without having to update the bash scripts.

Change-Id: I9fbd8e0386d7e3c39da853a89bf4aff08cb916f0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 04:08:52 +00:00
Mateusz Kozlowski
a3c78e150b lib/ftl: Remove separate dma_buf allocation in band
Since 4k alignment is no longer required for I/O buffers, the
band doesn't need a separate lba map and dma buffer and can use the
same memory location.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Iea127e8c2f39e6de5d57258098b2dc6be56f439f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
a4b3481242 lib/ftl: Remove buffer alignment requirements
Ftl should no longer require 4k alignment for I/O buffers for either
internal or external requests. The 4k alignment was needed due
to a bug in QEMU when handling internal SGL buffers in LNVM commands.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Icafc14b7811c9c0ffa13789e341c453503cf2821
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
1897943987 env: define minimum alignment for dma-able memory
DPDK defines the minimum alignment as "suitable for any
kind of variable (in the same manner as malloc())", but
internally the alignment is always rounded up to the
cache line size, even if the requested alignment is 0.

We would like to start relying on this behavior in FTL,
where lba maps are allocated using DMA-able memory and
are constantly looked up or modified by different threads.
By having the lba maps unaligned, we risk having those
threads pollute each other's cache lines.

Rather than enforcing this memory alignment in FTL, we
do it in spdk_*malloc directly. In general it makes
sense to have DMA-able memory always cache-line-size
aligned for the same reason as above.

Change-Id: Ib6edda4a7bf3f4952eb1875a4e1753be96bed642
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 04:00:05 +00:00
yidong0635
e3b90fe2d5 bdev: Fix warning about potenial memory leak about histogram.
Pointer passthrough g_histogram and histogram implicitly ,but the checker
still complains about this, change to spdk_histogram_data_free(histogram)
directly, no need any assert any more.

bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Change-Id: I609c1c92643ad03b8e3c62cd8434882e7942efa4
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462789
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 03:50:05 +00:00
Seth Howell
a78b7bb18c test/nvmf: avoid running bdevperf on top of nvmf_tgt
When you run verify on bdev perf, it rejects any user provided core
masks and pins itself to core 0. We were also running the NVMe-oF target
on cores 0-3. Occasionally I was seeing the shutdown tests failing
during the waitforio step. It seemed like the target and bdevperf were
both still online during this failure, so my hypothesis is that the
bdevperf poller was stealing all of the cycles from the NVMe-oF poller
completing I/O so the I/O wasn't completing.

Even if that's not what's happening, it is probably a good idea to split
the NVMe-oF application and bdevperf onto different cores.

Change-Id: Ib3b5b00e639ebd14bd1ed2cfb4b7782076ca364c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462551
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 03:48:08 +00:00
Jesse Grodman
53c9f4350a docs: update concurrency.md regarding thread creation
Fixes #issue number 866

Signed-off-by: Jesse Grodman <jgrodman@gmail.com>
Change-Id: Ia102fbb285ef8914008d1a92347080953d394d2a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462878
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 01:28:05 +00:00
x00267701
a50246fb64 log: optimize spdk_log() when logging is disabled
If the log level of the output log is higher than
the process's log level, the system does not output
it, so we needn't generate the formatting logs.

Change-Id: I36be0e6807ed575fcbf1d0ae01f064a6ca2c4539
Signed-off-by: Huiming Xie <xiehuiming@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462790
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:42:24 +00:00
yidong0635
c0d4f7145e crypto: Fix scanbuild issue on Fedora30.
Issue reports:
vbdev_crypto.c:695:12: warning: The left operand of '+' is a garbage value
                iv_ptr = rte_crypto_op_ctod_offset(crypto_ops[crypto_index], uint8_t *,

Change-Id: I1a4f89111e94a13d8daf38070aa69306783a83c5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462576
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:38:27 +00:00
yidong0635
2ab927030d bdev/vbdev_lvol: Fix scanbuild error about Null poniter parameter.
Issue reorts:
vbdev_lvol_ut.c:97:7: warning: Null pointer passed as an argument to a 'nonnull' parameter
                if (strncmp(alias, tmp->alias, SPDK_LVOL_NAME_MAX) == 0) {
                    ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Change-Id: I0cc95c735ea7486263842f91a5866710f9ff61a5
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462598
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:17:25 +00:00
yidong0635
425b89a4ed compress: Fix scanbuild error about dereference of null pointer.
Issue reports:
vbdev_compress.c:468:34: warning: Dereference of null pointer
                 src_mbufs[iov_index]->userdata = reduce_cb_arg;
                    ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
Add assert here is enough.

Change-Id: I8dccf107a89f7af46437a1f678adccf122beb4b2
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462592
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:17:25 +00:00
Tomasz Zawadzki
b3d9ebae21 example/perf: remove libaio from perf example
Original intention most likely was to compare nvme lib to
aio, but since then aio bdev was developed.

There should be no need to keep the support for AIO in
perf example app.

Change-Id: I716f2164b8eaef16db05b4fffa35393c7d916e0b
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461593
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 00:13:35 +00:00
Wojciech Malikowski
4c5282bcf7 lib/vmd: Clear prefetch base and limit registers
During PCI bridge initialization not all
registers was cleaned properly.

Change-Id: Ie0752bb303ba06e2a21669908efd9f746f6b0039
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460197
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-22 20:02:54 +00:00
yidong0635
a1de2fe007 compress: Clear some simple scanbuild errors.
On Fedora30, GCC9+ reports errors:
Issues report:
vbdev_compress.c:482:17: warning: Assigned value is garbage or undefined
        comp_op->m_src = src_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:502:17: warning: Assigned value is garbage or undefined
        comp_op->m_dst = dst_mbufs[0];
                       ^ ~~~~~~~~~~~~
vbdev_compress.c:491:3: warning: 1st function call argument is an uninitialized value
                rte_pktmbuf_attach_extbuf(dst_mbufs[iov_index],

compress_ut.c:826:2: warning: Attempt to free released memory
        free(g_mbuf_mp);
        ^~~~~~~~~~~~~~~
Should remove free(g_mbuf_mp), for pointer exchange.

Change-Id: I1a48b7c309a4e2aa0aa513f9be44ae3504d9e385
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462574
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 14:10:32 +00:00
Konrad Sztyber
38889bbfd6 test/ftl: reduce the number of required parallel units
One of the tests required the OC device to have at least 8 parallel
units.  It's not really necessary as we can test the same thing on
fewer (4) units.

Change-Id: Ib3ccd7fee0b8ecd2663b12e4aa9aa802ebe238a8
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461877
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-22 09:23:16 +00:00
Alexey Marchuk
0754417fa9 rdma: Use optimal ceiling integer division
This form of the celinig division allows to remove an extra condition

Change-Id: I8a2de792172ec9115563e7fb914745c476f16e8d
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462198
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 09:22:11 +00:00
Pawel Kaminski
de1d0f8fe9 jsonrpc: don't check returned value from spdk_jsonrpc_begin_result()
As spdk_jsonrpc_begin_result() is not allowed to return NULL we can
remove these checks. We didn't have any tests cases that goes this path
anyway.

Change-Id: I0894e76c0162591e550e70b172566b9060a6dd5f
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459199
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 05:13:30 +00:00
Tomasz Zawadzki
37acf6814b lib/ftl: assert that chunk is never NULL
Code on line 613 should never be reached for last
chunk on the band, since it is checked on line 606.

This assert verifies that, all chunks before last
are not NULL.

Change-Id: I78a9967f91c0873981b3308e888d85fa7c8d6641
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462488
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-22 04:53:05 +00:00
Tomasz Zawadzki
94bf29d268 lib/ftl: verify that iov is not NULL during read header completion
Moved the spdk_bdev_io_get_iovec() after the verification that io
succeeded.
Then added return path when iov could not be retrived from bdevio.

Change-Id: I2638e6657fd08a2dd85ab0906383cecef8fb4f4a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462507
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-22 04:51:34 +00:00