Commit Graph

9361 Commits

Author SHA1 Message Date
paul luse
907ec7268d lib/reduce: conditionally memcpy data to host follow decompression
If the decompression code path runs into a chunk that wasn't
compressed, the data needs to be copied to the host as the
compression engine was not engaged.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Ibabd6d6bd2bc5db79953a0a55d7f49d556c08af7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462892
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-25 14:50:25 +00:00
paul luse
0242a80793 lib/reduce: fix problem with size of chunk map area in PMEM
This fixes a segfault found by using a very small logical volume.
The PMEM file chunk map area wasn't including the chunk map struct
itself.  In PMEM, a single entry as returned by _reduce_vol_get_chunk_map()
is made up of a 'struct spdk_reduce_chunk_map' plus one uint64_t for
each backing IO unit for the volume (# depends on vol parms). So when
an app got close to the end of the chunk map indices it would run off
the end of PMEM and segfault.

With this fix that no longer happens however data integrity issues now
show up with fio. I believe this fix is valid though and am pursuing the data
integrity issue as something new.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: I69a223064c7bbd44bcd0a62d155cf409603970a3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461316
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-25 14:50:25 +00:00
Seth Howell
30ca1a9083 test/nvmf: fix edge condition in check_ip_is_soft_roce
When individual tests were run with --iso --transport=tcp, the
check_ip_is_soft_rocefunction incorrectly marked the loopback ip address
as soft_roce due to 'lo' matching with a notice message from the rxe_cfg
config command "module rdma_rxe is not loaded". Instead, grep
specifically to make sure that the argument we are matching against,
which is supposed to be an rxe interface name, contains rxe.

rxe interfaces are numbered sequentially as rxe0 rxe1 . . . rxen

Change-Id: Ibe8afb3e038e104173eb1e2bbf671ce9f417ffeb
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463127
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-25 14:49:17 +00:00
Ben Walker
0e6ddfd66a build: Allow the user to set target architecture in configure
Change-Id: I7f2a1464c0c8872e5e7856a830aaf7ea8cdf4e81
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463013
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 22:55:06 +00:00
Ben Walker
ebd32197ba build: Allow TARGET_ARCHITECTURE to be specified on ARM
Previously it always forced armv8-a+crc. Now it only does that
if not otherwise specified.

Change-Id: I7c1d73fca8e89779cc3e8780a38f52975488d97a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 22:55:06 +00:00
Darek Stojaczyk
30ff7f7d85 dpdkbuild: completely rebuild dpdk on SPDK config change
Configuring SPDK with different options might result
in enabling different CONFIG_RTE_LIB* options, but since
it does not modify any DPDK config files, DPDK doesn't
get rebuilt and keeps using older, potentially outdated
build options.

We'll now clear dpdk/build whenever SPDK config is modified,
which will result in automatically re-configuring DPDK.

Change-Id: I9c7c3434dc317e4f47767d7b3df3df48db2b0d5b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-24 21:27:16 +00:00
Karol Latecki
5f96e92c8c bdev/pmem: add more descriptive rpc error messages
Improve error messages where possible.
Also change return code values to negative in bdev_pmem.c so that
it follows the same pattern as other bdev modules.

Change-Id: I81b205a41104e15457154d1e31540210636ef9d3
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461548
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:28:46 +00:00
Karol Latecki
0fc5acea8c bdev/nvme: add more descriptive rpc error messages
Improve error messages where possible.

Change-Id: Ief04cf32f7a65ef94f3d994ca76da2637ba66ca1
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461875
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:27:44 +00:00
Tomasz Zawadzki
438d1beee7 lib/blob: amend docs regarding io_units for spdk_blob_io_*()
Some of the spdk_blob_io_*() still described pages as units
for submission instead of io_units.

No API change is occurring with this patch.

Change-Id: I6530db59de97a494ca7c5bf191e26d2a207c1159
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/463067
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Niu Yawei <yawei.niu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:26:21 +00:00
Vitaliy Mysak
b2caa1f0eb test/spdkcli: test vhost_ctrl show_details method
Add a match test for vhost_ctrl show_details method.
In spdkcli source the equivalent function is `UIVhostCtrl.show_details`.
This method is one of few that left untested in spdkcli.

This patch is related to trello task:
https://trello.com/c/CHOOxcGj/151-spdkcli-extend-test-coverage

Change-Id: I4f13f863bdc815430c5fb2b67b12056f01201f8c
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462860
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:24:22 +00:00
Vitaliy Mysak
56c696a9a7 test/spdkcli: test set_coalescing method
Add spdkcli command that will invoke set_coalescing method
  during vhost test.
This is the only usage of set_coalescing in our tests,
  so it would be nice to at least have an invocation of it.

Following commits will add a match-test that will check if
  set_coalescing worked as expected.

This patch is related to trello task:
https://trello.com/c/CHOOxcGj/151-spdkcli-extend-test-coverage

Change-Id: I15f54d92c4af2cf3f400743425e268b3645a5b3c
Signed-off-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462859
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:24:22 +00:00
Karol Latecki
4462653272 scripts/rpc: change construct_crypto_bdevs args to positional
Arguments for this function were added as optional arguments
and were listed as such by argparse when using rpc.py --help.

This is confusing as all of these arguments are obligatory
in order to create a crypto bdev in SPDK configuration.

Change-Id: I7c31413314f28e44ee008d3d7c28759063700b61
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460968
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Vitaliy Mysak <vitaliy.mysak@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:23:02 +00:00
Tomasz Zawadzki
99b25f1c98 ut/cunit: added missing semicolon on CU_ASSERT macros
This was inconsistent on all CU_ASSERT macros,
so now it is fixed.

Change-Id: I6d20f98528c4041baf02cb419ab9644a45b807c4
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/459760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 18:21:22 +00:00
Mateusz Kozlowski
cd115ecfcd bdev/ftl: Add defrag parameters to construct_ftl_bdev rpc
Added parsing and saving of (optional) parameters which are responsible for
relocation in FTL. Changing the parameters may affect Write
Amplification Factor and overall performance, especially during random
write workloads. If parameters are not specified, default values will be used.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ic0229e39109460f3541c31e1cbf2a485efe408c6
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460504
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
7a867c4fdb lib/ftl: Pass ftl conf to write_config rpc function
Changed write_config to use the ftl configuration file instead of
allow_open_bands directly. This allows for easier expansion of saved
fields.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ie1d480983636f26840dba9a3a6c93b2883903bdb
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460519
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
a039d024ff lib/ftl: Remove anonymous defrag structure in config
Merged configuration fields for defrag into the main config structure.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I315c78e2843bd6a310b484af5b688aca424336b0
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460518
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:21:07 +00:00
Mateusz Kozlowski
5a16b8e671 lib/ftl: Segmentation fault fix after write error handling
Fixed touching NULL write pointer, which would happen after some other
I/O in the same band would receive error and issue its deletion. Any
subsequent write callbacks would then be unable to find and update the
information.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: I07d9791f1b1b673a48356fae55b784c0bf6a6f7a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462387
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-24 18:20:48 +00:00
paul luse
eb1ddcb664 lib/reduce: remove dead code clearing unused backing io units
At the start of time all of values of io_unit_index[] array in
the chunk maps are FF's.  When a partial chunk is written, only
some of the io_unit_index[] vals will be assigned values from the
allocated_backing_io_units bit array.  When that chunk is written
again, it is not done so in place and at that time in when
_write_write_done() is called, the io_unit_index[] values are
set to FF again and the bit array bits are cleared for the old
chunk so that it is fresh again for use by another write.

This change removes code from an early stage of the write process
where a 'fresh' chuck in allocated.  Because of points made earlier,
it is not possible for any of the io_unit_index[] values to be
assigned or have a corresponding bit set in the bit array.

The prev version of this patch cleared the bit array because the
io_unit_index[] array was being cleared however after further
investigation, and for the reasons stated earlier, the io_unit_index[]
did not have a valid value so clearing the bit array was simply wrong.

Removing this dead code clears things up and leaves it to one place
to clear the io_unit_index[], when a chunk is overwritten.  Well, 2
places including init time.

Change-Id: I5c4e5360c9c13817d2fa7d3c7d004b44537af7f4
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/458520
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:12:05 +00:00
Tomasz Zawadzki
8df52a0f4a lib/nvme_tcp: assert tcp_req->req before it is dereferenced
The value of tcp_req->req was asserted after it was already
dereferenced. This patch fixes that.

Change-Id: I5eb01e88be09d41fb8e632c49d5a7ccf2315788f
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462508
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 18:09:33 +00:00
Tomasz Zawadzki
f75d1540e9 lib/ftl: check namespace for NULL during initialization
spdk_nvme_ctrlr_get_ns() can return NULL, which was
not verified, but dereferenced anyway.

Change-Id: Ib0554148cc47ea3ff2afa1862ac962f284fe6d43
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462486
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-24 18:09:29 +00:00
Tomasz Zawadzki
eb84a1133f lib/ftl: assert write pointer from band
This is just assert to assure the Klocwork
static analizer that wptr is not NULL.

Change-Id: I003620431afb8497a57c50af8480c34ec3fac6d6
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462489
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-24 18:09:25 +00:00
Karol Latecki
db330b7441 test/nvme: do not use mounted nvme disks in perf script
For kernel-* mode do not use mounted nvme disks for tests.

This was not checked previously and could result in losing
all data on NVMes not meant to be used in test.

Change-Id: I336c95b8e79e2592897c6b354e5a20a9c86f1482
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460326
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: John Kariuki <John.K.Kariuki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-07-24 17:57:27 +00:00
Pawel Kaminski
aec3021924 scripts/vagrant: Add possibility to create vm image with devstack
Change-Id: I880dbe9d29ec4822193e6f570107f7fc12e56e3d
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456325
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-07-24 17:50:48 +00:00
Maciej Szwed
b7408943d1 test/json: Fix json tests for NVMe with multiple namespaces
Currently json tests try to remove NVMe controller for
each NVMe bdev, but with NVMe that has multiple namepsaces
it actually tries to remove same controller multiple times.
This patch fixes that issue.

Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I555cb2a9aa78fd4cc700ddda0f5c1026329a2d3a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461561
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-24 11:57:46 +00:00
Changpeng Liu
3fe300609e nvmf: check HOSTNQN access right for discovery service
Initiator can use `nvme discover` command to display all
the subsystem's information, because we don't check
the allowed HOSTNQN for Discovery service, so here
adding this feature so that only return the log pages
to the allowed hosts.

Fix issue #576.

Change-Id: I51e6770bd67ea0b41caf9de3a8899923377e6255
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462440
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Changpeng Liu
234eb48bf6 nvmf: save hostnqn to controller data structure
When creating a new controller in the NVMe-oF target, hostnqn is
a must parameter, so we save the hostnqn to controller data
structure, and it can be used to verify the access right of
Discovery service.

Change-Id: I86a6f50d3209d5bbb8ac85508288173d826ea216
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462439
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
2019-07-24 11:25:59 +00:00
Alexey Marchuk
f0b7a6e7d1 rdma: fix possible double free on qpair destruction
Update rqpair->last_wqe_reached in the context of thread that owns qpair's poll group to avoid possible double free
This patch fixes #858

Change-Id: If5422944b7928c2cc05af528fbcc4482aeef22df
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462012
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:57 +00:00
Darek Stojaczyk
34cdf61f80 ut/nvme: add missing probe_ctx.init_ctrlrs tailq initialization
Detected by scanbuild, but it's actually a programming
error. It could make the failed test segfault instead
of cleanly printing the total number of failures.

Change-Id: I12f4bc2b1ccd2034251c1840381875d51708c6fb
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462871
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
bc6a51aaff ut/ftl_reloc: silence scan-build errors
Scan-build thinks we do TAILQ_REMOVE on an element that's
not in the tailq. Add fatal asserts to prove it wrong.

Change-Id: Ie5d999c2772d42e471cd4c9a11a6f3d2125cb0bc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462870
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
a7ff5ff558 env_dpdk/memory: silence scan-build errors
Scan-build complains that spdk_vtophys_notify() can segfault
on vaddr == 0. We know that and it's not a programming error.
In fact, SPDK with UIO pci driver can segfault upon registering
any non-mapped memory address. The user is just not supposed
to do that.

Assert vaddr != 0 to silence scan-build.

Change-Id: I9a14696361eca0aeea5ede9f9f2956fcbf59bdb5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462869
Reviewed-by: yidong0635 <dongx.yi@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:56:23 +00:00
Darek Stojaczyk
bb63fe6fc3 blobstore: don't realloc any memory under scan-build
Scan-build has a real issue with reallocs. The original
error from latest version of scan-build is rather complicated,
but it can be greatly simplified with the following change:

> diff --git a/lib/blob/blobstore.c b/lib/blob/blobstore.c
> index 7580c9dd2..6a594edf3 100644
> --- a/lib/blob/blobstore.c
> +++ b/lib/blob/blobstore.c
> @@ -1147,8 +1147,9 @@
> _spdk_blob_persist_clear_clusters_cpl(spdk_bs_sequence_t *seq, void *cb_arg, int
>         } else if (blob->active.num_clusters != blob->active.cluster_array_size) {
>                 tmp = realloc(blob->active.clusters, sizeof(uint64_t) * blob->active.num_clusters);
>                 assert(tmp != NULL);
> -               blob->active.clusters = tmp;
> -               blob->active.cluster_array_size = blob->active.num_clusters;
> +               ctx->blob->active.clusters = tmp;
> +               assert(ctx->blob->active.clusters[0] != 14213);
> +               ctx->blob->active.cluster_array_size = ctx->blob->active.num_clusters;
>         }
>
>         _spdk_blob_persist_complete(seq, ctx, bserrno);
> ```

Scan-build will then complain:

blobstore.c:1151:10: warning: Use of memory after it is freed
                assert(ctx->blob->active.clusters[0] != 14213);

Asserting blob == ctx->blob, blob->active.clusters == ctx->...,
or even tmp != blob->active.clusters doesn't work, so use the
last resort scan-build weapon - #ifdef __clang_analyzer__.

The realloc in this case is just down-sizing a buffer to
save some memory. For scan-build, just don't do it. This
finally silences all scan-build false positives.

Change-Id: Ib88ea145370f5035eedd2412e98ee61f96ad1915
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462868
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:56:23 +00:00
Alexey Marchuk
5282edfd7b rdma: fix double free of qpair struct in case of failed initialization
qpair structure is freed and an error code is returned to the caller in the case of failed qpair initialization in function spdk_nvmf_rdma_qpair_initialize (e.g. bad return value of rdma_create_qp).
The return code is handled by nvmf_tgt_poll_group_add function which destroys the qpair for the second time.
This patch fixes #857

Change-Id: I0773652ecccbbd634ad272106e0a93c1e591d7d2
Signed-off-by: Alexey Marchuk <alexeymar@mellanox.com>
Signed-off-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Signed-off-by: Evgenii Kochetov <evgeniik@mellanox.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462011
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Lorne Li <lorneli@163.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:55:43 +00:00
Tomasz Zawadzki
deacebd5d5 test/blockdev: remove dependency on brd kernel module
AIO bdev works on any file available from kernel.
blockdev.sh is the only palce that uses the ramdisk
in the current autotest.sh.
After removing the /dev/ram0, aio bdev still gets tested
with regular file.

Note: the QoS functionality done on AIO0 still gets
tested in usual qos.sh tests.

Change-Id: I8d219206d49626582d5848239fb0c36b25ef92a1
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461594
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 22:54:39 +00:00
Maciej Wawryk
ca03bc5cf7 test/vhost: check block size for vhost boot test
Resolves #695

Change-Id: I627b06bc32e2a3aa18fb92636da384f60fb5db2d
Signed-off-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462446
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 22:51:28 +00:00
Tomasz Zawadzki
06c7ef7f6a lib/ftl: check for io not being NULL before dereferencing
ftl_io_free() handles NULL io being passed, but shouldn't
dereference it before returning.

Change-Id: Ia705d4ab9f8b3569e9704a1f97e70f7579084efb
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462487
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:30 +00:00
Tomasz Zawadzki
8500db5ee0 lib/vmd: check header_type after the vmd_adapter structure is verified
First check both vmd_pci_device for not being a NULL,
same as vmd_adapter structure being present.

Otherwise dev was dereferenced before checking for NULL.

Change-Id: I3719f33c1762f789b0eb63b0f2ddce8258843ded
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462509
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 19:55:04 +00:00
lorneli
ba323d44ca nvmf/rdma: log spdk_nvmf_rdma_destroy_defunct_qpair
Func spdk_nvmf_rdma_destroy_defunct_qpair is a "last chance option"
to destroy qp manually if some driver/hardware doesn't drain qp's
failed wr as expected.

There's a probability that ibv_poll_cq polls wr of the destoryed qp
after spdk_nvmf_rdma_destroy_defunct_qpair's execution. Although in
practice the risk of this situation is minimal(if not non-existent),
add a log here so that we could detect this situation easily.

Change-Id: Ifa9534397513bcea34c18fbb8168eef8f53599c1
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462441
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
lorneli
b4d3066890 nvmf/rdma: defer qp destruction until nvmf layer closes qp
Currently rqpair will be destroyed directly in ibv_poll_cq path
if it has been drained, regardless of whether there are outstanding
I/Os issued to bdev layer. So after outstanding I/Os completing,
spdk_nvmf_rdma_close_qpair will be called from nvmf layer, accessing
a destroyed qp.

This path defers qp destruction in nvmf_rdma_destroy_drained_qpair
func until nvmf layer closes qp.

Fixes 851

Change-Id: I8bcce66f8053ddb105702ac603d5d73af54bdcfc
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461237
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 19:35:16 +00:00
Ben Walker
7dbdf87f63 make: Allow users to specify the target build architecture
make TARGET_ARCHICTECTURE=broadwell

Change-Id: I868c2f5c203dfc1a4e6cf48c39b391e174d5d33b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461601
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 17:01:17 +00:00
Seth Howell
8f12bc5b69 test/nvmf: disable the shutdown tests for the tcp transport.
This test is very unstable on TCP. Until we can get to the bottom of the
latent failures, we should disable it. This will allow us to continue
testing a substantial amount of tcp functionality while avoiding latent
failures in the test pool.

Change-Id: I3c7ec2709ad81a3ec5c1a4d56027b27a9c6ffc7e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462919
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-07-23 08:50:57 +00:00
yidong0635
3e19963297 nvme_qpair_ut: Fix scanbuild warning about using freed memory.
Issue reports:
spdk/test/common/lib/test_env.c:103:2: warning: Use of memory after it is freed
        HANDLE_RETURN_MOCK(spdk_zmalloc);
        ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
spdk/include/spdk_internal/mock.h:106:3: note: expanded from macro 'HANDLE_RETURN_MOCK'
                return ut_ ## fn; \
                ^~~~~~~~~~~~~~~~
1 warning generated.

Here includes test_env.c. That spdk_zmalloc will call HANDLE_RETURN_MOCK ,that leads
requesting memory in functions which call spdk_zmalloc(including nvme_qpair_init/
spdk_nvme_qpair_add_cmd_error_injection) failed.

Change-Id: I6c14dfd2fc7b7078a18a46ef8ff830f85940487e
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-07-23 04:50:29 +00:00
Mateusz Kozlowski
5094302f5a test/ftl: Changed bdev_svc usage to spdk_tgt
Changed all ftl tests to use spdk_tgt instead.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Ibb19b233f5cb855e1b6edb41145890e5b103493c
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/461213
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 04:09:32 +00:00
Seth Howell
e394fb3bed test/fuzz: Make test timeout configurable.
We should be able to change the runtime of the application based
on command line arguments without having to update the bash scripts.

Change-Id: I9fbd8e0386d7e3c39da853a89bf4aff08cb916f0
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462791
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 04:08:52 +00:00
Mateusz Kozlowski
a3c78e150b lib/ftl: Remove separate dma_buf allocation in band
Since 4k alignment is no longer required for I/O buffers, the
band doesn't need a separate lba map and dma buffer and can use the
same memory location.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Iea127e8c2f39e6de5d57258098b2dc6be56f439f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462042
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
a4b3481242 lib/ftl: Remove buffer alignment requirements
Ftl should no longer require 4k alignment for I/O buffers for either
internal or external requests. The 4k alignment was needed due
to a bug in QEMU when handling internal SGL buffers in LNVM commands.

Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Change-Id: Icafc14b7811c9c0ffa13789e341c453503cf2821
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460106
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
2019-07-23 04:00:05 +00:00
Mateusz Kozlowski
1897943987 env: define minimum alignment for dma-able memory
DPDK defines the minimum alignment as "suitable for any
kind of variable (in the same manner as malloc())", but
internally the alignment is always rounded up to the
cache line size, even if the requested alignment is 0.

We would like to start relying on this behavior in FTL,
where lba maps are allocated using DMA-able memory and
are constantly looked up or modified by different threads.
By having the lba maps unaligned, we risk having those
threads pollute each other's cache lines.

Rather than enforcing this memory alignment in FTL, we
do it in spdk_*malloc directly. In general it makes
sense to have DMA-able memory always cache-line-size
aligned for the same reason as above.

Change-Id: Ib6edda4a7bf3f4952eb1875a4e1753be96bed642
Signed-off-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/460329
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-07-23 04:00:05 +00:00
yidong0635
e3b90fe2d5 bdev: Fix warning about potenial memory leak about histogram.
Pointer passthrough g_histogram and histogram implicitly ,but the checker
still complains about this, change to spdk_histogram_data_free(histogram)
directly, no need any assert any more.

bdev_ut.c:1486:2: warning: Potential leak of memory pointed to by 'histogram'
        poll_threads();
        ^~~~~~~~~~~~~~
1 warning generated.

Change-Id: I609c1c92643ad03b8e3c62cd8434882e7942efa4
Signed-off-by: yidong0635 <dongx.yi@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462789
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 03:50:05 +00:00
Seth Howell
a78b7bb18c test/nvmf: avoid running bdevperf on top of nvmf_tgt
When you run verify on bdev perf, it rejects any user provided core
masks and pins itself to core 0. We were also running the NVMe-oF target
on cores 0-3. Occasionally I was seeing the shutdown tests failing
during the waitforio step. It seemed like the target and bdevperf were
both still online during this failure, so my hypothesis is that the
bdevperf poller was stealing all of the cycles from the NVMe-oF poller
completing I/O so the I/O wasn't completing.

Even if that's not what's happening, it is probably a good idea to split
the NVMe-oF application and bdevperf onto different cores.

Change-Id: Ib3b5b00e639ebd14bd1ed2cfb4b7782076ca364c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462551
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-07-23 03:48:08 +00:00
Jesse Grodman
53c9f4350a docs: update concurrency.md regarding thread creation
Fixes #issue number 866

Signed-off-by: Jesse Grodman <jgrodman@gmail.com>
Change-Id: Ia102fbb285ef8914008d1a92347080953d394d2a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462878
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Chunyang Hui <chunyang.hui@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 01:28:05 +00:00
x00267701
a50246fb64 log: optimize spdk_log() when logging is disabled
If the log level of the output log is higher than
the process's log level, the system does not output
it, so we needn't generate the formatting logs.

Change-Id: I36be0e6807ed575fcbf1d0ae01f064a6ca2c4539
Signed-off-by: Huiming Xie <xiehuiming@huawei.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/462790
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-07-23 00:42:24 +00:00