Commit Graph

6379 Commits

Author SHA1 Message Date
Piotr Pelplinski
ddb17216a7 nvmf: provide ctrlr to spdk_nvmf_ctrlr_identify_ns
Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I90f68e9dbe00854f569f6574b9c89508f89caa43
Reviewed-on: https://review.gerrithub.io/428876
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:47:19 +00:00
Karol Latecki
1250a41664 scripts/perf: fix nvme perf scripts
Due to change of defauly Python interpreter to Python3
we need to decode byte object from check_output()
to utf-8, otherwise there is an error.

Change-Id: I83e2d79ec8c3934c5c6d00768288fbb4c5a50914
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/428172
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-11 20:46:18 +00:00
Seth Howell
5faf653f0c pkgdep.sh: Fix error handling for ipsec submodule
If the ipsec submodule is registered to spdk, an empty intel-ipsec-mb
directory will be created. We could potentially try to run make inside
of this empty directory, so instead do a preemptive submodule update.

Change-Id: I367fdef468bf21ef91b8354155d199cea97c3daa
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428404
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-11 20:45:47 +00:00
Ben Walker
e3cd058cb4 nvme: Don't alias sgl1 when building request
This makes this particular function consistent
with all of the other functions in this file, and
I feel it is slightly more readable.

Change-Id: I99ace5b9eb45b0f706ca85a64b155444f45c9815
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/428730
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:45:27 +00:00
Ben Walker
6b2fb186f6 nvme: Make it clearer which RDMA sgl element is being filled out
While more verbose, this makes it much more obvious that
an array of SGL elements is being filled out.

Change-Id: I98b8e5d46af32c5d7dbb990e267fdfd594942081
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/428729
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:45:27 +00:00
Ben Walker
3fe0db6ca3 nvme: Correctly set RDMA num_sge value for every command
Also add some comments.

Change-Id: I97c3a44f97aa3dadc114005c10bec83ae75994cf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/428728
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:45:27 +00:00
Ben Walker
b99136081f nvme: Add clarifying comments when building RDMA SGLs
Change-Id: I9e0fc92e422de3fc65c5048a63f4c7dcc46f7324
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/428727
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:45:27 +00:00
Liu Xiaodong
c10f8e160e jsonrpc-client: add new C client library
It's a C libary for client to call rpc method.

Change-Id: I5378747bd9dab83a41801225ba794b3910d1f5a5
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424061
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 20:07:44 +00:00
Lance Hartmann
8337cc2d0c env_dpdk: Eliminate unneeded start group ld opt
The ENV_LINKER_ARGS was employing both the linker --[start|end]-group
and --[whole|no-whole]-archive options around the DPDK_LIBs. With
the use of whole/no-whole, the start/end bracketing is unnecessary.

Change-Id: I97a2ac22df8c6b48ba674b9b292f5eea01823901
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/428737
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 20:05:29 +00:00
Darek Stojaczyk
f4cb6c90b4 memory: register each hotplugged page separately
The problem with registering the entire hotplugged memory
region is that it won't necessarily be unregistered in one
go. Registering each hugepage separately solves that
problem.

This puts a limitation on the number of pages that can
be allocated when using RDMA. We'll hopefully lift this
limitation sometime in future - probably levereging
ibv_rereg_mr, but for now we'll have to resort to either:

 a) using 1GB hugepages
 b) preallocating memory (with [-s|--mem-size <size>] app
    param) as it will be registered as just one region no
    matter what size it is. This memory won't be returned
    to the system until the SPDK app exits.

Change-Id: I6de997fb4901b772730ba6fe995dcc0640b85749
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428716
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
3372a72ca9 memory: allow unregistering multiple adjacent regions at once
All mem_maps will still receive separate unregister
notification for each registered region, but the
public memory unregister API is more flexible now.

This follows the VFIO_TYPE1v2_IOMMU interface, which
allows the same.

Change-Id: Ifc008afdc6bff39d9b3b4c892c379ade10c3098e
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428715
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
08039550d9 memory: don't allow unregistering a part of the registered region
Added sanity checks to prevent unregistering a memory
range that wasn't registered as a one, complete region.

Change-Id: I819b57560b2e48b0802113ffff9f72949d7a148a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/425556
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
a33e0943b0 memory: forbid registering a memory region more than once
Removed the reference count from the registrations map.

Although technically supported, registering a single memory
region more than once had a lot of unhandled cases and could
easily lead to a segfault.

RDMA maps require all memory to be unregistered in the same
chunks the memory was registered, which is often impossible
to achieve if a region was registered more than once:

1. register region    0x0 - 0x3 -> it gets mapped to
                                   a single ibv_mr
2. register region    0x1 - 0x2 -> nothing happens, this region
                                   is already registered
3. unregister region  0x0 - 0x3 -> 0x0-0x1 gets unregistered as
                                   one region. 0x2-0x3 gets
                                   unregistered as another
                                   (leading to segfault in the
                                   the current RDMA implementation)

The problem is that the last two regions share the same ibv_mr,
which SPDK tries to free twice. The second free causes a segfault.
vtophys map handles this case by registering each 2MB chunk
separately, but this solution cannot be applied for RDMA, as
NICs put a limitation (~2048) on the number of regions registered.

Another option is to keep a refcount of each ibv_mr allocated,
and free it only when the entire region was unregistered from the
SPDK mem map. This is however very tricky and RDMAmojo mentions
that freeing a memory buffer before unregistering its ibv_mr
may lead to a segfault.

Change-Id: I545c56e24ffa55bda211dea22aeb8a55d9631fe5
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/426085
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
6bcd929500 memory: do not merge adjacent registrations
Prevents us from unregistering two or more separately
registered regions with a single notification.

This fixes an ibv_mr leak in RDMA. When multiple registrations
were unregistered with a single notification, only the first
ibv_mr one would be freed and the remaining memory would
possibly still remain DMA-able.

As of now, unregistering multiple complete regions with
a single unregister call is not possible. It will be
implemented later, after the rest of the code is cleaned up.

Change-Id: I7d61867fa61fd7a4a8a644ff45cab17125d63e1b
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/425555
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
6c30e783c1 ut/memory: refactor mem_map_alloc() tests to CUnit tests
Moved them from test/env/vtophys to test/env/memory
because that's where they should land in the first
place. Our memory ut already test allocating a mem_map,
so just extend them with an extra test case now. Since we're
a unit test rather than a fully-fledged SPDK app, we can
simplify the code a lot now - we no longer have any memory
(hugepages) registered at the beginning of our test case,
hence we no longer need to alloc multiple dummy mem maps to
iterate through all registrations - we can simply hardcode
and predict which registrations are there.

Change-Id: I82cd00ea2ad2370bdc63846874885f8c55e11d53
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428714
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
cb9f0f3354 memory: fix initial mem_map notifying at vaddr 0
Newly registered mem maps weren't notified at vaddr 0
as we used value "0" to denote an unitialized, unset
variable. Since we *do* register vaddr 0 in our memory
unit tests let's switch unitialized vaddr value to
something different - like UINT64_MAX.

Change-Id: I9c902165e76155e068642abb9a656f3ae8ca1105
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428713
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-11 18:59:11 +00:00
Darek Stojaczyk
0d57730e29 memory: fix segfault when freeing mem map with no notify_cb
In spdk_mem_map_alloc() we only do the memory walk when
notify_cb is provided, but spdk_mem_map_free() does the
memory walk undonditionally. Not anymore.

Change-Id: Ic8dfdc5cb2c99dc58e62ab0523cf5a18ba8691cc
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/428722
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-11 18:59:11 +00:00
Seth Howell
2b6d9ce21f configure: Move checks for nasm and ipsec
The way they were previously being checked was triggering the trap and
printing out a "Configuration failed" message even though the
configuration was successful.

Change-Id: I9de4f390c603631ebf5af5555ea7164aae2b6213
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428663
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-11 18:50:27 +00:00
Ziye Yang
5c263046db sock: Update the spdk_sock_getaddr function.
Purpose: We need to get the port info in other applications
(e.g., NVMe-oF TCP/IP transport)

Change-Id: I3a4636e764e44425436bb064cb0062c6f3e44035
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/428313
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-11 04:58:49 +00:00
wuzhouhui
5d5181db70 nvme/rdma: fix a stack-buffer-overflow error
spdk_mem_map_translate() dereference a uint64_t * to get a
8-bytes long integer, but nvme_rdma_build_sgl_request() just passes
a 4-bytes long integer as last parameter, this causes a
stack-buffer-overflow error.

Reported in 3ba5ea9087.1539172863/fedora-05/build.log

Change-Id: Id1cda22114fef466dbb930b502e3a68310331f0e
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/428693
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-11 01:58:48 +00:00
Shuhei Matsumoto
5616c1ed9c bdev: Change split IOV submission from sequential to batch
Large read I/O will be typical in some use cases such as
web stream services. On the other hand, large write I/O
may not be typical but will be sufficiently probable.

Currently when large I/O is submitted to the RAID bdev,
the I/O will be divided by the strip size of it and then
divided I/Os are submitted sequentially.

This patch tries to improve the performance of the RAID bdev
in large I/Os. Besides, when the RAID bdev supports higher
levels of RAID (such as RAID5), it should issue multiple
I/Os to multiple base bdevs by batch fasion in the parity
update. Having experience in batched I/O will be helpful
in the future case too.

In this patch, submit split I/Os by batch until all child IOVs
are consumed or all data are submitted. If all child IOVs are
consumed before all data are submitted, wait until all batched
split I/Os complete and then submit again.

In this patch, test code is added too.

Change-Id: If6cd81cc0c306e3875a93c39dbe4288723b78937
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/424770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-10 17:19:32 +00:00
Ni Xun
08ed55bb1d gpt: add support for bdev_io_wait when no memory
according to commit:
     bdev: add spdk_bdev_queue_io_wait
This patch will make io_wait to support gpt

Change-Id: I73795756f232f1b596b65ecbc7e3ba4d990c9fe0
Signed-off-by: Ni Xun <nixun@baidu.com>
Signed-off-by: Li Lin <lilin24@baidu.com>
Signed-off-by: Zhang Yu <zhangyu31@baidu.com>
Reviewed-on: https://review.gerrithub.io/428338
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-09 23:17:32 +00:00
wuzhouhui
3785a4d83b bdev/qos: fix a heap-use-after-free error
When destroy qos, spdk_bdev_qos_destroy() allocates a new qos, and swap
old one. After spdk_bdev_unregister() frees the new qos, the old qos poller
may still reference new qos via bdev->internal.qos. Fix this error by
using old qos in _spdk_bdev_qos_io_submit().

Reported in 72aac51430.1539054028/ubuntu16.04/build.log

Change-Id: Id1bce6c8b1cefae604dd2c69e8f3482ec34b1b54
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/428444
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 22:17:03 +00:00
Seth Howell
346fefc364 shared_lib: add as_needed to the libspdk.so linker script
specifying as_needed in the linker script prevents us from creating more
runtime dependencies than necessary for a given application linking to
libspdk.so

Change-Id: Ifd3788e210c5e6a0e19749cb575c3dfb494a41d2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/427808
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 21:45:26 +00:00
Seth Howell
89ef21520d Make: Add deps for each shared obj individually
This prevents the SPDK build from requirng unnecessary dependencies when
not compiling certain features. Also, fixes github issue #434

Change-Id: I7d0520474f3656ae32670313f2290e6b741c5ca8
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426131
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-09 21:45:26 +00:00
Seth Howell
c779298802 Make: Build shared libs while building static libs
This will allow us to specify more granular system dependencies for each
.so file.

Change-Id: I759876e09ffb36380a503962b4be8fc9d7c71b63
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/426129
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-09 21:45:26 +00:00
Seth Howell
460cf3e62b test/unit: add nvme_rdma.c unittests
Change-Id: Ia7228bfdfae4df53a68d77d41b643c107a73e832
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/428116
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-09 21:17:17 +00:00
Seth Howell
935cdbe497 lib/nvme: Enable multi SGE support in the NVMe-oF host.
Change-Id: Icbbea0d586ae086314085e682fc13aa63fa3c167
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/427543
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 21:17:17 +00:00
paul luse
16f9d58ab5 doc: small update to iscsi bdev readme
Added reminder to build with the iscsi configuration option.

Change-Id: I1e4e325394368f2bc3462761a3793f4b0edb065c
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/428397
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 21:15:56 +00:00
GangCao
868c28cd13 QoS/Bdev: add the RPC support for the bandwidth rate limit
This patch added the support of RPC method to enable,
adjust, disable the bandwidth rate limit on the bdev.

And it can work together with the existing IOPS rate limit.

The RPC method has been consolidated to support both IOPS
and bandwidth rate limits as below:

usage:
rpc.py set_bdev_qos_limit [-h]
                          [--rw_ios_per_sec RW_IOS_PER_SEC]
                          [--rw_mbytes_per_sec RW_MBYTES_PER_SEC]
                          name

positional arguments:
  name       Blockdev name to set QoS. Example: Malloc0

optional arguments:
  -h, --help show this help message and exit
  --rw_ios_per_sec RW_IOS_PER_SEC
             R/W IOs per second limit (>=10000, example: 20000).
             0 means unlimited.
  --rw_mbytes_per_sec RW_MBYTES_PER_SEC
             R/W megabytes per second limit (>=10, example: 100).
             0 means unlimited.

Change-Id: I9c03cd635280add01801a81c6a6c02f0cf85bee1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/416511
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:26:44 +00:00
Changpeng Liu
d9ecb5724e nvme: broke up NS construction with extra states
Change-Id: I4e95e6283283be48cc8682a5e18a84618e2f34d9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/425704
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:09:32 +00:00
Changpeng Liu
5b0e370bf6 nvme: broke up nvme_ns_construct() with IDENTIFY functions
Change-Id: I78f0a5ada3a0ec5e20d6e80f7442bac0c6461fce
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/425703
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-09 18:09:32 +00:00
Mike Altman
a077b477ca ut/log: test print level set and get functions
Coverage report indicated these were not currently
covered.

Change-Id: Ic67f1d6321c39c59ede8a379051e238d3b0fecb6
Signed-off-by: Mike Altman <maltman@scalecomputing.com>
Reviewed-on: https://review.gerrithub.io/428447
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-09 18:08:49 +00:00
Ziye Yang
5c9dccc962 nvme: fix the parse of spdk_nvme_transport_id_parse
Normally, there will be only one separator in transport id,
for example, either ':' or '='. But the users may input
this: trtype=PCIe traddr=0000:81:00.0.

Thus, there will be two diffrent separator '=' and ':',
and our function doest not handle this case correctly.
And this patch can fix this issue, and also update the
test case.

Change-Id: Ic3f10dc1e37c66647fede37c5cf9523fc2652677
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/428307
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2018-10-09 07:38:21 +00:00
chenlo2x
93ee8c7a7f scripts/check_format: fix an escaping issue.
Some older versions of Fedora require putting the git grep contents
in quotes before feeding to the while loop.

Change-Id: I649d5520f4a8c0a903aed2e2dda337e91b5b596e
Signed-off-by: chenlo2x <longx.o.chen@intel.com>
Reviewed-on: https://review.gerrithub.io/428332
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-09 04:45:00 +00:00
Jim Harris
7c56c393ba bit_array: return UINT32_MAX if no cleared bits found
spdk_bit_array_find_first_set() returns UINT32_MAX if no
set bits are found.  But spdk_bit_array_find_first_clear()
would return the size of the bit array instead in this case.
(Note: the comments say size of the bit array + 1 which was
incorrect)

So this patch makes spdk_bit_array_find_first_clear()
consistent with spdk_bit_array_find_first_set() and returns
UINT32_MAX if no cleared bit is found.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I44c1e674149f8c2e87122800d5db45a2851f0bef

Reviewed-on: https://review.gerrithub.io/428225
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2018-10-08 17:03:33 +00:00
Pawel Kaminski
76c5ec4ca5 test/spdkcli: Fix. Add shebang line
Change-Id: I89a50f262a088a50fddcd1515cc2151013e89e08
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/428243
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-08 16:27:13 +00:00
Ziye Yang
b8c1262ed8 test: fix the test error reported by the testpool
Remove the unnecessary line

Change-Id: I0fd23cca90bb9c785c1a2c0c4abdc7b29d0acc60
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/428314
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: GangCao <gang.cao@intel.com>
2018-10-08 05:30:49 +00:00
Shuhei Matsumoto
a8780177c6 ut/bdev: Exercise the ENOMEM path in the IO splitting
Splitting IO in the bdev layer is a little complex and increasing
the coverage in unit tests will be good.

Change-Id: I9d42cedc795b2804a472bdc310e936135c424fd3
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427948
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-05 21:57:34 +00:00
Shuhei Matsumoto
fb0fe6b1be ut/bdev: Fix scan build error for descriptor got by bdev_open
Change-Id: I2a7399dc8ec4b67eee6966d9bc3543468c76567f
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427975
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 21:57:34 +00:00
Shuhei Matsumoto
8f33ef2475 ut/bdev: Setup all expected values before starting tested split I/Os
Expected values are managed by linked list now. Hence we can set
up all expected values before starting tested split I/Os.

This change will make the logic clearer.

Change-Id: I9226619468220132aebd55a5da36490a145c128a
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427947
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 21:57:34 +00:00
Shuhei Matsumoto
8a295129e1 bdev: Fix scan build error by checking the limit of parent iovec
scan-build requests to check the size of parent iovec by using
artificially large LBA in unit tests.

Fix the error by using not pointer but position and checking if
position is less than count of parent iovec.

Change-Id: I74c4f6d1b68ecfca93e9247acc5ac6bd5412a960
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/427965
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 21:57:34 +00:00
Darek Stojaczyk
803449ca7b env_dpdk/memory: handle spdk_mem_map_notify_walk() failures
If during mem_map creation we managed to register only a part
of the total registered memory and then failed in the middle,
we will now undo our registrations and gracefully return
an error code leading to a spdk_mem_map_alloc() failure().

Change-Id: Id549f35109bd381318adc676392d9ee26d035359
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/423538
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-05 21:01:27 +00:00
Pawel Niedzwiecki
598d3cfbd3 test/spdkcli: Add a match test to iscsi spdkcli test
Change-Id: Ic48d0f2abb0455c0a983e4d93506935fb2339509
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/427356
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 17:48:09 +00:00
Pawel Kaminski
7d79e5b3a9 test/spdkcli: Add spdkcli tests for iscsi node
Test to create and delete full configuration for iscsi node.

Change-Id: I1c09741bf5afcc7cf9ce58c08973439088ea3ba4
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/425179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 17:48:09 +00:00
Pawel Kaminski
3d0488d28a spdkcli: Add iscsi subsystem.
Add create/delete methods from iscsi subsystem to spdkcli.

Change-Id: I950b3f29720fd5f09e4144d139e6d0e5885c69c8
Signed-off-by: Pawel Kaminski <pawelx.kaminski@intel.com>
Reviewed-on: https://review.gerrithub.io/424607
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Paweł Niedźwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-10-05 17:48:09 +00:00
Piotr Pelplinski
c3d9530cd0 spdk: fix check_format.sh
Patch 427539: Check file permissions in the check_format script
introduces at least two errors.
- exit too early from check_format.sh
- wrong file permisions on file vbdev_crypto.c
This patch fixes them.


Signed-off-by: Piotr Pelplinski <piotr.pelplinski@intel.com>
Change-Id: I346523bce7448e749471bd78a06e9233e0d35ef2

Reviewed-on: https://review.gerrithub.io/428188
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-10-05 16:02:48 +00:00
Jim Harris
b3c330339d bit_array: clarify initial bit values for create()
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: If0ee1ebfcf3400efcf79185adf55dc9308f06fc4

Reviewed-on: https://review.gerrithub.io/428079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-10-05 00:04:15 +00:00
wuzhouhui
74a28c199f vhost: rpc: add an optional parameter -n/--name for get_vhost_controllers
User can use this optional parameter to get the information of specific
vhost controller.

Change-Id: I3911c6c7d4e7b75e82277d1e4690d5e40019aa06
Signed-off-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-on: https://review.gerrithub.io/425451
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-10-04 23:33:04 +00:00
Pawel Niedzwiecki
90051287d8 test/vhost: test vm booting from vhost controller
Boot os from a vhost controller using lvol bdev on a device with
512B block size. It performs fio integrity test.

Change-Id: Ic30a5dc33d9dd072559d9101f31e22e9e96862a7
Signed-off-by: Pawel Niedzwiecki <pawelx.niedzwiecki@intel.com>
Reviewed-on: https://review.gerrithub.io/421758
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Piotr Pelpliński <piotr.pelplinski@intel.com>
2018-10-04 23:27:40 +00:00