Commit Graph

3713 Commits

Author SHA1 Message Date
Tomasz Zawadzki
2fbeb7ea52 ut/compress: remove rte_eal_get_configuration stub
This went unused in the unit tests.
Tested with DPDK 19.08 and DPDK 19.11.

Change-Id: I738919379b5751697f9533f72fbaf77993cb6fb5
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477840
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
2019-12-17 09:29:39 +00:00
Shuhei Matsumoto
3c7890d24f lib/iscsi: Move iscsi_conn_handle_queued_datain_tasks() from iscsi.c to conn.c
Operations to queue iSCSI tasks are in iscsi.c and conn.c and cross
references due to this separation makes us difficult to create unit
tests.

This and subsequent patches will try to disentangle cross references
by moving some functions from iscsi.c to conn.c.

This patch moves spdk_iscsi_conn_handle_queued_datain_tasks() from
iscsi.c to conn.c.  For unit tests, we don't add anything new in
this patch and just create necessary simple stubs.  After code
movement, new unit tests will be added.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: If5b8501a1ef7ea53682a3437c7eb2375aa52ee3b
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477416
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
ff7e7c83e6 ut/iscsi: Replace trace library link by stubs
Subsequent patches will change trace functions to be called in
unit tests.  Using stubs is enough as done in other libraries
and do the change in this patch.  For iscsi_ut.c, add stubs to
not iscsi_ut.c but common.c to cover other files.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ia14bc932f451b14bdb0ba6e0cb949463490a879e
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477414
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
d4914dee06 ut/iscsi: Initialize struct to zero by not memset but "={}" in tgt_node_ut.c
Using "={}" at definition is clearer and saves lines.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I75a99583f1ecc84d4c47cacd9ce58034edb4e756
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477413
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Shuhei Matsumoto
ce3b8472ad ut/iscsi: Initialize struct to zero by not memset but "= {}" in iscsi_ut.c
Using "= {} at definition is clearer and save lines.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I410279bd3640e03aa9b6eeb488be4404d110b0bd
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477412
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:29:10 +00:00
Konrad Sztyber
1ad438311e bdev/ocssd: request resubmission
If a spdk_bdev_io cannot be completed, because it is sent to a zone that
is currently busy, reschedule its submission.  This mechanism will be
used by appends, as mutliple append commands can be directed to the same
zone at the same time.

Change-Id: I60da2bd1835380812d22536ea275fb8fed9f8561
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477437
Reviewed-by: Mateusz Kozlowski <mateusz.kozlowski@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
153fbcb1f8 bdev/ocssd: JSON configuration
This patch adds support for saving JSON configuration of the OCSSD
bdev module required to recreate the current state.

Change-Id: Iedbdb8b4a2b7dd02a223ce6f073553e71b9c040a
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/469090
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
88406f3302 bdev/ocssd: retrieve chunks' info during initialization
OCSSD bdev needs to keep track of some portion of zone's state (write
pointer, its capacity, whether it's busy) in order to support the append
operation.  To achieve that, all the info needs to be retrieved for all
of the chunks and translated to zone format.

Change-Id: Id88282576aba337b50e7c7e1b9e9ff5aba25a3db
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468937
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
7ac802e848 bdev/ocssd: get zone information command
Added zone information command translating Open Channel's chunk
information to spdk_bdev_zone_info structure.

Change-Id: Ifdb15f2e0ea2fb8422810fa64f18942fcb6e4582
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/468213
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-17 09:28:35 +00:00
Konrad Sztyber
704830b3b7 bdev/ocssd: zone reset command
Added zone reset command.  It's very similar to a regular data set
management deallocate command with the only difference being that it
operates on zones instead of blocks.

Change-Id: Iae4d77c6877b6906231336b13fad1167de149f20
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/467920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-17 09:28:35 +00:00
Karol Latecki
b9395621ae test/vhost: restore possibility to use custom Qemu binaries
Use env variables to select custom qemu-img and qemu-system-x86_64
binaries. This is needed to test versus latest Qemu versions from
Github.
If env variable QEMU_BIN is not present then the defaults for test
are system-wide installed packages.

This partly reverts changes done in patch:
5502294d76
which removed possibility to run with custom Qemu binaries.

Change-Id: I8138a079ba7013dcd97aad95ae2eed6417f5b18c
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477776
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-16 12:34:47 +00:00
Ben Walker
5d497f6cf5 nvmf/tcp: Use writev_async for sending data on sockets
This eliminates the flushing logic, simplifying the tcp
transport.

This also happens to greatly improve performance, especially
on random read tests. The batching done in spdk_sock_writev_async seems
to be more effectively than the previous batching logic in the tcp
transport.

Change-Id: Id980ac6073e380dc75f95df3f69cb224f50fb01b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470532
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-16 12:34:02 +00:00
Ben Walker
7db1ed8b67 sock/posix: Use sendmsg instead of writev when flushing
This is preparation to add flags to the call.

Change-Id: I36109c069b42bd3ec22e023079c62d41a435f44c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471771
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-16 12:34:02 +00:00
Ben Walker
ab22d249e2 sock/posix: Add a pending list for asynchronous requests
Add an additional queue for requests that have been sent on the network
but aren't complete yet. As of this patch, the code
is still calling writev with no flags in the POSIX layer, so it completes
synchronously. That means requests pass through this new pending list
only very briefly inside of one function.

Change-Id: Iaab6efc118a6d5fe9589199515eb3a7293db4b8e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/471768
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-16 12:34:02 +00:00
Ben Walker
b7ad942612 sock: Add an asynchronous writev
Add spdk_sock_writev_async for performing asynchronous writes to
sockets. The user of this call is responsible for allocating their own
spdk_sock_request structures to pass to this call.

spdk_sock_writev_async will not return EAGAIN and will instead leave the
requests queued until they are fully sent or aborted due to socket
error.

Change-Id: Idf3239e65d26a3024e578122c23e4fb8f95e241b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/470523
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-16 12:34:02 +00:00
Jin Yu
f1d47d69ef test: add the UT of the set IO timeout
Add the unit test to verify the submitted list,
timeout IO poller and timeout IO check.

Change-Id: Ie7832cc29e19a624328a1a96cffe313bf9f18a69
Signed-off-by: Jin Yu <jin.yu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476051
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-16 12:20:43 +00:00
Tomasz Kulasek
bd3244325c test/nvme: add cuse smartctl tests
Change-Id: Id6ed39a4e6c4d360111aea96f630bb678f5b306e
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475754
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-12 14:52:20 +00:00
Shuhei Matsumoto
54c3d1bc67 lib/iscsi: Replace DMIN32/64() by spdk_min()
The macro DMIN32/64() are local in iscsi.c.  Replace them by the
generic macro spdk_min() will improve the portability.

Replace it in test/unit/lib/iscsi/conn.c/conn_ut.c together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I5f6992b3dc091cd748b4e138810fb01761a1ab24
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477202
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-12-12 14:41:41 +00:00
Shuhei Matsumoto
4c577b1fce lib/iscsi: Change spdk_iscsi_is_deferred_free_pdu() to private and rename it
spdk_iscsi_is_deferred_free_pdu() is called only in a single
place of conn.c.  So change it to private in conn.c.  Additionally,
iscsi_is_free_pdu()_deferred() may be a little more meaningful
and rename to it.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Ic1e3d7ff435c454f40e81f9a4f90fe76589ec7b2
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477189
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-12-12 14:41:41 +00:00
Shuhei Matsumoto
62d14e2ca1 ut/iscsi: Re-order operations in abort_queued_datain_task_test()
Re-order operations in abort_queued_datain_task_test() to align it
with comment, and fix a type error.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Idde4726bfe370ad742b2de67e0786aebb7237d45
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477409
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-12 14:41:41 +00:00
Shuhei Matsumoto
19eb63fcad lib/iscsi: Stop splitting immediately when aborting SCSI read I/O
We changed read I/O submission to stop splitting when detecting LUN
hotplug very recently. We can do the same refinement for read I/O
abortion.

In _iscsi_conn_abort_queued_datain_task(), set all remaining length
to the new task and complete it immediately. We keep the code to
process the case that queued_datain_task completed but is still in
queue, but we can change its if condition to assert.

Simplify the corresponding unit tests accordingly, and set
task->scsi.transfer_len in abort_queued_datain_tasks_test() to
exercise the changed paths.

In iscsi_pdu_payload_po_scsi_read(), if task->scsi.transfer_len is not
larger than SPDK_BDEV_LARGE_BUF_MAX_SIZE, no minimum calculation is
necessary and we can substitute task->scsi.transfer_len to
task->scsi.length simply.  This change is too small to be an independent
patch and is done together.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: Iea93e51b103eae141a007a0abdaf13cbe6d5287f
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476984
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-12 14:41:41 +00:00
Ben Walker
bed4cdf6c7 nvme: Use sgls, if available, even for contiguous memory
The hardware sgl format can describe large contiguous
buffers using just a single element, so it's more
efficient that a prp list even for a single memory
segment. Always use the sgl format.

Change-Id: I9c62582829f0d64dcd1babdbc48930ddb4d9e626
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475542
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-11 11:06:57 +00:00
alokkataria
6d6052ac96 env: Use rte_malloc in spdk_mem_register code path when possible
For TCMalloc regions which we register with spdk at runtime in the MMapHook, we
need to ensure that SPDK doesn't do any allocations in that path otherwise we
will hit a livelock situation. MmapHook is invoked when TCMalloc is out of free
memory and needs to get more memory from the system, for the hugepage case it
gets via mmap.

In the current code, we could end up calling malloc in the spdk_mem_register
call via the following call path.

spdk_mem_register -> spdk_mem_map_set_translation -> spdk_mem_map_get_map_1gb

To avoid this livelock situation we call rte_malloc instead which shouldn't
invoke the system allocator. Note that in try_expand_heap_primary() which is
invoked in the rte_malloc code path, we can still call malloc, so we need to
only use this when dynamic memory allocation is disabled via --legacy-mem.
It is possible in the future we could work around even this limitation,
but for now this implementation will be much simpler.

Have verified this change fixes the livelock condition which I was hitting in
my setup without this fix.

Change-Id: I69d0813a70da1f26f8c4d9d8895e406c026be18b
Signed-off-by: Alok N Kataria <alok.kataria@nutanix.com>
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475943
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-11 11:05:10 +00:00
Jim Harris
396c445cb1 env_dpdk: tell spdk_mem_map_init whether legacy_mem was specified
We will use this in a future patch to determine whether it's safe
to use DPDK allocated memory when allocating new 1gb page entries.

We could use it in this patch to decide whether or not to register
the memory hotplug handler, but there's really no harm registering
it even when it's not needed.

Ideally DPDK would provide some kind of API to query how DPDK was
configured.  In the normal case we know whether legacy-mem was
specified, but if users initialize DPDK themselves and then call
spdk_env_dpdk_post_init(), we won't know if legacy-mem was specified.
So in that case, we will just assume that it wasn't specified.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ied0e5ff777c8ee651043f46a37ce62e44bfcc5fe

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477086
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Community-CI: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-11 11:05:10 +00:00
jiaqizho
ab04d59c57 ut/tcp: add nvmf incapsule data unit test
This patch is used to test the pending buffer policy
of nvme request which has incapsule data. When
group->pending_buff queue not null , we should make
sure spdk_nvmf_tcp_capsule_cmd_hdr_handle method still
can handle the request.

relate to : issues/995

We try to simulate the following manner: If there
is req waiting for the buffer in the list, the
nvme request which has incapsule data will be
handled firstly even it is added into the
pending_buf_queue. The reason is that currently,
those command will get the buffer from its own queue.

We made these test steps:

 1. submit a tcp request into spdk_nvmf_tcp_req_process
    and make sure it pause in group->pending_buff queue
 2. submit a incapsule tcp request into
    spdk_nvmf_tcp_capsule_cmd_hdr_handle method , and
    this method will call the spdk_nvmf_tcp_req_process
    method to handle this request.
 3. check whether the incapsule tcp request is handled
    correctly. and check the group->pending_buff queue
    remain.

Signed-off-by: jiaqizho <jiaqi.zhou@intel.com>
Change-Id: I85fcbb49e309e1203b4b308115e3bfefc0fcba2a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/472665
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-10 17:33:11 +00:00
Seth Howell
5170ac8d83 test/blockdev: move configuration before tests.
This is a significant simplification to the whole test script. Note, it
does force us to run the hello_bdev example against a gpt bdev instead
of an nvme bdev directly, but I see this as an acceptable tradeoff since
it seems like the purpose of running the hello_bdev example is mostly to
ensure that the example itself is not broken.

Change-Id: I8c2a1ede8e7ece2a82222593a7ae46704cc658af
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476935
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
b9ca149ee7 test/nvmf: convert filesystem to use run_test
This test specifically gets a little more complicated looking when we do
the test this way, but the goal is to enable deterministic reporting of
test completions on a per build basis.

Change-Id: Ica36986f821337c22654fb4cc25f9a72a3d73517
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476825
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
67645f4dea test/nvmf/filesystem: explicitly use nvme0n1
This test has always relied on /dev/nvme0n1 being the location of our
test drive which means that it is the only one in the system at the time
the test is run.

So instead of trying to loop multiple non-existant nvme drives, just
call it out directly.

Also, add a todo to remove the dependence on NVMe0n1 since we could run
this test on a machine where the main filesystem is on an NVMe drive
which would break this test.

Change-Id: Ibb2448d9367a5c80d85a1c91b0c6e44e58237751
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476952
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
d29169bd77 test/iscsi: convert data_digests tests to run_test
We had very granular timing reporting on this test, but we really only
had two test cases running for the whole thing.

Change-Id: I11650e124e24e93b1f217c6b8efd2fb006284515
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476824
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
c6767000a4 test/nvmf: convert target_disconnect to run_tests
Change-Id: I8a632fb9c782b50246123f6c2d1cfd2e3f333471
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476823
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
f470cf3d0c nvmf/shutdown.sh: convert to run_test framework
This cleanns up the logs for this test and makes it easier to follow.

Change-Id: Ifc6fcea8672d4d186e3e622eddfc200fdfd076ce
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476822
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
968d4dfd65 test/vmd: convert to run_test framework
This makes the script much more readable and helps delineate test cases
more clearly.

Change-Id: I729ca0380a7173151d074ebd73ed9766c5008480
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476821
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-10 17:12:03 +00:00
Seth Howell
a0304f54f9 test/nvme: convert tests into run_test
This unifies the script and makes it easy to see what exactly we are
testing.

Change-Id: Icdbc08077ea4397bfde50493c718f99c3c499062
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476809
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
785c3f0fe4 test/ioat: clean up timing calls.
Change-Id: Ife273ac10968ad3fc19f8a46f69e50fa755ba815
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476807
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
3c87c8973c test/rocksdb: convert test cases to run_test
cleanup.

Change-Id: I8e358a1cb327215d0b3b932bd9b1d28d495501a4
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476806
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
fe4d1dce58 test/env: convert program calls to run_test
allows us to remove a lot of extra timing_* calls.

The next several commits are aimed at converting the majority of the
tests to this single framework of using the run_test suite or run_test
case function calls. Hopefully it makes it much easier to understand the
logs over time.

Change-Id: I6d46982108be469a05f70c24b373e60726aa061e
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476805
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
36ac0c4340 test/lvol: use the run_test framework directly
Change-Id: Iabec093a7cd632cf35fb163cfcc0048ed66fc856
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/477077
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
1fafd71bc3 test: add timing calls to run_test
And clean out the corresponding calls to timing_enter and timing_exit
from the various test scripts.

Change-Id: I0759417b5a529e4c3649ce04cca1799c089da278
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476804
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
2019-12-10 17:12:03 +00:00
Seth Howell
fd17f1aabc test: make run_test more robust
make sure we have enough arguments.

Change-Id: I76ce35635ef14289061323ee401d93d8d081888c
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476801
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
b8042c3850 test: use human readable names in test logs.
Change-Id: I163a7e08bfeff3da2ee627d9fad9f2f1a249c593
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476820
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
2019-12-10 17:12:03 +00:00
Seth Howell
3710048472 test: add a test_name param to run_test
This will allow us to use timing_enter and timing_exit directly
inside the run_test function. That function already lends itself well to
nesting the way we do our timing.

This patch series is aimed at combining the timing_*, run_test, and
report_test_completions calls all into a single place. This will greatly
reduce the number of lines of code in our bash scripts devoted to
tracking timing, formatting, and test completion. It will also enable us
to expand on the reporting of test completions. Further down the line,
this will also allow us to unify test case documentation.

Change-Id: I8e1f4bcea86b2c3b88cc6e42339c57dfce4d58f2
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476799
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-10 17:12:03 +00:00
Tomasz Kulasek
716f5e77d0 test/nvme-cli: check if cuse device is destroyed after detach
Change-Id: If1fdafd282c898ccd4e7427d575507c1df4bd778
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/473294
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-09 17:40:35 +00:00
Tomasz Kulasek
b7b45bc7bc lib/nvme: remove device name parameter from nvme cuse
This patch removes posibility to set cuse device path. Instead
"/dev/spdk/nvme*" path is used.


Change-Id: I7c3087772a3661eebe03fce21356c35cc8204b49
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/474598
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
2019-12-09 17:40:35 +00:00
GangCao
5d106e75e0 net: add the check for the add and delete IP operation
To properly return the error case when adding an existing
IP or delete a not existed IP. Proper test case is also
updated.

This is also to fix below issue:
https://github.com/spdk/spdk/issues/992

Change-Id: Ia4d3af8cc86d9bdb66b18a165510cd08f9bfa555
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476543
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-09 17:39:50 +00:00
Karol Latecki
a34d7d7b2f test/nvme: wait after reloading nvme module
Make sure that all setting files under /sys/block/nvme*
have time to get created properly.

Change-Id: I9e3bf973fc09d2f5c38be3b29aa1fea4137c0402
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476557
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-09 17:39:22 +00:00
Karol Latecki
cfb6aec8ab test/nvme: fix typo in performance script
It's "iostats", not "io_stats".
Remove extraneous "$" from variable substitution.

Change-Id: I287797a8ab44032ce5c32ee48efdec52bba59f53
Signed-off-by: Karol Latecki <karol.latecki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476556
Reviewed-by: Maciej Wawryk <maciejx.wawryk@intel.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-12-09 17:39:22 +00:00
GangCao
841b7a786e test/vhost/fuzz: remove the unncessary killproces
Change-Id: Ida9b2d6aade8ab12004fccd8830452c93eca30eb
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476702
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Liang Yan <liang.z.yan@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell@intel.com>
2019-12-09 17:38:46 +00:00
Changpeng Liu
f4e3f59e0e nvme: fix potential memory leak when there is controller scan failure
The nvme_transport_ctrlr_scan() may return failure while there are
multiple controllers, so the probe context's init_ctrlrs list may
not null for this case, so when free the probe context, let's ensure
there is no controller in the init_ctrlrs list.  Also added a UT to
cover this case.

Fix issue #1095.

Change-Id: I4d9a10ad73cf00bbe159edd1f5b919797333feb6
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476969
Community-CI: Broadcom SPDK FC-NVMe CI <spdk-ci.pdl@broadcom.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-12-09 13:58:01 +00:00
Seth Howell
61537a190e nvme: replace nvme_qpair_state_equals.
nvme_qpair_get_state fits more closely with the semantics in other
modules.

Change-Id: I6ea8e02abe27253d9b4d779a43ac1963be56356a
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/476920
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Alexey Marchuk <alexeymar@mellanox.com>
2019-12-09 13:55:41 +00:00
Seth Howell
3911922005 nvme: remove redundant transport_qp_is_failed checks
The qpair state transport_qpair_is_failed is actually equivalent to
NVME_QPAIR_IS_CONNECTED in the qpair state machine.

There are a couple of places where we check against
transport_qp_is_failed and then immediately check to see if we are in
the connected state. If we are failed, or we are not in the connected
state we return the same value to the calling function.

Since the checks for transport_qpair_is_failed are not necessary, they
can be removed. As a result, there is no need to keep track of it and it
can be removed from the qpair structure.

Change-Id: I4aef5d20eb267bfd6118e5d1d088df05574d9ffd
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/475802
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-12-09 13:55:41 +00:00