5372 Commits

Author SHA1 Message Date
GangCao
9aed5fa5b6 iscsi: update the version error message
Change-Id: I3a2488cdd81812f1fce136b97e49eee18b49fbc1
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456915
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-06 00:59:57 +00:00
Ziye Yang
679257db88 nvme/tcp: Properly deal with supporting single r2t
According to the TP 8000 spec in Page 26:
Maximum Number of Outstanding R2T (MAXR2T): Specifies the maximum
number of outstanding R2T PDUs for a command at any point in time
on the connection.

This patch makes the current host driver implementation support one r2t.
We cleanup the code to do the right advertising to the target in the
icreq and avoid attempts to deal with multiple rt2s.

Reported-by: Or Gerlitz <ogerlitz@mellanox.com>

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: If06ad2e8bde31c2fd7e1c3739f651fb64040e3a9
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455750
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-06 00:58:58 +00:00
paul luse
83afe68690 dpdk: build and link QAT for both crypto and compression
In prep for full QAT compression support later in this patch
series. dpdkbuild/Makefile slightly refactored for readability,
x86 crypto check removed as it pre-dated checks we now have in
configure.

Signed-off-by: paul luse <paul.e.luse@intel.com>
Change-Id: Iaaaf51b9eb5e18840f47d2d4f431c5a6e8c420ee
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456408
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 19:07:32 +00:00
Li Feng
02363ccb3b lib/iscsi: fix iscsi connection schedule
If the qualified core wasn't found, then we should compare
spdk_env_get_core_count() with i, instead of lcore.

Signed-off-by: Li Feng <fengli@smartx.com>
Change-Id: Ie92f56712b7f0e51636008fe12fff5584b6be8ab
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456415
Reviewed-by: wuzhouhui <wuzhouhui14@mails.ucas.ac.cn>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 17:39:12 +00:00
GangCao
3a0886c408 iscsi: correct the error detail
According to https://tools.ietf.org/html/rfc3720#page-196,

Error detail of "Authorization failure" is for the case that
the initiator is not allowed to the given target.

Change-Id: Ie628c4c857e965aa6694399a9832ce0501e50745
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 16:48:48 +00:00
paul luse
d5d7c57043 bvdev/compress: move comp transform structs to static globals
Code cleanup.

Change-Id: I9f96a889ed39afbd14a42b50e2537b1c4844f514
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456517
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:48:36 +00:00
paul luse
5ef2b14e3c bdev/compress: rename function to avoid collision with bdev layer
Makes debug easier...

Change-Id: I3b975e0a494c3920387b2d1921ebf8089b935bf5
Signed-off-by: paul luse <paul.e.luse@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456512
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-05 16:48:36 +00:00
Ziye Yang
0bb626672b nvmf/tcp: Support single r2t usage
According to the TP 8000 spec in Page 26:
Maximum Number of Outstanding R2T (MAXR2T): Specifies the maximum
number of outstanding R2T PDUs for a command at any point in time
on the connection.

Note that by the spec, the target may only support single r2t
(which is the minimum possible), it doesn't have to use multiple r2ts
even if the initiator supports that. So remove the maxr2t and
pending_r2t variable in the tcp qpair structure.

In the original design, we think that maxr2t is the maximal active
r2t numbers for each connection. So if the initiator sends out maxr2t=16,
it means that all the commands of a qpair can use such number of R2T pdus.
So we need to wait for the available R2Ts for the request when the maxr2t
reaches the maximal value. But it is the wrong understanding of the spec.

In fact, each command has its own number of maximal r2t numbers, then we
do not need to use the wait method for R2T method anymore. So we remove
the state TCP_REQUEST_STATE_DATA_PENDING_FOR_R2T. Futhermore, we adjust
the related SPDK_TPOINT_ID definition.

In current patch, the target will support one active R2T for each
write NVMe command. Thus, we remove the function spdk_nvmf_tcp_handle_queued_r2t_req.

Reported-by: Or Gerlitz <ogerlitz@mellanox.com>

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I7547b8facbc39139b4584637ccc51ba8b33ca285
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455763
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Or Gerlitz <gerlitz.or@gmail.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 16:46:55 +00:00
Ziye Yang
fe2dddbbbc nvme/tcp: Correct nvme_tcp_qpair_disconnect behavior
The current nvme_tcp_qpair_disconnect behaviour
is not exactly correct, we do not re-initialize
the state of some data structures of the tqpair.
And this caused the coredump.

Purpose: Fixes #808.

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I4d2cad8fc0712dbebfc2f3e52373cbe3b9908bf7
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456755
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 16:13:55 +00:00
Wojciech Malikowski
f97809e3cb lib/vmd: Moved MAX_VMD_TARGET definition to public header
MAX_VMD_TARGET defines size of array that need to be passed
to spdk_vmd_pci_device_list()

Change-Id: Ib2a33fe50072036e6a8f1709ac4e3ee82c1bb3f1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456480
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 15:19:51 +00:00
Wojciech Malikowski
d3a0531bc1 lib/vmd: Removed typedefs from structures definitions
Removed typedefs and unused structures.

Change-Id: I711094097520333cbeda2f26a1a7d403750070b1
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456479
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 15:19:51 +00:00
Wojciech Malikowski
f7e68be171 lib/ftl: Minor cleanup in error path from nvme layer
Change-Id: If64d721fac28683b9b385238ae1c63aad77c0914
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451869
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 06:50:03 +00:00
Nikos Dragazis
c8202a4d79 vhost: fix vhost memory registration in case of vvu transport
The memory API has been refactored. It is not possible anymore to
register a memory region more than once. This has been introduced in
this patch: https://review.gerrithub.io/426085

In case of vhost with vvu transport, it often happens that two
consequtive vhost memory regions are mapped to virtual addresses that
lie within the same 2MB address range. This means that the vhost memory
regions may not be 2MB-aligned in the process virtual address space. As
a result, the `FLOOR_2MB()` of those addresses gives the same address.
Thus, we end up trying to register the same 2MB memory range twice.

This issue does not appear in case of AF_UNIX transport. Vhost memory
regions in case of AF_UNIX transport are hugepage backed. Therefore, the
mmapped virtual addresses of those memory regions are always
2MB-aligned. On the contrary, in case of vvu transport, the vhost memory
regions are segments of the PCI memory address space of the
virtio-vhost-user PCI device. This MMIO space is mapped in its entirety
by the DPDK vfio interface along with the other PCI BARs.  Ultimately,
the vhost memory regions correspond to offsets in this mmapped PCI
memory region and thus there is no warranty that the mmapped virtual
addresses are 2MB-aligned.

This issue is fixed by skipping the already-registered 2MB memory
regions.

Change-Id: I62c9c257e6f172c894cd3454d2cbeee1986e6189
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/441057
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 06:43:01 +00:00
Wojciech Malikowski
122a64a359 lib/vmd: Removed SUPPORT_ALL_SSDS define
SUPPORT_ALL_SSDS define is not needed.

Change-Id: I9f81dfa0da10c4c83bbc6e051cd83ac6318651b3
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-05 05:33:35 +00:00
Wojciech Malikowski
2f4e74fe95 lib/vmd: Remove not needed NULL ptr checks
Some existing NULL ptr checks was never hit so
there should be removed.

Change-Id: I80f4e1f063564f2a6915d0a1c548bcf22c4b3075
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456477
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-05 05:33:35 +00:00
Wojciech Malikowski
4d117a8e9a lib/vmd: Replace printf() usages with SPDK_*_LOG
Removed all printf() usages and replaced it with
SPDK log macros.

Change-Id: Ibe536a9bb696ebf3ad7dd1f402b050eca0dfd375
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:33:35 +00:00
Wojciech Malikowski
7a9ef7d2f6 lib/vmd: Fixed scan-build issue in vmd_pcibus_remove_device()
Added assertion checking if next element on list
is valid.

Change-Id: I9f4d969dc84e5fbee9d72d764f57fbd9480ab197
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456774
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:33:35 +00:00
Shuhei Matsumoto
57fe0394a3 iscsi: Use 0 instead of SPDK_SUCCESS in spdk_iscsi_read_pdu to match caller
spdk_iscsi_read_pdu has been used only in a place,
iscsi_conn_handle_incoming_pdus. iscsi_conn_handle_incoming_pdus
classifies return code of spdk_iscsi_read_pdu to
SPDK_ISCSI_CONNECTION_FATAL, 0, or 1.

Using 0 instead of SPDK_SUCCESS in spdk_iscsi_read_pdu matches
iscsi_conn_handle_incoming_pdus and is done in this patch.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I231c2e70a094c26af2c8eb60d77b92d880674901
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456770
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:31:22 +00:00
Shuhei Matsumoto
0f5314eb05 iscsi: Consolidate error paths in spdk_iscsi_read_pdu
spdk_iscsi_read_pdu() returns three return values,
SPDK_ISCSI_CONNECTION_FATAL, 0, or 1.

Consolidating error paths clarifies the logic.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I66a734f37bfd3bbd0ae0974813086b24cb3cd432
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456769
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:31:22 +00:00
Shuhei Matsumoto
1554a344c9 env: Add free_space parameter to spdk_ring_enqueue
DPDK rte_ring_enqueue_bulk() has free_space parameter to return
the amount of space in the ring after enqueue operation has finished.
This parameter can be used to wait when the ring is almost full and
wake up when there is enough space available in the ring.

Hence we add free_space to spdk_ring_enqueue() and spdk_ring_enqueue()
passes it to rte_ring_enqueue_bulk() simply.

Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Change-Id: I9b9d6a5a097cf6dc4b97dfda7442f2c4b0aed4d3
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456734
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-05 05:23:58 +00:00
Ziye Yang
31607f3f9e nvme/tcp: fix the user iov length caculation in nvme_tcp_build_sgl_request
The length should be no larger than the remaining_size.
For example, The remaining_size(firstly, assigned by payload_size) is 128KB,
and user's sgl length is 1MB. Since we already split the I/O, so we should
not use the original length(1MB), but use the remaining_size.

Fix issue reported by: https://github.com/spdk/spdk/issues/808

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I0a7d0f2282c8ad0e253d8de7091b6c5b87018e9a
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456760
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 01:46:52 +00:00
Ziye Yang
5391b29c79 nvme/tcp: Fix the issue of handling send pdu failure
Previously, if the return value of nvme_tcp_qpair_process_send_queue
is not zero, we directly return but not continue receiving the pdu.
But this is wrong, we should only handle the case when the
return value is negative.

Reported-by: Or Gerlitz <ogerlitz@mellanox.com>

Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Change-Id: I83453733f5a3e3350a0461b4cb0bc409fde32fea
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455899
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-05 01:44:49 +00:00
Seth Howell
d7ead05419 lib/ftl: fix NPA in ftl_band_reloc_free
Stumbled on this one trying to fix new scan-build errors for clang 8 on
fedora 30. scan-build fails because there are paths where this can
result in a null pointer access.

Change-Id: I408376e7eab79ab6d99c35cc662f5737e3518e71
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456671
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-05 00:07:18 +00:00
GangCao
9a73633190 iscsi: add a cleanup utility function
Added a utility function for the cleanup related operations.

Change-Id: I4e49dd9c2da899a5bda289bc36778b636af9c5df
Signed-off-by: GangCao <gang.cao@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456599
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 23:51:15 +00:00
Tomasz Zawadzki
e5b7f59d9c bdev: directly use spdk_bdev_io_get_io_channel() where needed
The function spdk_bdev_io_get_io_channel() was added,
but remained unused. Couple places done the exact same thing,
so now they just use it directly.

Change-Id: Ifa332ce3a489256ccf2f5d0dd9c74a3215c544ce
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456435
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 00:08:53 +00:00
Tomasz Zawadzki
6b395277b2 bdev: directly use spdk_bdev_io_get_thread() where needed
The function spdk_bdev_io_get_thread() was added,
but remained unused. Couple places done the exact same thing,
so now they just use it directly.

Change-Id: Ia25abf57eb88c8df47c83e4edd761a9e02cc1618
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456436
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-06-04 00:08:05 +00:00
Konrad Sztyber
c92720211d lib/ftl: decreased the size of the metadata buffer
The metadata buffer was allocated in multiples of blocks (as well as
aligned to a block). It's not necessary and is a waste of space.

Change-Id: Icb13c664e82b23ea09bec16fa3c18fa98b722d57
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455922
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 00:07:02 +00:00
Konrad Sztyber
e1dbe1c84c lib/ftl: fixed metadata pointer arithmetic
The current location of the metadata pointer should be calculated based
on the metadata size, not block size.

Change-Id: I90177557cf4e194cd77adfbfe1c5a6e0e6df7967
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455921
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 00:07:02 +00:00
Jim Harris
6550abbac1 nvme: prefetch stailq before freeing pcie request
We will need to put the recently completed nvme_request
object on the qpair's STAILQ.  We don't reference any
real data from the nvme_request in the completion path
since we've already stashed the cb_fn and cb_arg in
the nvme_tracker.  But we will need to reference the
STAILQ_ENTRY to put it back in the qpair's STAILQ, so
prefetch that cacheline.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id76122afe4150c84a61fbe38bc874f10d606b3b3

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456673
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 00:01:35 +00:00
Jim Harris
b3d884b700 nvme: assign qpair when req is allocated
There's no need to set this every time we allocate
a request.

While here, fix a typo near where we needed to modify
the unit test to remove the qpair assertion.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8af41a6c483415950f625d1ed2ef46088b75a622

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456270
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-04 00:01:35 +00:00
Changpeng Liu
4f95ec427c scsi: add a helper function to check the LUN holds a reservation or not
Change-Id: I8cf71783ad4af4f7bdef823018f907933b793c9d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455765
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-03 23:59:49 +00:00
Konrad Sztyber
d69fe7b7b9 bdev/nvme: separate metadata buffer support
Handle IOs with metadata being transferred in a separate buffer if the
device underneath supports it.

Change-Id: I38887a24d2aad51f674a840367b5dcdeda2d5a8b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451467
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-03 23:35:42 +00:00
Konrad Sztyber
8ba138e56e bdev/nvme: store bdev/bdev_io in separate variables
Change-Id: I6eb3ae54fc419f65e5bc718cc78150c99fea296b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454742
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Maciej Szwed <maciej.szwed@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-06-03 23:35:42 +00:00
Konrad Sztyber
58663640d2 lib/bdev: separate metadata buffer IO functions
This patch adds *_with_md family of functions allowing for IO with
metadata being transferred in a separate buffer.

Change-Id: I842d5a00a532cf5d0b0f0738535ea46903674140
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/451465
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 23:35:42 +00:00
Wojciech Malikowski
617a004fb9 lib/vmd: Merge vmd.c and vmd_pci.c into one file
There is no need to keep this functionality in separate
files.

Change-Id: Ie998324cbcfcdc384f912fe36124b082774c0dc8
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456475
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 23:34:05 +00:00
Nikos Dragazis
5f4e42b80b vhost: abstract vring call mechanism as it is transport-specific
vring notification mechanism is transport-specific.  At present, vhost
dataplane code in `lib/vhost/vhost.c` triggers guest notifications with
`eventfd_write()` system call. But this is an AF_UNIX specific
notification mechanism. This patch replaces `eventfd_write()` with the
existing generic `rte_vhost_vring_call()` function that is part of
DPDK's librte_vhost public API.

`rte_vhost_vring_call()` takes a vring_idx as an argument to associate
the `struct spdk_vhost_virtqueue` instance with the relevant `struct
vhost_virtqueue` instance. We introduce a new `vring_idx` field in
`struct spdk_vhost_virtqueue` to enable this association. This field is
initialized in `start_device()`. In addition, a stub for
`rte_vhost_vring_call()` is added in the vhost unit test file.

SPDK's internal `rte_vhost` copy will not be updated in order to support
the virtio-vhost-user transport. However, an `rte_vhost_vring_call()`
function is introduced in SPDK's `rte_vhost` in order to have a solid
API. This function is just a wrapper of `eventfd_write()`.

Change-Id: Ic93e25cd3f06e92f04766521bc850f1ee80b8ec8
Signed-off-by: Nikos Dragazis <ndragazis@arrikto.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/454373
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 20:14:08 +00:00
lorneli
a5dfbc4daf nvme: zero request->submit_tick in allocation
Request may be submitted several times via nvme_qpair_submit_request
function, such as request in queued_req queue being re-submitted.

With enabling timeout feature, nvme_qpair_submit_request compares
request->submit_tick to zero to check if this is the first submission
for this request. If true, record submit_tick for this reuqest.

So request->submit_tick needs to be set zero in allocation.

Change-Id: Ie3f420aa337802c5ad3962c3fdcd680dec1ccdcb
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456328
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-06-03 19:15:13 +00:00
Wojciech Malikowski
d6d710212e lib/ftl: Removed not needed assertion from ftl_submit_write()
This assert is not needed anymore.

Change-Id: I7bc85c980c2e120b105b35763b9bea5960564acc
Signed-off-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456590
Reviewed-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 10:46:27 +00:00
Konrad Sztyber
1d97387549 lib/ftl: use pre-allocated iov buffer
Keep an array of iovecs tied to each IO. Internal IOs can have
iov_cnt > 1 without having to allocate additional memory. User's iovec
can now be modified too (e.g. when splitting the request).

Change-Id: Iec73c6dad59acdc331a1461fd7feac085138a8de
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455525
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
feb2c6be42 lib/ftl: keep resubmitted writes on a queue
Replaced single IO pointer with a queue. This allows multiple requests
to be queued when the writes cannot be scheduled.

Change-Id: I0cde50e7378108a6aab4ef6fe601cb854244f5d5
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455524
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
8e6a5601e4 lib/ftl: allocate ftl_io inside ftl_io_user_init
On the user IO path, the flow always follows the same procedure:
allocate ftl_io and initialize it using ftl_io_user_init. There's no
point in separating these two actions.

Change-Id: I578347ccc7c85e5945368dd6bf76c58985af3939
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455523
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
cee780d504 lib/ftl: added device pointer to ftl_io_channel
Change-Id: I3dbbaf11e4b1472983a7c98e9ddde53a0525b4fc
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455522
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
0ee4bb7789 lib/ftl: report all errors in completion callback
There's no point in synchronously returning an error from ftl_io_write
/ ftl_io_read as they're also reported in the IO's completion callback.
All errors are now reported through io->status.

Change-Id: I4adf4e13221b63715625276042e1172cd63b8f9b
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455521
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
2b4609c55b lib/ftl: moved next_ppa to ftl_submit_read
Select the method of choosing next PPA to read inside ftl_submit_read,
as it can be deduced from other arguments.

Change-Id: I6cbd37c2c6d7f073bf735491b4960d72e327c4e6
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455520
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
1d29386066 lib/ftl: stop splitting internal IO data buffers
Some of the internal IOs split the contiguous data buffer into multiple
iovecs to separate the requests. It's unnecessary and can be replaced
by checking the position within the data buffer.

Change-Id: I9255ea0072fee6c4e6a7dca21e4008780bc45610
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455519
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Konrad Sztyber
b52a5a45d8 lib/ftl: automatically advance parent's IO position
Advance parent's IO position when children positions are advanced.
There's no need to call ftl_io_advance twice for the same request
anymore.

Change-Id: I1f7f04a3a83575b11e7bf0b13c0c8f3bf98b5522
Signed-off-by: Konrad Sztyber <konrad.sztyber@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/455518
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Wojciech Malikowski <wojciech.malikowski@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-06-03 04:39:37 +00:00
Jim Harris
da366fd09f nvme: explicitly mark _nvme_ns_cmd_rw as inline
This is a small optimization.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ib593908d3aeb17aac55be06b8e3be42e28a23061

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456268
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-06-03 03:11:08 +00:00
Changpeng Liu
9da1c7384d bdev: don't call spdk_bdev_free_io() for the error case
Existing code in spdk_bdev_write_zeroes_blocks() will call spdk_bdev_free_io()
for the error case, which will cause assertion because the bdev_io isn't
submitted to the backend yet, so we will check the condtion first to
avoid the error case.

Change-Id: If27d78217f709a3315e74c00869d345abd6b9a69
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/453491
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-05-31 18:53:48 +00:00
Jim Harris
f758598c44 nvmf: fix assert in spdk_nvmf_tcp_req_fill_iovs
It's OK for iovcnt to equal SPDK_NVMF_MAX_SGL_ENTRIES.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ic95d04f5667858e7fbb025f469c027e2d47b8ba1

Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456111
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-05-31 14:46:35 +00:00
Seth Howell
c680a79237 json_util: fix typo in debug message.
We were printing out the wrong value here.

Change-Id: I7b5f4eaca41317a69167ad5413a1b1913e9c0842
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/spdk/spdk/+/456278
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-05-31 14:39:53 +00:00