688 Commits

Author SHA1 Message Date
Changpeng Liu
bad30d5366 nvme: add the asynchronous controllers probe/poll APIs
User can create a probe context to probe and attach controllers
asynchronously, the controllers will be added to the context list
for the first step, then users can poll the context until the list
becomes empty.

Change-Id: I3a96e2d8a9724332ff15542f78f9553fdab505e2
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442664
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-15 03:14:20 +00:00
Changpeng Liu
3306e49e24 nvme: introduce probe context data structure and API
Existing NVMe driver uses a global list g_nvme_init_ctrlrs
to track the controllers during initialization, and internal
function will start each controller in the list one by one
until the list is empty.  We introduce a probe context
and move the global list into the context, with the context
we can enable asynchronous probe API in the next patch, also
this can enable parallel probe feature.

Change-Id: I538537abe8c1a4a82fb168ca8055de42caa6e4f9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/426304
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-15 03:14:20 +00:00
Changpeng Liu
207353960f nvme: broke up spdk_nvme_probe_internal() into two stages
Previously, function spdk_nvme_probe_internal() will probe
NVMe controllers and then bring up probed controllers
into the ready state after that.  Broke up original two parts
with probe and start stage, this will help us to introduce
a probe context in the next patch.

Change-Id: Ie0c55a6a5463fb437f84349b0b2b33a217ba63e0
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/426303
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-15 03:14:20 +00:00
Changpeng Liu
7d4d22a846 nvme: add a wait for completion timeout API
Althrough SPDK already provides a API to users which
can process runtime timeout NVMe commands, but it's
nice to have another API here, SPDK NVMe driver can
use it to break the endless wait.  Also use the API
first in the initialization process, because we don't
want to add another initialization state with Intel
only supported log pages.

Change-Id: Ibe7cadbc59033a299a1fcf02a66e98fc4eca8100
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444353
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-14 03:47:13 +00:00
Changpeng Liu
2c026cf430 nvme: remove unused minimum period timeout value
Change-Id: I4277166ef5c1ffb5f1d1962ccc5b74d807ef637f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/444352
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-14 03:47:13 +00:00
Darek Stojaczyk
655d54f3f1 nvme: remaning changes related to nvme hooks
Change-Id: I07f3f403bef26a7c3e41b3c9f74e7ba4e378b2cc
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/443650
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2019-02-12 19:43:02 +00:00
Seth Howell
e7beb0d1fd nvme_rdma: don't put req until both send and recv have completed
This prevents us from overrunning the send queue.

Change-Id: I6afbd9e2ba0ff266eb8fee2ae0361ac89fad7f81
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/443476
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-12 18:45:11 +00:00
Shuhei Matsumoto
9562a5c7c1 nvme: Add parse and output strings of prchk flags
The next patch will use the string "prchk:reftag|apptag" as
per-controller prchk options for .INI config file.

Hence add helper functions for them beforehand.

Change-Id: I58c225cc36cc84bf594f108e611028996b5eedb9
Signed-off-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-on: https://review.gerrithub.io/c/443834
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-02-12 09:14:31 +00:00
Ziye Yang
55be9a57a6 nvme/tcp: fix the lvol creation failure issue
The patch is used to fix issue:
https://github.com/spdk/spdk/issues/638

Reason: For supporting sgl, the implementation of
function nvme_tcp_pdu_set_data_buf is not correct.
The translation is not correct for incapsule data
when using SGL. In order not to do the translation
via calling sgl function again, we use a variable
to store the buf.

Change-Id: I580d266d85a1a805b5f168271acac25e5fd60190
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/c/444066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-02-12 03:52:48 +00:00
lorneli
4b6621d08e nvme/pcie: mark infrequent cases as unlikely in submission path
Currently infrequent cases in request completion path are marked as
unlikely. This patch applies that to submission path.

These cases are infrequent and marked using unlikely marco:
a. The sq tail reaches the end of queue.
b. The sq tail equals to sq head. (never happen if FW runs correctly)
c. The qpair is admin queue.

Change-Id: I8b873a18615788f2efbf7c683aad710c7007a082
Signed-off-by: lorneli <lorneli@163.com>
Reviewed-on: https://review.gerrithub.io/c/443451
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2019-02-06 18:37:40 +00:00
Ben Walker
993c4a0799 nvme: Add a function to query controller memory buffer support
Change-Id: Id539f4eaabe2038d4925eaa140864c0abd9b2649
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442635
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
2019-02-06 16:01:56 +00:00
Ben Walker
d3e3f7622b nvmf/tcp: Remove forward declaration of nvme_tcp_req from nvme_tcp.h
This type was actually two entirely different types for
the initiator and the target, so just make it void.

Change-Id: I15512d9d4efd790dce0fa4323b7230de66144bc6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/442438
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-02-04 16:24:00 +00:00
Ben Walker
e8207e9d5d nvmf/tcp: Eliminate atoi use
Use spdk_strtol instead, which does better error detection.

Change-Id: I14236a0b3e42f39a65d67149dd91d791be9f55f1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441983
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
2019-01-29 17:39:18 +00:00
Changpeng Liu
44c6faac9a nvme: move hardcoded keep alive timeout value to macro definition
Change-Id: I27ab6ea046ade42f941b323cea5f104bb952c53d
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/441994
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Arshad Hussain <arshad.super@gmail.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-25 18:52:45 +00:00
Ziye Yang
3608464f04 nvme: fix the aer request sent to disabled controller
The purpose this patch is to fix the following issue:

https://github.com/spdk/spdk/issues/568.

The root cause of issue is in nvme_rdma_fail_qpair
since we want to recycle all outstanding rdma_reqs.
There is an aer req, the callback of which is:
nvme_ctrlr_async_event_cb. In this function, we
will call nvme_ctrlr_construct_and_submit_aer again,
however the nvme controller is already in shutdown state.
(The ctrlr->vcprop.cc.bits.en is set to 0).

Change-Id: I422f0fe5faf472e9a1cb6bbd174e806e6405b95c
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440014
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-18 17:44:08 +00:00
Changpeng Liu
d9e865a885 nvme: report SQ deletion code to outstanding admin requests
RDMA transport will report SPDK_NVME_SC_ABORTED_POWER_LOSS code
when fail the admin queue, however, SPDK_NVME_SC_ABORTED_SQ_DELETION
makes more sense here, because we know we are going to shutdown
the controller.

Fix issue #568.

Change-Id: I31da095ec92c06079511d89cc2743654ba2c001b
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/c/440132
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-17 21:04:43 +00:00
Seth Howell
3018bf907b nvme_rdma: multi-element sgl support for inline reqs
Necessary to avoid erroring out in the edge case where we have an SGL
request sent with two buffers that fit in the incapsule data size.

Change-Id: If51fb69c402482b564c737319584378cb03e7213
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/c/436062
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-15 17:25:35 +00:00
Darek Stojaczyk
eecc6dc8e6 memory: replace all hardcoded 0x200000 with a define
Makes the code slightly more readable.

Change-Id: Iebf8fb07bceacf433d4bdad0a30419a3faab7eee
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439370
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2019-01-13 00:47:26 +00:00
Darek Stojaczyk
3edc4db7a7 env: move VALUE_2MB defines into a new internal header
We use those values in various places in SPDK,
so let's define them in a single place now.

Change-Id: Iad9a5745d69166a6e6032370d4e5a0e604914e45
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/439369
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-13 00:47:26 +00:00
Darek Stojaczyk
8c4ed83b49 vtophys: add length parameter to the vtophys function
This follows the same trend as the mem_map APIs.

Currently, most of the spdk_vtophys() callers manually
detect physically noncontiguous buffers to split them
into multiple physically contiguous chunks. This patch
is a first step towards encapsulating most of that logic
in a single place - in spdk_vtophys() itself.

This patch doesn't change any functionality on its own,
it only extends the API.

Change-Id: I16faa9dea270c370f2a814cd399f59055b5ccc3d
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438449
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: wuzhouhui <wuzhouhui@kingsoft.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2019-01-09 08:24:59 +00:00
Igor Konopko
2077fbd7e4 nvme: do not fail init when Intel log pages are not supported
Currently for all the Intel drives nvme driver tries
to add Intel VS log pages support. When this log pages
are not supported whole init process fails.

This patch changes this behaviour by allowing to init
Intel drives which rejects VS log pages. This is valid
scenario for drives which are in states other than
healthy. Such a drives are still accesible via admin
queue, but does not expose some of the features, such
as this particular VS log pages.

Change-Id: I3764f2d67fd7153b6b1889273a9fedeb9c4213d3
Signed-off-by: Igor Konopko <igor.j.konopko@intel.com>
Reviewed-on: https://review.gerrithub.io/c/437162
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
2019-01-07 16:23:21 +00:00
Chunyang Hui
19feb4e181 nvme: add security receive and security send wrapper
Change-Id: Id25040d62f89d4e8f2268bb3383c5665c0508f5a
Signed-off-by: Chunyang Hui <chunyang.hui@intel.com>
Reviewed-on: https://review.gerrithub.io/c/438776
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2019-01-07 05:51:07 +00:00
Seth Howell
8b4534e43e nvmf_rdma: When destroying a qpair, fail the I/O.
This helps us get rit of outstanding requests at the bdev layer.

Change-Id: I362c7c0c6641715fcd96e8eb465b308c368d34fc
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/431844
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Paul Luse <paul.e.luse@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-23 00:59:42 +00:00
Seth Howell
675c5592e7 nvme_bdev: add parsing for hostaddr and hostsvcid
This allows us to specify the host-side configuration for each
controller to which we connect.

Change-Id: Iac2aed3934d4a326f45546f2f541e374308e2589
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436219
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-20 17:31:43 +00:00
Seth Howell
672115fef4 nvme.c: break out parsing from trid parse
This code snippet will be generally useful for parsing information when
we add support for host address configuration.

Change-Id: Ic90f485de5a5db699901da029c9a29be4db477c7
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/437739
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-19 22:45:13 +00:00
Ziye Yang
b7b2c972c9 nvme/tcp: solve the adminq is not created issue.
If the adminq is not created, we should call
nvme_transport_ctrlr_destruct (i.e., nvme_tcp_ctrlr_destruct
in tcp transport).

Change-Id: If2362c47ac89fd80289ddd8402cf9f74d7bb5801
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436613
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-12-12 20:44:01 +00:00
Lance Hartmann
e865a52415 nvme: Eliminate identify errors to Discovery ctrlr
The nvme/identify cmd issued some cmds to a ctrlr irrespective
of its type, and when the target was a Discovery ctrlr which only
accepts a very limited cmd set, that would result in errors observable
both on the initiator side (from nvme/identify) and in the output on
the target (nvmf_tgt).  Introduce new API, spdk_nvme_ctrlr_is_discovery(),
and alter identify to make use of that in determining which commands
to send to the target.

Change-Id: I974a569843f1d2b9e1ece7bd3bf9ceee1bfae872
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/436225
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-12-11 17:39:52 +00:00
Seth Howell
a451c8385e NVMe-oF: Add explicit reports for MR-split buffers:
This is a failsafe for finding and reporting data buffers that span
multiple Memory Regions. These errors should never be triggered, but
finding and reporting them will help any debugging.

Change-Id: I3c61e3cc510f5a36039fc1815ff0de45fce794d5
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/436054
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-10 23:47:38 +00:00
Ziye Yang
be4fbb2141 nvme_tcp: Make the header and data digest configurable.
Change-Id: Ia65e235a85207c128ba274e1bab38d6c35344239
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435563
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 23:24:12 +00:00
Ziye Yang
71cd1ea7e7 nvme/tcp: Fix the term req data len calculation.
Fix the issue in both target and host sides.

Change-Id: I1bf31072b2164a3035b443fe6c5418a6a7829d81
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/436099
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 18:26:03 +00:00
Darek Stojaczyk
576f08f732 pci: remove nvme-specific attach functions
As a part of cleanup they're replaced by a device-agnostic
attach API, which is easier for us to manage.

Change-Id: I7558590e41e5c580a130a6aba7ae4f7dcff58da8
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/436478
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-07 17:24:19 +00:00
Ziye Yang
b4692083f1 nvme: Fix the race condition in nvme_ctrlr_get_cc
When the applications call spdk_nvme_ctrlr_alloc_io_qpair,
there will be cmd to the admin qpairs in nvme_ctrlr_get_cc,
so there is contention. We should use the lock to protect
nvme_ctrl_get_cc.  Otherwise, the multiple threads will have
contention on the admin qpair, thus there will be coredump issue.

We get the bug when testing NVMe-oF TCP transport, and this
patch can address this issue.

Change-Id: I7247f98cdf890c2eafaf8fb94580ecd714010bd5
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435577
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Seth Howell <seth.howell5141@gmail.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-05 00:32:21 +00:00
Ziye Yang
20ccf47aed nvme_tcp: correctly support the incapsule data size.
According to the TP 8000 spec, the maximal in capsule
data size is defined as follows:

1 For the Fabrics command and admin, it should not exceed
8192 bytes.

2 For I/O command, it shoudld be defined according to ioccsz
in the Identify controller data.

Change-Id: Ic13eda33e1516858e1e8749ee89459e3148d9e37
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435826
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Sasha Kotchubievsky <sashakot@mellanox.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-12-03 22:05:29 +00:00
Jim Harris
72f8c6a1f3 log: remove "trace" from internal API
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I8b1c0d4b00d5d41aae89d3b33f18d1ae957567dc

Reviewed-on: https://review.gerrithub.io/435344
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Xiaodong Liu <xiaodong.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-12-03 19:50:15 +00:00
Lance Hartmann
0891f506fc nvme: Silently ignore ns key in transport id string
spdk_nvme_transport_id_parse() does not recognize the
namespace id, "ns", key as part of the transport id string
and thus logs an error message, but does not fail the call.
However, some SPDK applications, e.g. nvme/perf, in addition
to using spdk_nvme_transport_id_parse() also check for the
existence of a "ns" key in the transport id string to limit
the target to a specific namespace.  This commit adds a
special case to spdk_nvme_transport_id_parse() to silently
ignore the presence of a "ns" key without logging it as an
error.

Change-Id: I49732b4d1b0227a38bb308eab1f6324dd241a2de
Signed-off-by: Lance Hartmann <lance.hartmann@oracle.com>
Reviewed-on: https://review.gerrithub.io/435192
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-12-03 19:50:07 +00:00
Darek Stojaczyk
1d3e0340b4 nvme: fix pci device leak when detaching a controller in primary process
This case isn't particularly supported, but still
caused a memory leak and rendered the pci device
inaccessible for the rest of the primary process
lifetime.

This happens when a controller is removed from the
primary process while a secondary process still
uses it. The controller will likely misbehave without
its primary process managing it, but at least there
won't be a leak.

Change-Id: I67581cffa33ce14ff516b5743d13c9ef7b351625
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434408
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-30 16:25:16 +00:00
Darek Stojaczyk
5557c59c15 nvme/pcie: don't allow constructing a controller from secondary process
With various possibilities to leak the rte_pci_device in the
primary process, we could technically construct the controller
in secondary. The nvme stack is not prepared for this and
will fail to initialize the device, but will still leak the
device object memory.

This patch adds an extra check to prevent any controller from
being constructed in secondary process.

Change-Id: I772f42b541c5db53310362b6595cebf9a30e8491
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/434407
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
2018-11-30 16:25:16 +00:00
Seth Howell
0e6a32deab nvme_rdma/nvmf: add cb_fns to check mr contiguity
This is necessary to confirm that a buffer that spans a 2_MB boundary is
still in a single MR.

Change-Id: If0d14e514ab2197a0d2e3af4f565f56d50591210
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/435179
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-29 15:16:13 +00:00
Ziye Yang
7a39a68c4f nvme/tcp: disable the digest by default.
This patch disables the header digest and data digest
by default, which will improve the performance.

In the another patch, we will make it configurable.

Change-Id: Icdf8cda28217ec35a6b87bb932cdb1e4f8492471
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/435209
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-29 05:02:32 +00:00
Maciej Szwed
5ecc5ef110 nvme: free tqpair on error paths
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: I7abcead55f2eda0a7b8308f81ace70d8a1a59f2d

Reviewed-on: https://review.gerrithub.io/434079
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-26 07:06:42 +00:00
Maciej Szwed
dfff013c04 nvme: unlock mutex on failure
Signed-off-by: Maciej Szwed <maciej.szwed@intel.com>
Change-Id: Id96bd06c61bc49d9ecbdb16f30804a7dda635771

Reviewed-on: https://review.gerrithub.io/434050
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2018-11-26 07:06:42 +00:00
Tomasz Zawadzki
65724c3f88 nvme/tcp: destroy controller when failing to create qpair
Change-Id: I5e7d16e377c03165f338709a71d6e4f03beffc0a
Signed-off-by: Tomasz Zawadzki <tomasz.zawadzki@intel.com>
Reviewed-on: https://review.gerrithub.io/434066
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-21 01:51:08 +00:00
Darek Stojaczyk
8ee5ca50e7 nvme_pcie: re-probe the pci bus on hotplug event
Explicitly attaching a PCI device with spdk_pci_device_attach()
bypasses any kind of blacklists and should be only used
on a user request. Hotplug uevent is certainly not a user
request and should respect the blacklist, hence it's now
changed to call spdk_pci_enumerate() to probe new devices.
The enumeration callback will reject devices other that the
one we got hotplug request for, so no behavior is changed
in that matter.

This patch also fixes undefined behavior caused by reading
unitialized struct nvme_pcie_enum_cb;

Change-Id: I1399fbdd426152a13ed75c85a52bc7f0491ce287
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/433867
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
2018-11-20 15:05:01 +00:00
Ziye Yang
e956be96eb nvme: Add the NVMe over fabrics TCP/IP transport support
It is the first patch to follow the NVMe over fabrics
spec and implmenent the NVMe/TCP transport. It can be
divided into work in the host and target sides:

Host side: Add the TCP/IP transport in nvme lib (lib/nvme).
Target side: Add the TCP/IP transport in nvmf lib (lib/nvmf).

Change-Id: Idc4f93750df676354f6c2ea8ecdb234e3638fd44
Signed-off-by: Ziye Yang <optimistyzy@gmail.com>
Reviewed-on: https://review.gerrithub.io/425191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-19 20:36:05 +00:00
Changpeng Liu
2706cd4238 nvme: add timeout for Admin commands when initialization
Currently there are no timeout mechanism for Admin commands
when initialization, the NVMe driver may enter infinite loop.
While here, add a new parameter to the controller initialization
options, NVMe controller will report an error when timeout
happens during initialization.

Change-Id: Id0c6b6fa15abe5227b486bee95c8e02914b0d358
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/424622
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
2018-11-16 15:29:33 +00:00
Seth Howell
e688d1ccf1 nvme_rdma: properly configure and store max_sges
The max_send_sge and max_recv_sge values can be set to any value from
0...dev_attr->max_sge. WHen we actually set the attributes, we will
receive a qpair with values for max_sge greater than or equal to what we
initially set. We need to store the maximum number of SGEs for later use
when constructing work requests.
Previously we have not relied on these values since we assumed that we
would always be able to have more sges than we asked for initially. This
may change as we try to allocate more SGEs to handle splitting buffers
across memory regions.

Change-Id: Ibbeae1908b86baa3a96d9c6cd2051401aaa2197b
Signed-off-by: Seth Howell <seth.howell@intel.com>
Reviewed-on: https://review.gerrithub.io/433307
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
2018-11-16 15:18:41 +00:00
Ben Walker
bf1a82cf5a nvme/rdma: Make RDMA memory map per-pd again
The RDMA memory map needs to be per-protection
domain, not per NVMe controller. Otherwise, when
an NVMe controller is removed, the memory map may
reference an invalid pointer to a detached
controller.

Change-Id: I0c5bd2172daee0c70efb40eab784839e0cde8bc4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-on: https://review.gerrithub.io/432590
Chandler-Test-Pool: SPDK Automated Test System <sys_sgsw@intel.com>
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-12 15:31:34 +00:00
Liu Xiaodong
5aace13984 lib/nvme: tolerate abnormal char device
In some special cases, NVMe device with cdata.nn=0
may be used to do validation or other test work.
cdata.nn=0 means the device can't support NS at all.

Change-Id: I55f75a8cb21b8d1b99c5318e27c876a4371d6dd4
Signed-off-by: Liu Xiaodong <xiaodong.liu@intel.com>
Reviewed-on: https://review.gerrithub.io/432191
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: joevannip <jparairo@nvxltech.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Shuhei Matsumoto <shuhei.matsumoto.xt@hitachi.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-08 23:35:28 +00:00
zkhatami88
9fb6947617 nvme: Add mechanism to override RDMA pd/mr behavior
Add a mechanism to modify the RDMA transport's behavior
when creating protection domains and registering memory.
This is entirely optional.

Change-Id: I7cd850e76a673bf5521ca4815b779c53ab9567e8
Signed-off-by: zkhatami88 <z.khatami88@gmail.com>
Reviewed-on: https://review.gerrithub.io/421415
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-05 17:22:30 +00:00
Darek Stojaczyk
04ee899fcf nvme: improve probe error handling in MP even further
In cases we probe without a specific trid, the underlying
rte_bus_probe() in spdk_pci_enumerate() might fail to
initialize some devices, but still return with code 0,
That's technically correct, as we asked just to probe
devices on the bus and that's what it did. Some devices might
have been initialized, others not. In secondary process we
blindly assumed all devices were probed successfully, which
might have eventually led to assert failures, as current
process was not on the ctrlr->active_procs list.

To fix it, just add an additional check before attaching
the controller in secondary process.

Change-Id: If015b1e562052a9189ed1a48091b209bd2dd5f2a
Signed-off-by: Darek Stojaczyk <dariusz.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/431727
Tested-by: SPDK CI Jenkins <sys_sgci@intel.com>
Reviewed-by: Ben Walker <benjamin.walker@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
2018-11-02 18:30:09 +00:00