Commit Graph

354 Commits

Author SHA1 Message Date
Ziye Yang
d61ddd3c93 nvme/rdma: Support directly connect via trid
Preivously, we only supports probe the NVMf target
via discovery info, now we can support to directly
to connect it.

Change-Id: I08ce1d95de6744286357e68b48c97b773b902ac8
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-13 10:57:03 -07:00
Ziye Yang
90f13aa634 nvme/rdma: Support sgl for readv/writev functions
(1) Add nvme_rdma_build_sgl_request function
(2) Merge nvme_rdma_pre/post_copy_mem to nvme_rdma_copy_mem

Change-Id: I86abab821b32b4da0aa9489a6b9f7dc430333159
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2017-01-11 12:36:52 -07:00
Daniel Verkamp
f80c0f4fdd nvme: remove transport ctrlr_attach callback
Now that the hotplug code is isolated in nvme_pcie.c, it can call the
PCIe transport attach function directly.

Change-Id: I2df3b9168473b537cc9b13367e06d3d3b6fa22be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-09 11:36:27 -07:00
Daniel Verkamp
df8129fb39 nvme: move num_entries to transport-specific qpairs
The spdk_nvme_qpair::num_entries value is never used in the common code,
so move it to the individual transport qpairs to make it clear that it
is a transport-specific implementation detail.

Change-Id: I5c8f0de4fcd808912ba6d248cf5cee816079fd32
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-05 15:49:09 -07:00
Daniel Verkamp
f1a9afa9fc assert.h: split UNREACHABLE into internal header
The definition of SPDK_UNREACHABLE uses the build-time DEBUG definition,
which is not available in the public API.

Change-Id: I1862c99fa5c85ccd3483f94e9c35de531da57f3c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-05 11:57:18 -07:00
Ben Walker
c2d38ee8da nvme/rdma: Improve error message when polling fails
Change-Id: Ia75d62669f3c1f923031f6a33e4797cc56ac1141
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
d9a3a8e50c nvme/rdma: nvme_rdma_recv now takes the response index
Instead of passing the work completion, just pass the
response index. This keeps the work completions localized
to the polling function.

Change-Id: I0e6a1d8564200b5ac3aa43dfd58ae152d439bbd8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
0ebf93e35c nvme/rdma: Pass qsize to nvme_rdma_ctrlr_create_qpair
This eliminates an if statement, since the two callers
of this function know the desired queue size.

Change-Id: I28fabac8613f7b8fc7d96cf95b085b6e4dcf985f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
c26c655d59 nvme/rdma: Eliminate nvme_rdma_ctrlr_construct_admin_qpair
Just call the regular qpair create function instead.

Change-Id: Ic35b1eb6fcdf0d82733ea573a493f583dd63d5bd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
3ad0c336c8 nvme/rdma: Eliminate max_queue_depth
Use the num_entries value in the generic qpair instead. These
values had to match anyway.

Change-Id: Ia6400fbaba97df3ef6db4dc07a2ab95af1e5143f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Ben Walker
10aed30149 nvme/rdma: Handle requested qsize not matching obtained qsize.
Change-Id: I44d9e01dba87c2f78ef13296b78532255d749432
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2017-01-05 09:24:02 -07:00
Daniel Verkamp
e1b514ce9a nvme/rdma: cache value of bb_mr->rkey
Avoid an extra level of pointer chasing when we are filling out the NVMe
SGL.

Change-Id: I1a40af16fda80f7480c419524876bfb1a1902eb8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2017-01-03 16:56:34 -07:00
Cunyin Chang
6152d5bf8f nvme: move hot-plug logic to nvme pcie layer.
Change-Id: I2042b34e4284a9c59aa3092ccd061c075748880c
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2017-01-03 16:56:13 -07:00
Jim Harris
4cc1cf888c nvme: break out request child creation code to separate function
This will allow it to be better be reused for some future patches
enabling splitting of non-PRP-compliant SGL-based requests.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ica38fd6cf191f72baa524bcc4896b3c9939ab762
2017-01-03 15:16:05 -07:00
Yongseok Oh
ee5b26a2d2 nvme: LBA variable fixed to 64bits for _nvme_ns_cmd_setup_request() function (#96) 2017-01-03 09:35:37 -07:00
GangCao
5a3def3936 nvme: free allocated contig_buffer
Change-Id: I1ce45d4937ee7a170d74f381cfaed263e3420ccd
Signed-off-by: GangCao <gang.cao@intel.com>
2016-12-20 13:49:31 -07:00
Ben Walker
bf89b4da10 nvme/rdma: Eliminate _nvme_rdma_ctrlr_create_qpair
This intermediate function is no longer needed.

Change-Id: I3523cc6d8f3b290165a953d42cca8b76eda762c5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
30677dc222 nvme/rdma: Move call to send fabric connect into qpair_connect
Sending the fabric connect command is part of establishing
a connection, so move it into the main connection-establishing
function.

Change-Id: I55e7ffdd16b576c81b51d7d3910203f9afc1f4c2
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
a96a6ecf58 nvme: Rename nvme_qpair_construct to nvme_qpair_init
This function initializes the members of an existing
qpair struct. It doesn't construct one from scratch.

Change-Id: I0b9afac1ad25cfb217efd146702f693c74f5f697
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
3e5c1d0a2b nvme/rdma: Don't allocate requests/responses until after connected.
No need to allocate all of the requests and responses until
we know a connection can be established.

Change-Id: I072a10aadfd7ced773634448f7d7e788622d0a4c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
ee80b31cb7 nvme/rdma: Delete config_rdma_req
The code is clearer if this function is incorporated
into its only caller.

Change-Id: I33901cddf80ae27896b2acfd1b9e7d212f21f5f3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
414702426d nvme/rdma: Rename nvme_rdma_bind_addr to resolve_addr
This is resolving the address and route to the target, not
binding a socket to an address.

Change-Id: I80055481ed2e020410a1e186a4e7371b60faaee9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
42dc2836bc nvme/rdma: Add a utility for getting the next cm event
Change-Id: Ia7650e729ab233be6464fb0a190c9cd29f462fcf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
fa97f35f06 nvme/rdma: Remove unused code in nvme_rdma_connect
Change-Id: I4f63136f57d074a9f7ff6027f2e8f6fb8a9c8dae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 12:51:50 -07:00
Ben Walker
df46c41a4c nvme: Unify spdk_nvme_discover and spdk_nvme_probe
They were very close to the same already, so finish the job.

Change-Id: Ifba9e3b2d11a3e70cbfbe46f57a67552db2757ed
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 11:22:57 -07:00
Ben Walker
c00bce397d nvme: Move nvme_hotplug_monitor up in the file
Avoid having to forward declare.

Change-Id: Ib1e5629f855fa70ba77247972823a8418a5fc7a9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-19 11:22:53 -07:00
Daniel Verkamp
b8da09b629 nvme/rdma: the NVMe SGL should use rkey, not lkey
We should be sending the bounce buffer's remote key to the target so it
can put it into an RDMA SGE on the remote side.

Change-Id: Icded155ad2292c67baa722f001c9c07178bc2754
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:53 -07:00
Daniel Verkamp
b74979421f nvme/rdma: remove write-only rdma_req->bb_len
Change-Id: Ied1bee7c8db87cca890d94e519ec07ae3fc658b2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:53 -07:00
Daniel Verkamp
e8184c1f25 nvme/rdma: change max queue size from 127 to 128
There is no particular reason for this to be 127; make it 128 to at
least be consistent with the PCIe transport.

Change-Id: I60500e0044d3549ba6350e1f35f09d624848bd21
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 16:39:28 -07:00
Ben Walker
b2a86421f4 nvme/rdma: Fix specification of R/W queue depth
The meaning of the parameters was swapped.

Change-Id: I3f4938c446fd06afd1852bcf08e779b7b988855d
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-15 16:35:00 -07:00
Ben Walker
375da65762 nvme/rdma: Remove nvmf_cm_construct
This function was only called from one place and saved no
lines of code.

Change-Id: If5e653732df57c1f2c93e20cf4f286eac31df91c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-12-15 16:35:00 -07:00
Daniel Verkamp
d24ea9009a nvme: add offset to spdk_nvme_ctrlr_cmd_get_log_page()
Change-Id: I1af4950995062f892062f29db85da2a8dd793d85
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:16:14 -07:00
Daniel Verkamp
9938985766 nvme/rdma: queue request if no rdma_req available
This matches the behavior of nvme_pcie, which queues a request if no
tracker is available.

Change-Id: Idbf6c951c89451cfea22ec6bc553ff46f988f818
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:13:09 -07:00
Daniel Verkamp
ee82713678 nvme/rdma: clean up nvme_rdma_req_init()
Make the caller pre-allocate an rdma_req and change req_init() so it
only does initialization, not allocation.

This is necessary to distinguish between rdma_req allocation failure and
other types of failures, which will become important in future patches
when requests will be queued if rdma_req allocation fails.

Change-Id: Ie6edebc1b5f05001b42fc959a29ce0ea6875e41e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:50 -07:00
Daniel Verkamp
be92f47fa8 nvme/rdma: merge rdma_qpair_construct into alloc_reqs
Simplify the control flow and match the name of the function to its
purpose.

Change-Id: I65bad7e3b2ef710ca29eff9799b8dcaae3999315
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:50 -07:00
Daniel Verkamp
0eb3125531 nvme: remove transport qpair_construct callback
Make the qpair construct functions private to the transports - it
doesn't need to be called from generic code.

Change-Id: I5f730a4bcf60ce231fe27bc8f4c3c39cb647dd2d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:49 -07:00
Daniel Verkamp
988906135c nvme: determine io_queue_size at startup
Add a transport callback to return the maximum queue size, and enforce
it in the generic nvme_ctrlr layer.

This allows the user to tell what io_queue_size was actually selected by
the transport via the ctrlr_opts returned during attach_cb.

Change-Id: I8a51332cc01c6655e2a3a171bb92877fe48ea267
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:48 -07:00
Daniel Verkamp
f505f57b36 nvme: add ctrlr function to set initial CAP value
Change-Id: Id7f68f279f87a08844015efcd35a790944b3bfbe
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 11:09:25 -07:00
Daniel Verkamp
fb31963c4b nvme/rdma: register cmds as a single MR per queue
Equivalent to commit 6ab28a201b except now
for commands instead of responses.

Change-Id: Ibe4382dc0f65c1b90c2cee2ad285bbdd21b96a89
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 10:58:23 -07:00
Daniel Verkamp
5ab3a71e2a nvme/rdma: remove unnecessary bb_sgl
The only field of bb_sgl that was actually used is lkey, and that is
already stored in bb_mr.

Change-Id: I790369a06ce223f88e356df20a9d9a74a93ff225
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-15 10:52:27 -07:00
Daniel Verkamp
5848faab5c nvme/rdma: split discovery probe into a new function
Refactor the discover log page processing into a loop that calls a
function for each log page entry.  This sets us up to add support for
multiple Get Log Page calls to handle larger discovery service lists.

Change-Id: I85676ada375d0dadda2a3f4ab6331123ac7aaf60
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 11:03:38 -07:00
Daniel Verkamp
32e6dd1573 nvme/rdma: bounds check discovery log page numrec
Change-Id: Ic7bcd6b7f42510be68f70fb3526a344754ecbbc0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 11:03:38 -07:00
Daniel Verkamp
23fc6682fb nvme/rdma: use trid.adrfam when resolving address
Change-Id: I3c3c0faa808e6047d1f9297dc6985597cc0c4aaf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 10:14:10 -07:00
Daniel Verkamp
0346dd775e nvme: add ADRFAM (address family) to transport ID
This isn't used yet in the NVMe library, but it will be necessary later
for supporting non-IPv4 addresses.

Change-Id: I167ce63ad25b0e0c9aa192b12d764c8d078e67f9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-14 10:14:10 -07:00
Daniel Verkamp
9c5df2c4bf nvme: make NULL trid equivalent to probing PCIe
Change-Id: I286c23d96d1d21abd102adf51c3d4675e131b11d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-13 15:29:53 -07:00
Daniel Verkamp
92ee5fc4be nvme: rename queue_size option to io_queue_size
This better describes what the field controls (it does not affect the
admin queue size).

Change-Id: I851ae46fb4ed0fce819af07ae235824e0fc817e6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-12-13 15:23:13 -07:00
Pawel Wodkowski
8a62ba51f2 lib/nvme: change in payload offset propagation path
For requests split in _nvme_ns_cmd_split_request() the payload offset is
set after children are created using recurrent call _nvme_ns_cmd_rw().
This makes impossible to reset SGL to proper offset in
incomming patches that split non-PRP complaint SGL requests.

To change this the payload offset is set after each request is allocated
in _nvme_ns_cmd_rw() not in _nvme_ns_cmd_split_request().


Change-Id: I9d3b2e3bbd9d93a4c8a37e1db8c4e01276e2cacb
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2016-12-13 10:40:02 -07:00
Pawel Wodkowski
636b078b09 lib/nvme: extract function for setting up request cmd
This is preparation for handling non-PRP compliant SGL.

Change-Id: I445790f9802292971256cf821d9730814c95a073
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2016-12-13 10:23:20 -07:00
Pawel Wodkowski
bcf546bb00 lib/nvme: extract function for removing child requests
This is preparation for handling non-PRP compliant SGL.

Change-Id: I49c3745498411c5ff9e17cd08f181d4d434c2d08
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
2016-12-13 10:23:11 -07:00
Cunyin Chang
ca618b2751 nvme: Fix the bug for address parse when get hot insert event.
Change-Id: Ic346173cb88a774ac4288ad3a5aa08fb3fe909ec
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-12-13 10:06:17 -07:00