624 Commits

Author SHA1 Message Date
Ben Walker
c6a608769d nvmf: Move sq_head updates to nvmf_rdma_request_release
Change-Id: Iaba621d54ae600015c9d1dbec6485a730da11bf3
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 16:28:33 -07:00
Ben Walker
130fec6636 nvmf: Add better tracing of RDMA operations
Change-Id: Icf5f39fad41d85bb6b325f9fc51b08a7e1055323
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 15:39:37 -07:00
Ben Walker
2d68928c3f nvmf: nvmf_rdma_accept now uses nvmf_rdma_poll
Instead of reimplementing handling for checking the
completion queue, nvmf_rdma_accept can now call
the general purpose poller.

Change-Id: Id2c899d1e500a8cb8491e51cc101a1bf0e167764
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 15:36:01 -07:00
Ben Walker
c9593cd17e nvmf: Add temporary special handling for AER
AER breaks our current model of requests/completion pairs.
Temporarily handle it by immediately re-posting the
capsule while we work on a real solution.

Change-Id: Ie7a4d88030b6fff5a11c4697eec0f024f9737f27
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 15:36:01 -07:00
Ben Walker
04a0ac723c nvmf: conn_poll now returns a count of requests
Change-Id: Ic239bfa072905bbb65574e344d6a060cb4ce44e5
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 15:35:53 -07:00
Ben Walker
6beb310cf3 nvmf: Remove nvmf_recv in RDMA layer.
Inline this code into the places that called it. These two
spots will be combined into a single path in a later patch.

Change-Id: Ice2f009ad56b783dc28ebbf1abbb877ce6000293
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:35:46 -07:00
Ben Walker
2625cf4261 nvmf: Move nvmf_request_prep_data into rdma.c
This is an RDMA-specific operation, so hide it inside
the transport-specific layer.

Change-Id: Iaa097e8dde78d820547b3a39e9717c992581340b
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:35:46 -07:00
Ben Walker
9d9dc8452c nvmf: Allocate all rdma requests up front.
Change-Id: Ia7fdb6994b8c167840d7335a2dfcad3ce6171d3a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:35:45 -07:00
Ben Walker
989859bbe1 nvmf: Combine alloc_rdma_queue and nvmf_rdma_queue_init
These can be done at the same time now that the queue depth
is known ahead of time.

Change-Id: I7ecef30ebb4311e0a1c88f37461d34534f8600bf
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:35:45 -07:00
Ben Walker
a9f5ffbd1c nvmf: Change algorithm for calculating queue depth
Calculate queue depth into a local variable without
touching the rdma_conn.

Change-Id: Ie804ed39ddecbf59015a4e4f7aa127f1381d9080
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:35:34 -07:00
Ben Walker
6a61126f37 nvmf: Eliminate conn_id local variable
Change-Id: Iac2371f60914d43a14adadd8c4ecd7663726584f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 14:18:56 -07:00
Daniel Verkamp
c943e9ff4f trace: hard-code lcore history array size
Make sure the trace history that is exported via shared memory is always
the same size, regardless of DPDK configuration.

Also removes the necessity of including DPDK headers from spdk/trace.h
(so we have to fix up other files to include what they use).

Change-Id: I32f88921fd95c64a9d1f4ba768ae75e2ca5d91da
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 12:50:44 -07:00
Daniel Verkamp
89031c3e27 nvmf: move discovery subsystem creation to conf.c
It is not currently configurable, but this will allow us to make the
discovery subsystem have config options (e.g. which lcore to run on).

Change-Id: I788a64ba4462b023453191e509ce8de59fd90ae4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 12:50:01 -07:00
Ben Walker
35bc1e93e1 nvmf: Add prints for queue depth calculations
Change-Id: Ibef84b9622530750fa75fbac4b95f0886e52fdd9
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 10:37:02 -07:00
Ben Walker
207e6a604e nvmf: Set minimum and maximum on allowed queue depths
Change-Id: I72988b0691154165757201f5f5bc89d199249660
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 10:37:02 -07:00
Ben Walker
90cd7fc5a1 nvmf: Take config file into account when setting queue depth
Change-Id: I9b388e6c0f19bf3fcd96ccf6e2934ac446d8399a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 10:37:02 -07:00
Ben Walker
7219b20124 nvmf: Set a minimum and maximum queues per session
Change-Id: I68d6fe5e3dfeb1e4709ce39654c61505189711bc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 10:37:02 -07:00
Ben Walker
944695ca21 nvmf: Remove controller.[ch] and probe for each subsystem.
This is a much simpler approach and is only slightly
less efficient.

Change-Id: I909de376d576a74156c1be447e90e7dbc240f025
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-22 09:24:45 -07:00
Daniel Verkamp
fd3f7ee3b3 nvmf: simplify nvmf_process_io_cmd()
Drop the redundant controller ready check.
nvmf_process_io_cmd() was checking CSTS.RDY, but this is not necessary,
since its only caller, spdk_nvmf_request_exec(), is already checking
CC.EN, which always matches RDY in our virtual controller
implementation.

The initialization of status is a dead store -
nvmf_complete_cmd() always writes the full response, and the only other
branch is the return immediately below the call, which also sets status.

Change-Id: I1ec2b8a225a91c4b2997d8ab4f45d050cc216de3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 09:14:47 -07:00
Daniel Verkamp
1f929aa55c nvmf: replace RTE_VERIFY with assert in request.c
No reason to use DPDK in this file just for an equivalent to assert().

Change-Id: Ic6932a16d0a36cd1a3cb25c8cc5e295c59f3e2db
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 09:14:07 -07:00
Daniel Verkamp
163127a65e nvmf: fix ioccsz (in-capsule data size)
Temporarily set the in-capsule data size to the maximum data transfer
length.  This should actually be updated by the transport layer, but for
now, the only transport (RDMA) supports the full bounce buffer size.

Also drop the check that prevents admin connections from using
in-capsule data; the host may send in-capsule data for the Connect on an
I/O queue, and we don't know the type of connection until after Connect
is processed.

Fixes: 828dca7 ("nvmf: Move some stray session init code to the right place")

Change-Id: I369ee5497247d7e875ad0b6f0aaf6c47c1d3887c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 09:13:59 -07:00
Daniel Verkamp
8807268cb8 nvmf: zero out response data for each command
Make sure no response fields are left over from the previous command in
the spdk_nvmf_request.

Change-Id: I42937e991d9dd6550fd4bc9b6d0dd66b44c6b83e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-22 08:45:01 -07:00
Cunyin Chang
d439f4408a nvmf: Add subsystem modes
Change-Id: I74f69eb10e4d8807a323f463775f4953fe0baee0
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-07-21 17:45:06 -07:00
Daniel Verkamp
861e78bf48 bdev: add block device abstraction layer
Change-Id: I235cf146a52714756c9782c03b118f518c5f5182
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 10:35:03 -07:00
Daniel Verkamp
7ec9bb11da pci: stub out kernel driver functions on FreeBSD
The kernel driver unloading/loading code is Linux specific; replace it
with stubs on FreeBSD for now.

Change-Id: Ic67c1d89b2fb9a65e9ce5b88d27b6cd6af5554a7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 09:23:24 -07:00
Daniel Verkamp
51bc5fce79 nvmf: drop redundant response CID assignments
spdk_nvmf_request_complete() always sets CID to the value in the
command, so there is no need to set it in the command execution
functions.

Change-Id: Ibbe745b862e27fff7c55e553758ef093e3ef7f6d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 09:20:55 -07:00
Daniel Verkamp
13c65342d4 nvmf: wrap number of queue lines in a nicer place
Change-Id: Ideffc66bd8051a8275c6692db08166f2b4ad6e9b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 09:20:55 -07:00
Daniel Verkamp
03cd283e43 nvmf: simplify Identify command handling
Use the passthrough command for all Identify commands except Identify
Controller.

Also only check the CNS field of CDW10 and use the new enumerated names
instead of magic numbers.

Change-Id: Ia94f820ac85a2d6b2d0ae02659e73c53f1b1a4cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-21 09:20:55 -07:00
Daniel Verkamp
fc07014a54 copy_engine: add I/OAT copy engine module
Change-Id: I670ab7c32aa7571def22c06dde091972808f215b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:57:22 -07:00
Daniel Verkamp
746dc47d62 copy_engine: add memory copy offload engine layer
Change-Id: I1aee004d6e0938972ece16eba6317d02ba3b0042
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:57:22 -07:00
Daniel Verkamp
336ba0d09c build: replace USE_PCIACCESS with config.h define
Drop the special-case preprocessor definition for PCI access library now
that config.h is available with an equivalent SPDK_CONFIG_PCIACCESS
define.

Change-Id: I4891d0f2fd7d3eea51b767df9e594555b36265ea
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:53:47 -07:00
Changpeng Liu
cd258ce089 nvmf: ack the disconect event before call rdma_destroy_id
If we connected a subsystem twice from the initiator, the second
connection will be rejected by the NVMf target, however, the previous
connection will also be impacted because we destroy the connection id
before ack the disconnect event.

Change-Id: Ib597cc68a7823524460693053898f4d6e5499eb4
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-07-20 09:45:16 -07:00
Changpeng Liu
080bdfb1da nvmf: destruct the session when no connections on it
Change-Id: I32a9d71ea139eec346f020c629260a395fd228b9
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-07-20 09:41:44 -07:00
Ben Walker
88245ea26b nvmf: Rename MaxConnectionsPerSession to MaxQueuesPerSession
Change-Id: I9891f9ed481b93312152c2a59f4263003167f1d6
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-20 07:26:58 -07:00
Daniel Verkamp
b5e6d221bb nvmf: use raw passthrough for all I/O commands
There is no need to handle Read and Write commands separately; the
generic raw I/O command case can handle them just as well.

Change-Id: I8475eed0a20bd809c447ed2ccac0b99f6c2a9b4d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 07:26:38 -07:00
Daniel Verkamp
77cb78fadf nvme: add enumeration of Identify command types
Change-Id: I2f8dc3de48c2e5dc06ccc9d007373e361bffcf63
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 13:31:06 -07:00
Daniel Verkamp
ffa936d94b nvmf: switch to rte_mempool_avail_count()
Replace use of the newly-deprecated rte_mempool_count() with the new
name, rte_mempool_avail_count().

Also add a compatibility wrapper so that builds against older DPDK
versions still work.

Change-Id: If3c44bdef4bbcf7a456a1dfa272348ccc6f35261
Reported-by: Jay Sternberg <jay.e.sternberg@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 10:02:20 -07:00
Daniel Verkamp
ec39dd62e7 nvmf: fail commands when controller is disabled
The host is not allowed to send normal admin or I/O commands until the
controller is enabled (via the Fabric Property Set command).

Change-Id: Ib62be3a3792fc0b36bace28b4c9afdf78dad3bcd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:28:15 -07:00
Daniel Verkamp
b794135bb1 nvmf: enforce fabric command order and type rules
Only allow Connect on a new connection (one that has no associated
session yet), and only allow Propert Set/Get on admin queues.

Change-Id: Iae22379ee47b095333372e6d151a7a1509acf654
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:28:15 -07:00
Daniel Verkamp
2d1c3419ad nvmf: verify correct IOSQES and IOCQES on connect
The NVMe spec requires that the I/O queue entry size values in CC are
set before any I/O queues may be created.

Change-Id: I4f0c9a9c20411223d281993745c85a8431197961
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:20:20 -07:00
Daniel Verkamp
43510e1292 nvmf: fail attempts to modify reserved bits in CC
Track each individual bit in the Set Property handler for CC, and fail
the request if any unhandled bits are modified.

Also add handlers for IOSQES and IOCQES (I/O submission and completion
queue entry size).

Change-Id: I374dc3c15197e029ba07fd9ee1cff0e38a0a884d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:20:20 -07:00
Daniel Verkamp
c53c5c6a9b nvmf: stub out CC.EN reset path
It is not implemented yet, but add a message to remind us to write it
later.

Change-Id: Ic1c35a0d35f728bc63b38c334d9c622493bee967
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:20:20 -07:00
Daniel Verkamp
b9a67e983a nvmf: remove nvmf_property_set() shutdown flag
Property Set of CC.SHN is not supposed to terminate the session - remove
the commented-out code that was attempting to do this.

Change-Id: I1db230df9be549764287a8fd45ccdebea1d22a8b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:20:20 -07:00
Daniel Verkamp
8a2395d0e6 nvmf: update CSTS when host writes CC.SHN
Set CSTS.SHST = 10b to indicate that shutdown is complete, and
CSTS.RDY = 0 to match the state of CC.EN.

Change-Id: Ia651c34427526a38f22cba3910df2cf7d4bedd92
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-19 09:20:20 -07:00
Daniel Verkamp
0cb9522781 build: include spdk.common.mk in lib Makefiles
Explicitly include spdk.common.mk at the top of all lib Makefiles so
that CONFIG options and other predefined variables are set.

Change-Id: I1e560c294fe8242602e45191a280f4295533ae44
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-18 16:17:40 -07:00
Changpeng Liu
cf29df4418 nvmf: fix the compile issue when enable CONFIG_RDMA in the CONFIG file
Change-Id: I3fb3b4cf762467f652af088b36669d30a8d8b726
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-07-18 16:07:39 -07:00
Daniel Verkamp
0480df054b nvmf: generate RDMA SGLs on the fly
There is no need to allocate ibv_sge structures within the RDMA request;
we can just fill them out on the stack right before submitting each
request.

Change-Id: I438ff0be2f6d07ffa933255c92c4ec964aa1b235
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 17:46:21 -07:00
Daniel Verkamp
2d75d67aa9 nvmf: don't count RDMA CQ entries
Just return success or failure - the actual count was not used.

Change-Id: I26e7c4c6319af444d221d9b0f313fb7071733619
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 16:34:48 -07:00
Daniel Verkamp
10687626e5 nvmf: factor out common RDMA completion code
All of the WC events that we handle map back to a request, so look it up
before checking the opcode.

Change-Id: I1b70a773374f64387df0a21a4f7fd64b26534b14
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 16:34:48 -07:00
Daniel Verkamp
e9658d055e nvmf: clean up RDMA tracelogs
Make sure all tracelogs in rdma.c use SPDK_TRACE_RDMA.

Change-Id: Idc3d3b6654215b5ab3ee84a106e46ffd3019cc7a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 16:34:48 -07:00