Commit Graph

373 Commits

Author SHA1 Message Date
Daniel Verkamp
70601d5e0e nvmf: add Doxygen comments to discovery log page
Also add the missing ASQSZ field.

Change-Id: I67e67a6b4076de226d30882aa40afd78bc8be2af
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 16:34:46 -07:00
Ben Walker
677a2dc5d7 nvmf: Merge conn.[ch] into session.[ch]
There was only one function and a structure declaration
left.

Change-Id: I63277b4182120e7a76a925ed0bf7378ec7c23f20
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 14:45:26 -07:00
Ben Walker
2b9d85c448 nvmf: Remove host.[ch] and port.[ch]
These can be simplified and merged into the subsystem.

Remove the concept of mappings from subsystems and replace
it with a list of hosts and ports. The host is optional -
not specifying a host means any host can connect.

Change-Id: Ib3786acb40a34b7e10935af55f4b6756d40cc906
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 14:35:24 -07:00
Ben Walker
d38d299576 nvmf: Round-robin allocate subsystems to lcores
Change-Id: I3f3937e0cdcf99f4e4be755df2865682ab230dfc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 13:12:18 -07:00
Daniel Verkamp
e3d9e24e20 nvmf: add discovery callback to transport
Make the transport responsible for filling out the fabric-specific
details in the discovery log entry.

Change-Id: I41d871c605becd557dca18f8ef7e80da66950257
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 13:11:03 -07:00
Daniel Verkamp
21c450e187 nvmf: add transport function pointer table
Make the core NVMf to transport interface generic and allow for multiple
transport types to be registered.

Change-Id: I0a2767a47d55999c45f788ae1318bb50af60ab4e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 13:11:03 -07:00
Daniel Verkamp
6a1383813c nvmf: rename FabricIntf to Listen, add transport
Change the Port configuration file entries to a new format:

[Port1]
  Listen <transport> <address>:<service>

Initially, this still only supports RDMA, but the new format will allow
specifying other transports once they are added.

Change-Id: Iadfd19b91db57b571064379368dbe77204ccecbb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 13:07:43 -07:00
Ben Walker
d6a499fec2 nvmf: Move poller from connection to subsystem
Change-Id: Iea6be8156152367c0fd48b8cee8e2ca1e67f340a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 12:22:53 -07:00
Ben Walker
96a54158bd nvmf: Assign each subsystem to an lcore
Each subsystem will run on a single core, which is more than enough
to fully saturate a device and a NIC. For now, all subsystems
run on the master lcore.

Change-Id: I95340a262d70fd346fa81fe519e7d4190a369e64
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 10:43:41 -07:00
Ben Walker
ca7a61e18a nvmf: New RDMA connections move to lcore after CONNECT capsule
Instead of starting the connection poller immediately upon
the connect event, wait for the first connect capsule to
start the poller.

This builds toward associating all connections with the same
session with the same lcore.

Change-Id: I7f08b2dd34585d093ad36a4ebca63c5f782dcf14
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-15 10:32:40 -07:00
Daniel Verkamp
0e1dc05efb nvmf: move TSAS to fabric_intf
It can be different per fabric interface within a single port.

Change-Id: If13590d7f12291499ccfd705efaf6d2b1b1d7003
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Daniel Verkamp
baf523b5d8 nvmf: add _subtype to TSAS structures
Fix the naming to match the spec.

Change-Id: I5e2fba54c796f8d1f11d61fb3d131aabc5c90503
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Daniel Verkamp
70e26ebd6f nvmf: remove unused port type field and enum
The type is already stored in the fabric_intf.

Change-Id: Icd33dd29f2fa1313329b4053892693c7ff90945d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Daniel Verkamp
2653cb49a7 nvmf: define transport-specific address union
For now, it just contains RDMA, plus a raw byte array to allow generic
copying.

Change-Id: I02fe11f99dd8b49000de0dba991cd34c99fd7a4a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Daniel Verkamp
ac470faa01 nvmf: remove unused #define SPDK_CN_TAG_MAX
Change-Id: I4ce711b3a9878fe06e508fbb1448791fda5c6917
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Daniel Verkamp
b10bf60cac nvmf: remove arbitrary 4-port limit
Change-Id: Id37a0f044845c175b2de43a5e1fa76513843f314
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-15 10:23:46 -07:00
Changpeng Liu
3c96acb4f4 nvmf: fix double free of session structure when exit the NVMf process
Change-Id: Iea34ca82898e58633c6f7edd2ab8ff04f5484220
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-07-15 10:15:03 -07:00
Daniel Verkamp
49e0054f18 nvmf: clean up redundant function name in TRACELOG
Change-Id: I87a01e0fe8c1437c084757e66377ac50ca83a0c2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 13:32:33 -07:00
Daniel Verkamp
eca013c2e1 nvmf: simplify RDMA queue depth calculation
Pull out the duplicated min checks against the ibdev_attr values.

Change-Id: I774c355ba669486afde5c05c55a4ed653723db98
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Daniel Verkamp
c3ed14d3a0 nvmf: refactor Connect command handling
Set a status code in the response capsule for each possible error case.

Also enforce CC.EN == 1 before I/O connect.
The NVMf spec requires that the controller is enabled before any I/O
queue Connect commands are allowed.

Change-Id: If56d6b4d6bedad00e9e845e77f05f715e3969f8b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Daniel Verkamp
31965a7021 nvmf: don't store connection QID
Drop the debug print in conn.c that was the only user.

We still have the connect data structure when determining the connection
type, and after that point, the queue ID is not needed.

Change-Id: Ida9e170099f977ec6b84478874863c40d6f7d8a1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Daniel Verkamp
3f80d55199 CONFIG: rename CONFIG_NVMF to CONFIG_RDMA
The NVMf target is being refactored to split the RDMA transport-specific
code into its own file.  Once this is complete, we should be able to
plug in other transports and build the NVMf target without any RDMA
dependency if desired.

To enable this, change the CONFIG option to RDMA; it still controls
whether the whole NVMf target is built for now, but once the RDMA
dependency is actually made optional, we will be able to build the
generic NVMf target code without libibverbs installed.

Change-Id: I8cd90a9aaa85dcefcc9b0f8f2e7b6af21958b2a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Daniel Verkamp
51701f0053 nvmf: add headers for in6_addr
Fix build on FreeBSD.

Change-Id: I2f50e79b71ebbe2bba98722be092a7a4c06c8f19
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Ben Walker
c253f18656 nvmf: Move subsystem parsing to conf.c
Move the configuration file parsing for subsystems
into the configuration file parsing file.

Change-Id: Ie16e73cdc65fae7f2f3c3b22f9cba7f167024fa1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:36:54 -07:00
Ben Walker
1c603c11db nvmf: Move parsing of NVMe section to conf.c
Change-Id: Ie178d1cc393e36dfbd7dbfa5d5f6aad7327c10ba
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:35:22 -07:00
Ben Walker
f20727afbb nvmf: Rename init_grp to host in config parsing
The code for parsing the configuration file still
referred to a host as an init_grp, so fix it.

Change-Id: Ifa250b09de495dd7d393ccc3557fd6d56a54e790
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:35:22 -07:00
Ben Walker
32a70c1ffc nvmf: Remove subsystem groups
This never really made sense, so replace it with a list of
subsystems.

Change-Id: Ie7a9400083c091ac7142d01c23948200f515bdf7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:35:21 -07:00
Ben Walker
cf199a3f0f nvmf: Move map from subsystem group to subsystem
Change-Id: I8961f0a3f93a2c088819d207bf70c64147a68533
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:34:04 -07:00
Ben Walker
b531f5914e nvmf: Remove NodeBase. Each Subsystem now defines its own NQN
This is just extra complication for no real benefit.

Change-Id: I528af98e799d0641e753390fe35ff561fa3d7d76
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:34:04 -07:00
Dave Jiang
175b12e813 nvme: detect all uio drivers instead of just uio_pci_generic
This allows the custom UIO hotplug driver to be used as well as the
uio_pci_generic driver.

Change-Id: Ica3316ed716827ad305eb4a146d0864d61ff190f
Signed-off-by: Dave Jiang <dave.jiang@intel.com>
2016-07-13 10:34:47 -07:00
Ben Walker
35b3091eba event: Add a lib to configure with rpc
Linking this new library allows applications using
the framework to be killed via an RPC call.

This only works if the RPC subsystem is loaded.

Change-Id: Ifcf91c212add620fe410589eba5490337c635776
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-13 10:13:49 -07:00
Ben Walker
44d131105e log: Add a lib for integrating the log lib with event framework.
Linking this library into an application makes the log
configurable via RPC. This only works if the rpc subsystem
is also loaded.

Change-Id: If1340cf2a845ef159290232c26f341150c98fb9a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-13 10:13:48 -07:00
Ben Walker
548ab005af nvmf: Remove duplicated nqn from subsystem group
The nqn is in the subsystem.

Change-Id: I101dc45b7958d1e9148a09e85604a90936498d8c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-13 09:57:53 -07:00
GangCao
cd393178ca nvmf: replace the LU string with Subsystem string
Change-Id: Ib4f99a680c4b6994ad7744a3e81ce9b5ff1e4a8b
Signed-off-by: GangCao <gang.cao@intel.com>
2016-07-13 09:30:27 -07:00
Daniel Verkamp
5da444122e nvmf: remove arbitrary 4 RDMA device limit
Use the number of devices returned by ibv_get_device_list() instead of
stopping at 4.

While we're here, drop the unused MAX_SESSIONS_PER_DEVICE definition
too.

Change-Id: I21ca6c6c95b7f2cccc1de4d0a34b95217a522bfc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 17:20:26 -07:00
Daniel Verkamp
82ea261126 nvmf: move nvmf_complete_cmd to request.c
This is the only file that calls it, so it can be static.

Change-Id: I47573b7b38b40ad37e758234245eedbe94ae0a12
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 17:13:53 -07:00
Daniel Verkamp
f6d8e3d421 nvmf: drop nvmf_initialize() and nvmf_shutdown()
These were internal-only APIs; initialize just checks to see that the
pool was initialized (which is already checked internally), and shutdown
just called spdk_nvmf_shutdown_nvme(), which we can call directly.

Change-Id: I95e1b912d61a38fa9934f58df7b1512678303452
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 17:09:49 -07:00
Daniel Verkamp
6dfbcfa883 nvmf: mark debug string array as static
It is not used outside rdma.c.

Change-Id: Ic32aff4e07d6a430688e76b2dd223d5a5a61964e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 17:00:51 -07:00
Daniel Verkamp
411df9ad9b nvmf: make request_release and free_req private
These can be isolated in rdma.c rather than being part of the generic
transport API.

Change-Id: Idc2b969a2f7685420cda2f7c4aa12495ffc3fcbc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 16:58:17 -07:00
Daniel Verkamp
353533e229 nvmf: drop pool size macros
Just calculate the required number of requests once and store it in a
global variable.

Change-Id: Iffeb637a3ac5f69ec89989b84f03699bac483b6e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 16:25:05 -07:00
Daniel Verkamp
6d49820adc nvmf: remove MaxSessionsPerSubsystem config option
There can be only one session per subsystem.

Change-Id: I8ba85a5ebd11dd71fda2a4bafa97a0935609379f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:52:02 -07:00
Daniel Verkamp
6e415c4411 nvmf: drop global nvme_request_pool
It is just a duplicate of the NVMe library request_mempool.

Change-Id: I2a5484e5d515b965503b2cfcd8d85ccfcb0dee05
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:33:55 -07:00
Daniel Verkamp
dcc055e31f nvmf: move RDMA internal structs into rdma.c
Clean up everything that isn't strictly necessary in rdma.h.

Change-Id: Ied9acbed5f5b64860eae39816cdcb74620009a79
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:24:49 -07:00
Daniel Verkamp
1ade6e1dc0 nvmf: nest NVMf conn inside RDMA conn
This essentially turns the current nesting (of RDMA conn inside NVMf
conn) inside out.  Now the transport owns the connection structure and
allocates it when necessary.

Change-Id: Ib5ca84e2a57b16741d84943a5b858e9c3297d44b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:03:26 -07:00
Daniel Verkamp
f542245706 nvmf: move conn allocation to transport layer
This sets up the RDMA layer to be able to embed the NVMf conn inside the
RDMA conn.

Change-Id: I5e3714ac8503826504d78d06fb5eaafabd025bb8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:02:34 -07:00
Daniel Verkamp
fdc1278440 nvmf: simplify session and connection cleanup
The whole cleanup process is now started by
spdk_shutdown_nvmf_subsystems().  Each subsystem will clean up its
session, if any, and each session will clean up its connections.

Change-Id: I9915d4547751ed4ffc4baa2c45c628698dd0b881
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 15:01:34 -07:00
Daniel Verkamp
9903050dc1 nvmf: remove unused session tailq entry
Change-Id: Iaa8f9281fd448f0f48752aa4ae147756f3462406
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 13:04:12 -07:00
Daniel Verkamp
58f62ad5b1 nvmf: remove unused g_num_connections array
The per-lcore connection counter was incremented and decremented, but it
is no longer actually read.  The lcore allocation should happen at the
session level instead.

Change-Id: I7bdf1b521bfda4892304338d43fad3ed5123c494
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 12:51:20 -07:00
Daniel Verkamp
2eff92ab88 nvmf: drop connection shm and use calloc() instead
Nothing actually maps the shared memory region, so there is no need to
allocate the array of connections that way.

Change-Id: I3d5eca748f892e37fbb0ec52942f1c510e9f9dc8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-11 12:50:21 -07:00
Ben Walker
1fc4a182bd nvmf: Subsystem now has a pointer to the session
There is only one controller per subsystem, so therefore
there can be 0 or 1 sessions. Change the list of sessions
to a pointer that can be NULL if no session exists.

Change-Id: I2c0d042d9cecacae93da3e806093faf0155ddd6e
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-11 11:20:19 -07:00