163 Commits

Author SHA1 Message Date
Daniel Verkamp
3f80d55199 CONFIG: rename CONFIG_NVMF to CONFIG_RDMA
The NVMf target is being refactored to split the RDMA transport-specific
code into its own file.  Once this is complete, we should be able to
plug in other transports and build the NVMf target without any RDMA
dependency if desired.

To enable this, change the CONFIG option to RDMA; it still controls
whether the whole NVMf target is built for now, but once the RDMA
dependency is actually made optional, we will be able to build the
generic NVMf target code without libibverbs installed.

Change-Id: I8cd90a9aaa85dcefcc9b0f8f2e7b6af21958b2a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-14 12:58:17 -07:00
Ben Walker
32a70c1ffc nvmf: Remove subsystem groups
This never really made sense, so replace it with a list of
subsystems.

Change-Id: Ie7a9400083c091ac7142d01c23948200f515bdf7
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-14 10:35:21 -07:00
Ben Walker
11b0c3cc51 nvmf: Standardize rdma init/fini functions
Change-Id: Ifa453e865e110cd534d34aae1bc036cc9f09b1bd
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-07-11 09:06:30 -07:00
Daniel Verkamp
0e93df5c34 nvme: combine CAP register into one 64-bit union
Previously, we used cap_lo and cap_hi to represent the 32-bit halves of
the full CAP register.  However, it is simpler to keep them in a single
64-bit structure, and is no less efficient on 64-bit platforms.

Also name the NSSRS field from NVMe 1.2, which was previously reserved.

Change-Id: I1d5d9b0dccbb12373b4aed3db29c883881d43223
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-07 13:52:16 -07:00
Cunyin Chang
adcbbe19ff event: Add dpdk framework start function into event.
Make sure the reactor mask in profile take effect.

Change-Id: Ia471b2b88a711f05738cf93068c4f3a8c9a3039d
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-07-01 10:07:42 -07:00
Changpeng Liu
4fbe54f10e nvmf: add discovery subsystem to NVMf target
Change-Id: I4ee79ad268ae75208feddd62e22d6210a9c0d944
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-06-30 16:20:09 -07:00
Daniel Verkamp
19fec6bb9c nvme: add remove callback to spdk_nvme_probe()
This will allow removal notifications to be propagated to the library
user (e.g. for hotplug).

The callback is currently unused, but this at least prepares the API for
the future hotplug support.

Based on a patch by Dave Jiang <dave.jiang@intel.com>

Change-Id: I20b1c2dbf5e084e0b45a7e51205aba4514ee9a95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:21:58 -07:00
Daniel Verkamp
e2d3cc6502 nvme: replace rte_memcpy with specialized function
Use the knowledge that both the source and destination of
nvme_copy_command() are aligned to emit the aligned variants of the
SSE2/AVX mov instructions.

Change-Id: I0a7e32a3bb10b9a1920cd85691b79fa7172eecb3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:17:21 -07:00
Daniel Verkamp
eb9d77a910 test/nvme: allocate aligned memory in unit tests
Match the expected alignment for nvme_alloc_request() and nvme_malloc()
to allow optimized memory copy code to work even in the unit tests.

Change-Id: I546692a6df9615a12a8209618fb6159a9c9e426b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:10:29 -07:00
Daniel Verkamp
2a3e39a154 nvmf: add spdk_ prefix to struct nvmf_request
Also finish up the req_state -> req conversion.

Change-Id: I131dd52dcd36a790b942e06f0207a3274cc04ffc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-27 10:24:00 -07:00
Ben Walker
d7fbeda710 nvmf: Move process_*_cmd functions to request.h/c
Change-Id: I16481463a8735f71387c69dd9ca4e187d743d8fc
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-24 11:30:59 -07:00
Ben Walker
2a2615ec6e nvmf: Unify completion path of fabric and other commands.
Fabric commands were skipping a step, so unify all
types of requests through the same completion path.

Change-Id: I5f38a7e1cdcdf33baf71486d5ddae9f5a6157fac
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-24 09:25:19 -07:00
Ben Walker
6b10df3576 nvme: Update nvme_spec.h to 1.2.1
This moves some definitions from nvmf_spec.h to
nvme_spec.h based on the latest publication.

Change-Id: I51b0abd16f7d034696239894aea5089f8ac70c40
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-24 09:18:14 -07:00
Daniel Verkamp
1d3ee6c5f1 nvmf: remove nvmf_request cb_fn field
It is always set to nvmf_process_async_completion and is only used
within the library.

Also rename nvmf_process_async_completion to spdk_nvmf_request_complete
to clarify its purpose.

Change-Id: Ie737fb60688329bfe329a8553c4a40ff2e5f8f1d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 11:12:19 -07:00
Daniel Verkamp
8836a17f80 nvmf: drop redundant nvmf_process_admin_cmd params
Everything necessary for processing an admin command is now stored in
nvmf_request.

Change-Id: I74e75a5b7bb3b406ad167c2b31cab1af7a1f270a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 11:12:19 -07:00
Daniel Verkamp
2f5ccef310 nvmf: remove redundant nvmf_process_io_cmd params
Everything necessary for processing an I/O is now stored in
nvmf_request.

Change-Id: I3f390707ebe83ea66a116dcfda4d0388a6823629
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 11:12:19 -07:00
Daniel Verkamp
39cc59635a build: include -lrt and -lpthread in LDFLAGS
Simplify the build rules so that common libraries are always linked.

Also fix up a couple of -lpciaccess instances that should have been
changed to $(PCIACCESS_LIB).

Change-Id: I4c50fa3aa59cae013d3385e38fbb830794299f6e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 09:04:02 -07:00
Daniel Verkamp
4db1b420f4 vtophys: fix usermode address range check
vtophys() was checking for out-of-range addresses incorrectly: vfn_2mb
is already shifted to account for 2 MB hugepages, but it was being
compared with a mask that did not account for the shift.  This would
allow out-of-bounds access to the 128tb map array for certain invalid
addresses (it had no effect on addresses within the valid userspace
range).

Change-Id: Ida7455595e586494c9025f9ba65d050abb16b1b9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-22 16:41:29 -07:00
GangCao
c65210d034 nvme: return error immediately when enabling controller failed
Change-Id: Id9cf6873cc831bd5099df49db95ef5073badf461
Signed-off-by: GangCao <gang.cao@intel.com>
2016-06-21 16:37:15 -04:00
Ziye Yang
7991eb1957 Create a utility and enable WRR arbitration mechanism
Change-Id: I2194863ae812ece72c17b78e003ccf7895b8a812
Signed-off-by: GangCao <gang.cao@intel.com>
2016-06-21 16:35:04 -07:00
Ben Walker
422b147499 nvmf: Standardize names in init_grp.h
Prefix with spdk_nvmf_host.

Change-Id: I053030febf5b7157cc2c7075ac6170bc59ce0ced
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-17 13:31:05 -07:00
Ben Walker
6eb1bee64b nvmf: Rename spdk_nvmf_init_grp to spdk_nvmf_host
Host is the term used in the specification.

Change-Id: Ie9a06e3ff176e00e2fdcdc5bede701fe768d4b35
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-17 13:31:05 -07:00
Daniel Verkamp
1c089ae19c nvmf: rename framework.c to nvmf.c
Change-Id: Ia9b4bf4738ad9e1ae158f3a610da8f0ab87e0215
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-08 13:15:26 -07:00
Daniel Verkamp
d237632434 nvmf: add unit tests
Change-Id: I22c57c3989f4b69cb35ef4e17507864d0f07f084
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-08 08:49:38 -07:00
Daniel Verkamp
c711736d6f test/log: fix typo
Change-Id: Iff84a6e17a2e6bf5848fee33d5bf5ed289058c7c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-01 09:48:39 -07:00
Daniel Verkamp
eeeac6676d Add event-driven application framework
Change-Id: Iba90db6d8853dde972b4eec2c35eb44eeddae780
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-31 09:58:05 -07:00
Daniel Verkamp
ab1f6bdc54 nvme: add enum for opcode data direction
NVMe opcodes contain a two-bit field that encodes the expected data
direction for each command.  Add an enum and a function to extract these
bits.

Change-Id: Ie214319f121cf0899c6aa5663866f2988b128dd2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-27 08:59:14 -07:00
Daniel Verkamp
376d117c90 jsonrpc: add JSON-RPC 2.0 library
Change-Id: I4f58792c3af1f85f55144717478f868ebe5b1700
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-23 10:28:58 -07:00
Ben Walker
ddb59105ee nvme: Add an fio plugin
Change-Id: Ife815e371325df7fbec388530dc09097c3e4ffac
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-05-18 13:51:36 -07:00
Daniel Verkamp
09d3e4c9dd test/nvme: use SPDK_CU_ASSERT_FATAL()
Use SPDK_CU_ASSERT_FATAL instead of CU_ASSERT_FATAL so static analyzers
recognize that g_request cannot be NULL in the following lines.

Change-Id: Ie7ab3bd34a177bea0d565441014e8db12be8bb01
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-16 10:43:10 -07:00
Daniel Verkamp
53cb5d8fa6 test/json: initialize variable to silence warning
If spdk_json_number_to_int32() works correctly, it will initialize the
value of i32, but since this is a unit test, we should make sure i32 is
initialized to catch bugs.

Fixes a clang warning during scan-build.

Change-Id: Iecf0531d2992acf584b10e2de5f8c53553a9f3b6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-16 10:41:04 -07:00
Changpeng Liu
ca3d1c5b45 spdk: add controller memory buffer support in driver
The D3700/D3600 series support Controller Memory Buffer(CMB) feature,
CMB is available for holding submission queues, for those controllers
which can support submission queues in CMB, user can set the option
whether to enable it or not.

Change-Id: I8b0dc9e28dd6f5bb01bee99a532087212c04e492
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-05-13 08:14:10 +08:00
Daniel Verkamp
ed694026b8 json: fix spdk_json_write_val with nested objects
When recursively calling spdk_json_write_val() on another object or
array, the child call will handle printing out the whole
subobject/array, so the parent call should skip over all of its values.

Also fix the return value for the array/object case - if we get to the
end of the array or object, we should return 0 for success.

Change-Id: I1da80c88ab8759620114c1ab141baaaaf9f0023a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-11 10:43:02 -07:00
Cunyin Chang
f2168e1d73 nvme: Add firmware upgrade interface and unit test suite
Change-Id: If66e5f97f6793df0388629fab7c3d0e9f9d5eb67
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-05-11 10:18:33 -07:00
Daniel Verkamp
aaa533adc7 log: add SPDK logging library
Change-Id: I3d4060b619a3976ffff75c05f86214e3fcd18fb5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 15:54:49 -07:00
Daniel Verkamp
cd48a01fcb build: wrap $(CURDIR) relative paths in $(abspath)
Resolve relative paths before using them to clean up command lines.

This should also help shorten the overall command line length that gets
embedded in the binary and used when locating the executable from a
coredump.

Change-Id: Ibff9849ede198bb04313496c8b7131485ffaf14f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 13:56:07 -07:00
Daniel Verkamp
f9193f4ce7 json: add JSON parser and encoder libraries
Change-Id: Id73fb7e300d66d31a7c3986c0334b6f56e284905
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 13:30:02 -07:00
Daniel Verkamp
0313b2bbed test/nvme: initialize payload.md in ctrlr_cmd_ut
Fix copy of uninitialized data in the unit tests.

Change-Id: Ib71a6fc90edb07f7ddac3067ff0efbda7938bf17
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-06 12:52:05 -07:00
Cunyin Chang
10f0f4b0e9 spdk: Add Intel specific log page.
This patch add support for Intel specific log pages :
marketing description page.

Change-Id: I87bccb2af286279598c9dd3c870094b384a0d2f7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-05-05 13:39:14 +08:00
Fangfang Wei
1e9c43ddc7 spdk: Add ns write with metadata unit test
Change-Id: I65dd28aed568dc5d9098ed389701f65e9d0e1925
Signed-off-by: Fangfang Wei <fangfangx.wei@intel.com>
2016-05-04 14:58:41 -07:00
Changpeng Liu
784182ed47 SPDK: Add end-to-end data protection support in driver
For those controllers which can support end-to-end data protection
feature, add the support in the driver layer.

Change-Id: Ifac3dd89dec9860773c850416a6116113a6ce22a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-04-28 15:46:13 -07:00
Liang Yan
b2db4f94ec nvme/test: Add hw sgl request unit test
Change-Id: I1364a6634b9be25ff647eee56c91e312ae3d004e
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-04-28 14:26:24 +08:00
Liang Yan
11932ec118 nvme/test: Update unit test for the changes of PRP entries number
Change-Id: Ib58a0da307f9c5db36f6b14acd122cf2e2a0d377
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-04-18 13:54:26 -07:00
Daniel Verkamp
2cf675bb29 nvme: rename remove_child_request for consistency
Rename nvme_remove_child_request() to nvme_request_remove_child() and
move it next to nvme_request_add_child() to make the symmetry clear.

Change-Id: I78747c44ab3db1a656b33555a45f634dc5a55b31
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-04-08 16:51:15 -07:00
Daniel Verkamp
87a3244f97 nvme: handle CC.EN = 0 && CSTS.RDY = 1 on startup
This can happen if the controller is still resetting as the SPDK NVMe
driver takes control.

Change-Id: I263ae8f2e7b271e0448450557452a115c90c4fb6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-04-08 16:47:25 -07:00
Ziye Yang
deb90a93de SPDK: add nvme_remove_child_request helper function
This patch is used to add a nvme_request remove child
helpler function

Change-Id: I1e5bb228d53333ca3601f4ae30fcd801ea39e532
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-04-08 09:36:57 -07:00
Daniel Verkamp
42e10f3e01 autotest: only run NVMe AER test in nightly tests
Change-Id: I39c54db5474ec5fe56f83f1297aa2d0c0445baaf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-28 15:11:55 -07:00
Cunyin Chang
eae688576a spdk: Add nvme format interface and unit tests.
Change-Id: Ie0506debf547a5fc011e116421387a5adb7abf0e
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-03-17 13:37:20 -07:00
Daniel Verkamp
e3eba25d92 test/nvme/sgl: free SGL element buffers
Change-Id: I1bfcbda4722ae42bd54433bf43e5c954b0775f01
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-16 15:53:56 -07:00
Daniel Verkamp
af058fd758 test/nvme/sgl: calculate physical address earlier
The next_sge callback can be called multiple times for a single I/O if
the I/O is split.  Get the physical address for each element once
instead of every time the SGL is walked.

Also bake the SGL element offset into the physical address.

Change-Id: I5d83426a234ffe9d13b89621077dbbb1561181c9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-03-16 15:53:56 -07:00