162 Commits

Author SHA1 Message Date
Daniel Verkamp
1a37acda67 eofnl: check for extra trailing newlines
Enforce exactly one trailing \n, and fix all of the existing cases.

Change-Id: I6218e4700e90aeb647eaee78089530c79993c8c8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-10-11 13:30:33 -07:00
Changpeng Liu
8646b06a6e nvme: add guard e2e data protection tests for extended lba format
Change-Id: Ic39df70ebf1dbc22c9d1898f0a9608446fad4656
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-10-10 12:51:41 -07:00
Ben Walker
0dd80395f3 env: Move pci.c from util to env
This allows users to swap their PCI library from
libpciaccess/dpdk to another mechanism using the standard
method for swapping out the env library.

Change-Id: Ib2248f8b43754a540de2ec01897e571f0302b667
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:53:24 -07:00
Ben Walker
b9fbdd189a env: Move malloc/free wrappers into env
Change-Id: Ief591f5e23c4ae06cb77fab647a7afd082450a73
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:52 -07:00
Ben Walker
a4747c6048 env: Make the environment library configurable.
This allows users to swap out SPDK's third party
libraries for an implementation based on their own
framework.

Change-Id: Ia0b7384ce5e31acba5ad0d7002dec9e95b759c52
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 11:51:37 -07:00
Ben Walker
a30b5532cb memory: Rename to env
The new env library will wrap all third-party library
calls and be easily swappable with alternate implementations
at build time. For now, it's just the memory library
renamed.

Change-Id: I26a70933289f8137107208ba75f7520fd7a33da0
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-05 09:49:35 -07:00
Ben Walker
6b1e4e732d Drop libpciaccess and switch to DPDK PCI
This patch also drops support for automatically unbinding
devices from the kernel - run scripts/setup.sh first.

Our generic pci interface is now hidden behind include/spdk/pci.h
and implemented in lib/util/pci.c. We no longer wrap the calls
in nvme_impl.h or ioat_impl.h. The implementation now only uses
DPDK and the libpciaccess dependency has been removed. If using
a version of DPDK earlier than 16.07, enumerating devices
by class code isn't available and only Intel SSDs will be
discovered. DPDK 16.07 adds enumeration by class code and all
NVMe devices will be correctly discovered.

Change-Id: I0e8bac36b5ca57df604a2b310c47342c67dc9f3c
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 15:59:00 -07:00
Ben Walker
f4140ad023 nvme: Change the deallocate interface to generic dsm
Provide a convenience wrapper for general purpose dataset
management commands. The previous wrapper for deallocate
was difficult to use correctly and only for deallocate.

Note that the name is "dataset_management" as opposed to
"data_set_management" to match the NVMe specification.
It's questionable whether "dataset" is valid English, but
it is best to match the specification.

Change-Id: Ifc03d66dbabeabe8146968cf8a09f7ac3446ad68
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 14:36:09 -07:00
Ben Walker
d7bbac146c nvme: Add asserts to nvme_ns_cmd unit test.
Change-Id: I3295cf9a44fb6818bc8afe7dbdabbbdfa0003ee1
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-10-04 14:34:07 -07:00
Ben Walker
b0e349a804 nvme: Use log library instead of nvme_printf
Change-Id: Ic9b2db9bff3a914b3e5021695287157f1e076f9f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-09-28 10:15:55 -07:00
Daniel Verkamp
bb817900c9 test/nvme_impl.h: restore posix_memalign() check
If posix_memalign() fails, it may not have updated buf, so set it to
NULL explicitly.

Change-Id: I756bdc59ec1e31987ad3e6754eec4e2194b95074
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-09-27 15:34:34 -07:00
GangCao
372942e569 nvme: move global request_mempool allocation into nvme_impl
The user no longer needs to create the request pool.

Change-Id: I83bb8948143d4cc961d232f9f30df3106d5e0eab
Signed-off-by: GangCao <gang.cao@intel.com>
2016-09-27 14:25:39 -07:00
Jim Harris
edbed73064 Rename and move fd related helper functions to fd.c.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Ieccdd778348e8709ca4ef6cdf4f58f40021638db
2016-09-14 10:51:25 -07:00
Daniel Verkamp
5e9d859327 nvme: alloc buffer internally for non-I/O requests
Rather than forcing the NVMe library user to pass a specially-allocated
block of memory (e.g. rte_malloc() in the case of the default
nvme_impl.h), just make the NVMe library allocate a suitable buffer
itself and copy to/from the user buffer as needed.

The fast path I/O functions still require special rte_malloc()
allocations, since we don't want to add an allocation and copy to the
I/O critical path.

Change-Id: I7fe88c0ba60c859a33bbe95b7713f423c6bf1ea8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-09-13 12:47:46 -07:00
GangCao
20953e1ccd nvme: add the APIs to use the shared memory zone
Change-Id: I7faca95a15d320f3e2940c112b91d05a69797c90
Signed-off-by: GangCao <gang.cao@intel.com>
2016-09-06 13:18:16 -07:00
Ben Walker
0606eaad1a No longer wrap assert()
assert is part of the C standard library and is available
on any platform we'd consider porting to. Don't put a
wrapper around it.

Change-Id: I0acfdd6a8a269d6c37df38fb7ddf4f1227630223
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-19 10:53:06 -07:00
Ben Walker
888014289c nvme: No longer abstract away pthread calls
pthreads are widely supported and are available on any
platform we currently foresee porting to. Use that API
instead of attempting to abstract it away to simplify
the code.

Change-Id: I822f9c10910020719e94cce6fca4e1600a2d9f2a
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-08-19 10:53:06 -07:00
Daniel Verkamp
0b8198ce27 overhead: remove redundant system lib options
spdk.common.mk already adds -pthread and -lrt.

Change-Id: If6ea5aa5875ddd5043c2915dbe3e785a4dab7f56
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-08-19 09:47:57 -07:00
Tsuyoshi Uchida
0d2330c2df nvme: Add some status codes specified in NVM Express 1.2b (#37) 2016-08-19 08:52:31 -07:00
Daniel Verkamp
0b3bd6a9c5 nvme/overhead: track min and max submit/complete
Add slightly better statistics to get a range of values instead of just
the average.

Change-Id: I159994dce38412755afdd8980030c407125957e9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-08-17 09:32:37 -07:00
Tsuyoshi Uchida
81976ebd55 nvme: modify not to retry IOs on reset (#33)
When a controller reset is finished, all outstanding IOs and all queued
IOs which submitted before the reset are returned to the caller.
2016-08-15 10:01:02 -07:00
GangCao
b724e2e5a6 nvme: change global g_nvme_driver object to a pointer
This is a step towards enabling sharing SPDK NVMe
device access from multiple processes using DPDK's
multi-process framework.

Change-Id: I57d5eec158b42addc1036bd2583596471a467a95
Signed-off-by: GangCao <gang.cao@intel.com>
2016-08-04 15:45:35 -07:00
Changpeng Liu
aeb3d50603 nvme: add gitignore to overhead directory
Change-Id: I83f18ad923fe8b96034f75e752180f5a4462ad1f
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-08-02 14:33:21 -07:00
Daniel Verkamp
d054c97e65 test/nvme: add overhead to autotest
Change-Id: Ic52f23c80f8698d361064365336c32b394fd2c98
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-08-01 16:43:06 -07:00
Jim Harris
766da93b27 test: only apply overhead test heuristic for AIO test case
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I29ee2575ba13724baad4caac72b35c57b92347ac
2016-08-01 14:32:08 -07:00
Jim Harris
da214ab254 nvme: add new test application to measure SW overhead
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I330ca1577e5725b0e135422156328d5b165e79a3
2016-08-01 12:58:30 -07:00
Daniel Verkamp
b1860be94a test/nvme: run hello_world example during autotest
Change-Id: I31d1a5a7a03eb4dc105f31ec031d722f5d5563e6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-20 09:05:10 -07:00
Daniel Verkamp
0e93df5c34 nvme: combine CAP register into one 64-bit union
Previously, we used cap_lo and cap_hi to represent the 32-bit halves of
the full CAP register.  However, it is simpler to keep them in a single
64-bit structure, and is no less efficient on 64-bit platforms.

Also name the NSSRS field from NVMe 1.2, which was previously reserved.

Change-Id: I1d5d9b0dccbb12373b4aed3db29c883881d43223
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-07-07 13:52:16 -07:00
Daniel Verkamp
19fec6bb9c nvme: add remove callback to spdk_nvme_probe()
This will allow removal notifications to be propagated to the library
user (e.g. for hotplug).

The callback is currently unused, but this at least prepares the API for
the future hotplug support.

Based on a patch by Dave Jiang <dave.jiang@intel.com>

Change-Id: I20b1c2dbf5e084e0b45a7e51205aba4514ee9a95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:21:58 -07:00
Daniel Verkamp
e2d3cc6502 nvme: replace rte_memcpy with specialized function
Use the knowledge that both the source and destination of
nvme_copy_command() are aligned to emit the aligned variants of the
SSE2/AVX mov instructions.

Change-Id: I0a7e32a3bb10b9a1920cd85691b79fa7172eecb3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:17:21 -07:00
Daniel Verkamp
eb9d77a910 test/nvme: allocate aligned memory in unit tests
Match the expected alignment for nvme_alloc_request() and nvme_malloc()
to allow optimized memory copy code to work even in the unit tests.

Change-Id: I546692a6df9615a12a8209618fb6159a9c9e426b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-28 14:10:29 -07:00
Ben Walker
6b10df3576 nvme: Update nvme_spec.h to 1.2.1
This moves some definitions from nvmf_spec.h to
nvme_spec.h based on the latest publication.

Change-Id: I51b0abd16f7d034696239894aea5089f8ac70c40
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-06-24 09:18:14 -07:00
Daniel Verkamp
39cc59635a build: include -lrt and -lpthread in LDFLAGS
Simplify the build rules so that common libraries are always linked.

Also fix up a couple of -lpciaccess instances that should have been
changed to $(PCIACCESS_LIB).

Change-Id: I4c50fa3aa59cae013d3385e38fbb830794299f6e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-06-23 09:04:02 -07:00
GangCao
c65210d034 nvme: return error immediately when enabling controller failed
Change-Id: Id9cf6873cc831bd5099df49db95ef5073badf461
Signed-off-by: GangCao <gang.cao@intel.com>
2016-06-21 16:37:15 -04:00
Ziye Yang
7991eb1957 Create a utility and enable WRR arbitration mechanism
Change-Id: I2194863ae812ece72c17b78e003ccf7895b8a812
Signed-off-by: GangCao <gang.cao@intel.com>
2016-06-21 16:35:04 -07:00
Daniel Verkamp
ab1f6bdc54 nvme: add enum for opcode data direction
NVMe opcodes contain a two-bit field that encodes the expected data
direction for each command.  Add an enum and a function to extract these
bits.

Change-Id: Ie214319f121cf0899c6aa5663866f2988b128dd2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-27 08:59:14 -07:00
Ben Walker
ddb59105ee nvme: Add an fio plugin
Change-Id: Ife815e371325df7fbec388530dc09097c3e4ffac
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-05-18 13:51:36 -07:00
Daniel Verkamp
09d3e4c9dd test/nvme: use SPDK_CU_ASSERT_FATAL()
Use SPDK_CU_ASSERT_FATAL instead of CU_ASSERT_FATAL so static analyzers
recognize that g_request cannot be NULL in the following lines.

Change-Id: Ie7ab3bd34a177bea0d565441014e8db12be8bb01
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-16 10:43:10 -07:00
Changpeng Liu
ca3d1c5b45 spdk: add controller memory buffer support in driver
The D3700/D3600 series support Controller Memory Buffer(CMB) feature,
CMB is available for holding submission queues, for those controllers
which can support submission queues in CMB, user can set the option
whether to enable it or not.

Change-Id: I8b0dc9e28dd6f5bb01bee99a532087212c04e492
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-05-13 08:14:10 +08:00
Cunyin Chang
f2168e1d73 nvme: Add firmware upgrade interface and unit test suite
Change-Id: If66e5f97f6793df0388629fab7c3d0e9f9d5eb67
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-05-11 10:18:33 -07:00
Daniel Verkamp
cd48a01fcb build: wrap $(CURDIR) relative paths in $(abspath)
Resolve relative paths before using them to clean up command lines.

This should also help shorten the overall command line length that gets
embedded in the binary and used when locating the executable from a
coredump.

Change-Id: Ibff9849ede198bb04313496c8b7131485ffaf14f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-09 13:56:07 -07:00
Daniel Verkamp
0313b2bbed test/nvme: initialize payload.md in ctrlr_cmd_ut
Fix copy of uninitialized data in the unit tests.

Change-Id: Ib71a6fc90edb07f7ddac3067ff0efbda7938bf17
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-05-06 12:52:05 -07:00
Cunyin Chang
10f0f4b0e9 spdk: Add Intel specific log page.
This patch add support for Intel specific log pages :
marketing description page.

Change-Id: I87bccb2af286279598c9dd3c870094b384a0d2f7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
2016-05-05 13:39:14 +08:00
Fangfang Wei
1e9c43ddc7 spdk: Add ns write with metadata unit test
Change-Id: I65dd28aed568dc5d9098ed389701f65e9d0e1925
Signed-off-by: Fangfang Wei <fangfangx.wei@intel.com>
2016-05-04 14:58:41 -07:00
Changpeng Liu
784182ed47 SPDK: Add end-to-end data protection support in driver
For those controllers which can support end-to-end data protection
feature, add the support in the driver layer.

Change-Id: Ifac3dd89dec9860773c850416a6116113a6ce22a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-04-28 15:46:13 -07:00
Liang Yan
b2db4f94ec nvme/test: Add hw sgl request unit test
Change-Id: I1364a6634b9be25ff647eee56c91e312ae3d004e
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-04-28 14:26:24 +08:00
Liang Yan
11932ec118 nvme/test: Update unit test for the changes of PRP entries number
Change-Id: Ib58a0da307f9c5db36f6b14acd122cf2e2a0d377
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-04-18 13:54:26 -07:00
Daniel Verkamp
2cf675bb29 nvme: rename remove_child_request for consistency
Rename nvme_remove_child_request() to nvme_request_remove_child() and
move it next to nvme_request_add_child() to make the symmetry clear.

Change-Id: I78747c44ab3db1a656b33555a45f634dc5a55b31
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-04-08 16:51:15 -07:00
Daniel Verkamp
87a3244f97 nvme: handle CC.EN = 0 && CSTS.RDY = 1 on startup
This can happen if the controller is still resetting as the SPDK NVMe
driver takes control.

Change-Id: I263ae8f2e7b271e0448450557452a115c90c4fb6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-04-08 16:47:25 -07:00
Ziye Yang
deb90a93de SPDK: add nvme_remove_child_request helper function
This patch is used to add a nvme_request remove child
helpler function

Change-Id: I1e5bb228d53333ca3601f4ae30fcd801ea39e532
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-04-08 09:36:57 -07:00