Make sure the trace history that is exported via shared memory is always
the same size, regardless of DPDK configuration.
Also removes the necessity of including DPDK headers from spdk/trace.h
(so we have to fix up other files to include what they use).
Change-Id: I32f88921fd95c64a9d1f4ba768ae75e2ca5d91da
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make SPDK_SUBSYSTEM_REGISTER export a symbol that
SPDK_SUBSYSTEM_DEPEND will reference as an extern.
This makes sure that dependencies are included by the linker without
needing -Wl,--whole-archive around the subsystem libraries.
Change-Id: Id2e3c589d5a49bba1df6603a8e27dd2a7e7b44b7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
These NVMf spec structure definitions are the same as the equivalent
NVMe structs.
Change-Id: I21c45973b7843e3767c48f97ec42e7b446df296f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For now, it just contains RDMA, plus a raw byte array to allow generic
copying.
Change-Id: I02fe11f99dd8b49000de0dba991cd34c99fd7a4a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Create a list of valid properties with get and set callbacks (set is
optional to allow read-only fields).
Remove handling for fields declared as "reserved" in the NVMe over
Fabrics 1.0 specification.
Also simplify the vcprop structure to only contain the required fields.
Change-Id: I14d3ddfd008c62b75fce8e64d193c87fb6f7b5ad
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This is intended to be used for examples/nvme/identify and similar
diagnostic utilities.
Change-Id: Ib2f941e9af7a3fb7555865ef253742e30ccad2b5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, we used cap_lo and cap_hi to represent the 32-bit halves of
the full CAP register. However, it is simpler to keep them in a single
64-bit structure, and is no less efficient on 64-bit platforms.
Also name the NSSRS field from NVMe 1.2, which was previously reserved.
Change-Id: I1d5d9b0dccbb12373b4aed3db29c883881d43223
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Make sure the reactor mask in profile take effect.
Change-Id: Ia471b2b88a711f05738cf93068c4f3a8c9a3039d
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
NVMe over Fabrics defines its own NVMe Qualified Name (NQN) format; it
does not use iSCSI Qualified Names.
Also change the default node base for nvmf_tgt to "nqn.2016-06.io.spdk".
Change-Id: I2b73c1426ef1d8c83cc2df499d79228ea61257cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This will allow removal notifications to be propagated to the library
user (e.g. for hotplug).
The callback is currently unused, but this at least prepares the API for
the future hotplug support.
Based on a patch by Dave Jiang <dave.jiang@intel.com>
Change-Id: I20b1c2dbf5e084e0b45a7e51205aba4514ee9a95
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
It is the same as spdk_nvme_cpl, aside from reserved fields.
Change-Id: I62b0718dd58c998b4d26a0d1b44ee16d37eff25d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This moves some definitions from nvmf_spec.h to
nvme_spec.h based on the latest publication.
Change-Id: I51b0abd16f7d034696239894aea5089f8ac70c40
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
Other subsystems can depend on this one and then use
SPDK_RPC_REGISTER to register RPC functions.
Change-Id: I557f774331ce7146d299d06b3f81426e2103a11f
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
NVMe opcodes contain a two-bit field that encodes the expected data
direction for each command. Add an enum and a function to extract these
bits.
Change-Id: Ie214319f121cf0899c6aa5663866f2988b128dd2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, there was no way to determine what namespace ID was assigned
when a namespace was created via the NVMe library interface.
Also drop the incorrect comment about calling
spdk_nvme_ctrlr_process_admin_completions(), since
spdk_nvme_ctrlr_create_ns() checks the admin queue internally.
Change-Id: If90a6e9fc773aefa220ebbf6effc2d033c9f20cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
These allow linear searches of the configuration file
sections.
Change-Id: I8d8b9594bc8a974c16d999689a6195434c1efac8
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
The D3700/D3600 series support Controller Memory Buffer(CMB) feature,
CMB is available for holding submission queues, for those controllers
which can support submission queues in CMB, user can set the option
whether to enable it or not.
Change-Id: I8b0dc9e28dd6f5bb01bee99a532087212c04e492
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
This patch add support for Intel specific log pages :
marketing description page.
Change-Id: I87bccb2af286279598c9dd3c870094b384a0d2f7
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
For existing \file markers, move them to the top of the header and tweak
the wording for consistency.
Change-Id: Icce748effe4dbe97d79a8c87d31caf0ee5797058
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Function to trim leading and trailing whitespace from a string.
Originally based on code imported from istgt.
Change-Id: I87abe584130bdf4930098fadb8e57291f18eda7f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Parsing function for delimited strings with embedded quotes.
Originally based on code imported from istgt.
Change-Id: I448feb53ea232048ed8c68738e12bc3660eb4235
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add function to convert string to lowercase in place.
Originally based on code imported from istgt.
Change-Id: Ica9fe2208e6ee09b22c9a652a33c5affe5be23cc
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For those controllers which can support end-to-end data protection
feature, add the support in the driver layer.
Change-Id: Ifac3dd89dec9860773c850416a6116113a6ce22a
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Fix up a few minor formatting issues while we're here.
Change-Id: I849572c64b4464257d878fd0a8b2dc2cc3fc6dcb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
spdk_nvme_qpair_process_completions() is already documented in
spdk/nvme.h, so merge the doc comment from nvme_qpair.c into the public
header.
Change-Id: Id7722d99d209852ee64286e0a3fa127b863e10aa
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Only 65535 I/O queues are allowed, not 65536.
Each I/O queue must have a unique queue ID, and since queue ID 0 is
reserved for the admin queue, there can be only 65535 unique I/O queue
IDs.
Change-Id: I51875e5a72e08ec1727365ca9706c43f3142fd83
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This does not work reliably, since the line number is not unique when
multiple headers that use the assert macro are included.
The automated test systems will catch any issues that would
theoretically be missed by contributors compiling with old
pre-static_assert compilers.
Change-Id: I0ded2eeb61d20a28065358cf568a329749283155
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The type field is in the 4 high bits per the spec, but nvme_spec.h had it
in the low 4 bits.
Change-Id: I04dcb8f194bdcd28da803dd512b0ac5709ce773f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Provide a new structure, spdk_nvme_ctrlr_opts, to let the user modify
the default controller initialization options during probe/attach.
Currently, only the number of queue pairs can be modified in this way;
other options will be added later.
Change-Id: Ie27b9429291d93a9353c0d820f0ad467d3b0e7cb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add a parameter to each I/OAT library function that requires a channel
instead of implicitly using the thread-local channel registration model.
I/OAT channels are already reported by the spdk_ioat_probe() attach
callback, so no infrastructure for channel allocation is necessary.
Change-Id: I8731126fcaea9fe2bafc41a3f75c969a100ef8f0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The previous method for registering I/O queues did not allow the user
to specify queue priority for weighted round robin arbitration, and it
limited the application to one queue per controller per thread.
Change the API to require explicit allocation of each queue for each
controller using the new function spdk_nvme_ctrlr_alloc_io_qpair().
Each function that submits a command on an I/O queue now takes an
explicit qpair parameter rather than implicitly using the thread-local
queue.
This also allows the application to allocate different numbers of
threads per controller; previously, the number of queues was capped at
the smallest value supported by any attached controller.
Weighted round robin arbitration is not supported yet; additional
changes to the controller startup process are required to enable
alternate arbitration methods.
Change-Id: Ia33be1050a6953bc5a3cca9284aefcd95b01116e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch just implement the feature of format progress indicator.
the NVMe available does't support FPI currently.
Change-Id: Ie937591fb1720d8a062354322aabcc95ff14b2d3
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
nvme_spec.h already has a structure with the correct bitfields for the
CSTS register, so use it in struct spdk_nvme_registers.
Change-Id: Id0663aee2611fb5195f9012a3176799e32701bb0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This function returns true if the namespace is active or false if it is
inactive (e.g. no namespace has been attached to the specified namespace
ID yet).
Also use the new function to add checks in the examples and tests where
applicable.
Change-Id: I35465b315ae1a1677c5a82191ad9b1da1c216d50
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Explicitly include system headers for types that are used in public
headers.
These were being pulled in by example code, so SPDK itself would build,
but other apps that did not include stdbool.h would fail to compile when
including spdk/nvme.h.
Also include nvme.h first in nvme_internal.h so this case gets tested
during normal compilation.
Change-Id: I8ed0fc4e0dcf71551738c461b4b825cc2ee1d233
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to support spdk_pci_device_cfg read/write
with byte and word size.
Change-Id: I49084e231bd6b5f5b22180a3eb36ddad4430b3a4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Also consistently place the extern "C" before any includes or other
declarations in files that already had it.
Change-Id: Ia316d5be3e509ec76c4a98cfa90ed516073351e0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Move toward collecting PCI IDs, class codes, etc. in pci_ids.h instead
of individual device-specific headers.
Change-Id: Icff162d48ac663db71d0576ceee16a9bd7a751cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
PCI_VENDOR_ID_INTEL -> SPDK_PCI_VID_INTEL
Also change the inclusion guard macro to be consistent with the other
SPDK headers.
Change-Id: I29346267172cb8c07cc4289eed4eca2d55e942d6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This doesn't need to be part of the public API. It is only used by the
NVMe quirk lookup tables.
Change-Id: I7662e333c70b7c5f814bd7c8a528b6bff1f0732e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to add enumerations for nvme
namespace attach, detach, create, and delete commands.
Change-Id: I03fb40438ba137affaf81dcf69479c83a4bb9c09
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Rename all functions with a spdk_ prefix, and provide enough of an API
to avoid apps needing to #include <pciaccess.h>.
The opaque type used in the public API for a PCI device is now
struct spdk_pci_device *.
Change-Id: I1e7a09bbc5328c624bec8cf5c8a69ab0ea8e8254
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This is a step toward abstracting PCI access so that libpciaccess can be
swapped out more easily.
Change-Id: I5491459460cbfbd0be471f70f9d07a7eb3175234
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Similar to the NVMe API change, this allows better abstraction of the
PCI subsystem.
Change-Id: I2b84d9c3c498a08d4451b4ff27d0865f0456c210
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch did the work to use pci related functions
provided by DPDK.
Change-Id: I263b79f1b42868ef0c1efcf1bc392a4b3a328e93
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
With CONFIG_PCIACCESS=y in CONFIG file, we can
use libpciaccess library; With CONFIG_PCIACCESS=n
in CONFIG file, we use pciaccess functions provided
in DPDK.
Change-Id: I786c5589b8e7909ba2e59d222938dd5ba45bf92d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
The new probing API will find all NVMe devices on the system and ask the
caller whether to attach to each one. The caller will then receive a
callback once each controller has finished initializing and has been
attached to the driver.
This will enable cleanup of the PCI abstraction layer (enabling us to
use DPDK PCI functionality) as well as allowing future work on parallel
NVMe controller startup and PCIe hotplug support.
Change-Id: I3cdde7bfab0bc0bea1993dd549b9b0e8d36db9be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For those Crystal Beach DMA channels which support block fill capability,
we add a fill API here that can zero out pages or fill them with a
fixed pattern.
Change-Id: I8a57337702b951c703d494004b111f6d206279fb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
spdk_pci_device_get_dev() needs to access the 'dev' member of a struct
pci_device, but the pointer to the struct is also called 'dev' in the
macro argument list. Change it to something else so the 'dev' member
doesn't get replaced by the macro argument.
This is the minimal bug fix; more cleanup will happen later as the PCI
interface is refactored.
Change-Id: I37b4a303f980d40c5b1528293ecfd22ad0be9dc7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch adds Intel NVMe device list and overrides the
supported log pages according to the quirk list.
In particular, the READ_CMD_LATENCY and WRITE_CMD_LATENCY pages are
supported on Intel DC P3x00 devices despite not being listed in the
Intel vendor-specific log page directory.
Change-Id: I3a2b6a5fa142c6e9c93567df65e85980bd3c7cc0
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Also add a space between Copyright and (c).
The copyright year can be determined using git metadata.
Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.
Performed using this command:
git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'
Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
1 Add new API nvme_ctrlr_is_feature_supported().
2 Add unit test for new API.
Change-Id: Ia6d8710755c3b13984fca9d56700efe043be1402
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
A namespace indicates support for reservations by reporting a non-zero
value in the Reservation Capabilities field in the Identify Namespace
data structure, and controller indicates support for reservation in the
Identify Controller data structure, Here we used namespace field as the
support flag.
Change-Id: I0e1e29548aa3fc8b6d3bbeb4149ec4864316f092
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Support for the Force Unit Access and Limited Retry
bits on reads and writes.
Change-Id: I9860848358377d63a967a4ba6ee9c061faf284d4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
This patch is used to wrap the inner implementation
of libpciaccess and prepare for the same interface
to applications in the future patch
Change-Id: I4d40fae0bd86b451ed38dbfd9bcc015f9bfc8436
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
NVMe reservations provide capabilities that may be used by two or more
hosts to coordinate access to a shared namespace, here we add the 4
reservation commands: reservation register/acquire/release/report.
Change-Id: Ib03ae2120a57dd14aa64311a6ffeb39fda73018c
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
For the purpose to support different types of input scattered payloads,
such as iovs or scattered list, we define common method in the NVMe
driver, users should implement their own functions to iterate each
segment memory.
Change-Id: Id2765747296a66997518281af0db04888ffc4b53
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Change nvme_ctrlr_is_log_page_supported() to match
nvme_ctrlr_cmd_get_log_page().
Change-Id: I4c8a1f11044b083f8f8990ef40a4f789fa3c24e3
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Request allocation may fail, so we need a way to indicate failure to the
caller.
Change-Id: I278c3f42e4d2fa1902bb0ab33ad3bf7c7007fd0d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
1 Add supported log pages data structure.
2 Bulid up supported log pages when NVME start.
3 Provide unified API for getting log pages.
3 Unit test suit optimization base on above modification.
Change-Id: I03cdb93f5c94e6897510d7f19bc7d9f4e70f9222
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
This more accurately represents what function it performs.
Also remove pci_device_has_uio_driver() from the public API. Callers
should use pci_device_has_non_uio_driver() instead.
Change-Id: I9623fe1345b43e981d5823804e33d01ac0d3bb1c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_ctrlr_process_io_completions() and
nvme_ctrlr_process_admin_completions() now return the number of
completions processed.
This also adds the possibility of returning an error from the
process_*_completions functions (currently unused, but this at least
gets the API ready in case error conditions are added later).
Change-Id: I1b32ee4f2f3c1c474d646fa2d6b8b7bbb769785f
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
_Static_assert is new in C11, and older versions of GCC don't support
it. Detect whether static_assert is available in <assert.h> and if not,
define a fallback macro.
Change-Id: Iaf513677e5f356832e0a3d1f6b1f4f9ea07e6627
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The ioat library currently only supports DMA copy operations, but the
hardware can do other types of transfers. Add a union of the hardware
descriptor structures to enable support for the other operations in the
future.
Also add a generic hardware descriptor type to allow access to the parts
of the descriptor that are common between all types.
Change-Id: I3b54421ce771f58b78910e790b53026f311f918e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
According to the specification, the dataset management for deallocate
attribute can support to 256 ranges, so we should use uint16_t
instead of uint8_t as the ranges parameter.
Change-Id: Ibacc00da8b4b9e2b2f3454d382aadf7ad353ff31
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Add support for the full set of Optional NVM Command Support bits from
NVMe 1.2, and interpret the various SGL-related feature bits.
Change-Id: Ie3fcca27571b5236479e05a1ea3b834adbeb18a8
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This lets us signal an error if the channel is halted in
ioat_process_channel_events().
Change-Id: Iffaf4fd1e27d1254f9d95a37d732ae4a5f3a0465
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
NVMe doesn't require the specific 64-bit MMIO ordering on 32-bit
platforms performed in spdk_mmio_read_8(), but it doesn't hurt.
We have to pick one of the two possible orderings, so pick the one
required by I/OAT.
Change-Id: I2b909d64d0c077b797d0f64a11d78d1ecc55eec7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The ioat driver supports DMA engine copy offload hardware available on
Intel Xeon platforms.
Change-Id: Ida0b17b25816576948ddb1b0443587e0f09574d4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
nvme_ctrlr_process_io_completions() now takes a second parameter,
max_completions, to let the user limit the number of I/Os completed on
each poll.
If there are many I/Os waiting to be completed, the
nvme_ctrlr_process_io_completions() function could run for a long time
before returning control to the user, so the max_completions parameter
lets the user have more control of latency.
Change-Id: I3173059d94ec1cc5dbb636fc0ffd3dc09f3bfe4b
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Like sprintf() with automatic buffer allocation.
This should help to avoid fixed-size buffers in
non-performance-sensitive code that formats strings.
Change-Id: I35209ae84014ed5daf41baa5b03af8a5f6b02b8e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This helps enable FreeBSD, where pciaccess pci_device_has_kernel_driver()
is not functional. The function will return 0 if there is no driver
attached, or the Linux uio or FreeBSD nic_uio driver is attached. It will
return 1 otherwise.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I0921e61c9040b1e0411b5dc40b36fc7f2721c8c5
Then only include this new file on Linux, to help enable building
SPDK on FreeBSD.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: Id818a2d8662c95e603d8b580f1f4b92889cc626e
