If any memory is leaked or other errors occur, valgrind will now exit
with an error code and fail the tests.
Change-Id: I6c65cae17dcd21eb97d70bd82df828eaa0f8f109
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This fixes a memory leak in the unit test.
Change-Id: Ie94459e8e46e966c437ad43702a04f1a8f9becdb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The parent field is in the cache line of nvme_request that is only
supposed to be accessed for split (child) I/Os. All accesses to parent
are done from child-specific calls now, so it does not need to be
initialized in the common case of a non-split I/O.
nvme_request_add_child() will set parent when splitting occurs.
Change-Id: Ib86c16ba1ea2ce32f62079831101da2a099047af
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Allows simplification of _nvme_qpair_build_sgl_request(), which does not
need to know whether a request is a child or not.
This also removes a read of req->parent for non-split I/Os; the parent
field is in the section of nvme_request that is only intended to be
initialized for split I/Os, which should be detected by looking at
num_children.
Additionally, this fixes a potential problem if requests were nested
more than one level deep (e.g. req->parent was not the original user
request).
Change-Id: I3ea1dc134bbb1e3b8c6b5a479f5d760bd97ea848
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch fixes the following issues in ioat_impl.h
while CONFIG_PCIACCESS is set to n:
1 lack of the definition of ioat_pcicfg_unmap_bar.
2 Define a macro to remove the duplicated vendor code
for ioat_driver_id enumration.
Change-Id: I4011098ac296d1ec320bffb5ffa6e098b70a5545
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Reference the public headers in the way that apps should include them
(e.g. spdk/nvme.h rather than just nvme.h).
Change-Id: I0489d906e9c55ff00ffa4e8c9532b0941ce69449
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Building the tip of the spdk master against the dpdk-2.2.0 fails with
inappropriate RTE_CACHE_LINE_SIZE error. The simple reversal of the RTE
include file order below fixed it for me.
Change-Id: I8782b7ee21d7f185e6e678f874fbdab9403117a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to remove the duplicated code.
As we found the structure in "ifdef and else" are same.
Change-Id: I1717ce3dcc14134ac59c165d801e5e811b987be5
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
This patch is used to support spdk_pci_device_cfg read/write
with byte and word size.
Change-Id: I49084e231bd6b5f5b22180a3eb36ddad4430b3a4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Also consistently place the extern "C" before any includes or other
declarations in files that already had it.
Change-Id: Ia316d5be3e509ec76c4a98cfa90ed516073351e0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Move toward collecting PCI IDs, class codes, etc. in pci_ids.h instead
of individual device-specific headers.
Change-Id: Icff162d48ac663db71d0576ceee16a9bd7a751cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
PCI_VENDOR_ID_INTEL -> SPDK_PCI_VID_INTEL
Also change the inclusion guard macro to be consistent with the other
SPDK headers.
Change-Id: I29346267172cb8c07cc4289eed4eca2d55e942d6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This doesn't need to be part of the public API. It is only used by the
NVMe quirk lookup tables.
Change-Id: I7662e333c70b7c5f814bd7c8a528b6bff1f0732e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to add enumerations for nvme
namespace attach, detach, create, and delete commands.
Change-Id: I03fb40438ba137affaf81dcf69479c83a4bb9c09
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Rename all functions with a spdk_ prefix, and provide enough of an API
to avoid apps needing to #include <pciaccess.h>.
The opaque type used in the public API for a PCI device is now
struct spdk_pci_device *.
Change-Id: I1e7a09bbc5328c624bec8cf5c8a69ab0ea8e8254
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This is a step toward abstracting PCI access so that libpciaccess can be
swapped out more easily.
Change-Id: I5491459460cbfbd0be471f70f9d07a7eb3175234
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Instead of writing the completion doorbell once per completion,
just write it once at the end of the completion while loop.
This reduces the number of mmio writes by coalescing several
writes into one when we get multiple completions at a time.
Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3cc3864dcfe43186bec51be1a732e84ef3be05ae
Similar to the NVMe API change, this allows better abstraction of the
PCI subsystem.
Change-Id: I2b84d9c3c498a08d4451b4ff27d0865f0456c210
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Since the grep would fail for most devices and the result wasn't
checked, the script would exit on the first device ID that didn't match.
Also initialize $rootdir so ioat_pci.h can be located.
Change-Id: I6a714772e22692ab002e1971134d2a86edc81d92
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch did the work to use pci related functions
provided by DPDK.
Change-Id: I263b79f1b42868ef0c1efcf1bc392a4b3a328e93
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
With CONFIG_PCIACCESS=y in CONFIG file, we can
use libpciaccess library; With CONFIG_PCIACCESS=n
in CONFIG file, we use pciaccess functions provided
in DPDK.
Change-Id: I786c5589b8e7909ba2e59d222938dd5ba45bf92d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
The new probing API will find all NVMe devices on the system and ask the
caller whether to attach to each one. The caller will then receive a
callback once each controller has finished initializing and has been
attached to the driver.
This will enable cleanup of the PCI abstraction layer (enabling us to
use DPDK PCI functionality) as well as allowing future work on parallel
NVMe controller startup and PCIe hotplug support.
Change-Id: I3cdde7bfab0bc0bea1993dd549b9b0e8d36db9be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Previously, ioat_init() in the example programs was returning 0 even if
initialization failed.
Change-Id: I96b2ec5646f7051ab881611acff424fb8547d5eb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This patch is used to detect the nvme and ioat devices
in the tested platform and bind it to uio_pci_generic
driver. Thus, when we can smoothly use pci device functions
provided by DPDK instead of using libpciaccess.
Change-Id: Ibfe504fd2864c9b4980eacd0adc5ff2a029b5eaf
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
For those Crystal Beach DMA channels which support block fill capability,
we add a fill API here that can zero out pages or fill them with a
fixed pattern.
Change-Id: I8a57337702b951c703d494004b111f6d206279fb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
spdk_pci_device_get_dev() needs to access the 'dev' member of a struct
pci_device, but the pointer to the struct is also called 'dev' in the
macro argument list. Change it to something else so the 'dev' member
doesn't get replaced by the macro argument.
This is the minimal bug fix; more cleanup will happen later as the PCI
interface is refactored.
Change-Id: I37b4a303f980d40c5b1528293ecfd22ad0be9dc7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Optionally enable and display the I/O latency histograms as reported by
the hardware if supported (e.g. Intel DC P3x00 NVMe devices).
Change-Id: I5c0138d51a282138b74f36fe8e1461c9444e6d0f
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
This patch adds Intel NVMe device list and overrides the
supported log pages according to the quirk list.
In particular, the READ_CMD_LATENCY and WRITE_CMD_LATENCY pages are
supported on Intel DC P3x00 devices despite not being listed in the
Intel vendor-specific log page directory.
Change-Id: I3a2b6a5fa142c6e9c93567df65e85980bd3c7cc0
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
Also add a space between Copyright and (c).
The copyright year can be determined using git metadata.
Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.
Performed using this command:
git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'
Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
1 Add new API nvme_ctrlr_is_feature_supported().
2 Add unit test for new API.
Change-Id: Ia6d8710755c3b13984fca9d56700efe043be1402
Signed-off-by: Cunyin Chang <cunyin.chang@intel.com>
This will allow replacing these _nvme_fail_request_bad_vtophys() calls
with the correct error later. vtophys is not actually used within the
SGL request builder, so this is the wrong error.
Change-Id: Ibc2a3b029a8abad1d563b9df200325d7d64498da
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
No code change, just moved into a function for readability.
Change-Id: I883443c06d961c6dbeffed1a6fb153177e6e3fcd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
This cleans up the I/O splitting code somewhat.
It also moves the SGL payload function pointers up into the hot cache
section of struct nvme_request without pushing the other important
members past the cacheline boundary (because payload is now a union).
Change-Id: I14a5c24f579d57bb84d845147d03aa53bb4bb209
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
