Commit Graph

7941 Commits

Author SHA1 Message Date
Daniel Verkamp
bba5f43ce5 kperf: use $(MAKE) to invoke make
Change-Id: I51e587abe2c032ff97ea3133e53dd3bc36dd58cf
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-23 16:36:37 -07:00
Daniel Verkamp
ab7541d810 kperf: add .gitignore entries
Change-Id: I80725118e44e147e12dcac31d0ba58f175a89d10
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-23 16:36:37 -07:00
Daniel Verkamp
add9186f73 README: cd to DPDK dir in a subshell
Tweak the build instructions slightly to make them simpler to follow.

If someone is reading the instructions top to bottom, the section about
how to build SPDK assumes the current directory is the top-level SPDK
repository, but if the reader just finished building DPDK, they would be
in the DPDK directory.

Change-Id: Ie3dd587bcf1ac41ac3579a75f9b329b0eeaafd7a
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-23 12:24:52 -07:00
Ben Walker
55dc5f2131 Add vfio support to scripts.
cleanup.sh and unbind.sh have been combined into a single
setup.sh that takes one optional parameter (reset). If no
parameter is given, the script will automatically bind
all NVMe and IOAT devices to either uio_pci_generic
or vfio-pci, as appropriate based on IOMMU settings. If
the reset parameter is given, the devices will be bound back
to the appropriate kernel drivers.

Change-Id: I25db3234f1ecfb352a281e5093f4c1aa455152ae
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-02-22 11:30:47 -07:00
Daniel Verkamp
c0a1cd3827 ioat.h: remove duplicated spdk/pci.h inclusion
Change-Id: I9371faa22bf79a670b1ffceb5620e5fad6e58079
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-22 09:57:11 -07:00
Daniel Verkamp
5dafa4940f nvme.h: include stdbool.h and stdint.h
Explicitly include system headers for types that are used in public
headers.

These were being pulled in by example code, so SPDK itself would build,
but other apps that did not include stdbool.h would fail to compile when
including spdk/nvme.h.

Also include nvme.h first in nvme_internal.h so this case gets tested
during normal compilation.

Change-Id: I8ed0fc4e0dcf71551738c461b4b825cc2ee1d233
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-22 09:57:11 -07:00
Daniel Verkamp
093d5479f9 spdk_nvme_ns_cmd_reservation_report: fix request allocation
payload here is a pointer to the buffer, not a struct nvme_payload.

Use nvme_allocate_request_contig() and pass the length in bytes rather
than dwords.

Change-Id: Idbbb3614b1d69148fe041d26e0c148bd9ce53724
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-19 15:24:09 -07:00
Daniel Verkamp
959bc24b0a autotest: run ioat_ut under Valgrind
Change-Id: I235bd39c0f4c85f12db20a87976ce73cfdb89bc2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-19 15:24:09 -07:00
Daniel Verkamp
85d69f7c36 autotest: enforce Valgrind checking for unit tests
If any memory is leaked or other errors occur, valgrind will now exit
with an error code and fail the tests.

Change-Id: I6c65cae17dcd21eb97d70bd82df828eaa0f8f109
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-19 15:24:09 -07:00
Daniel Verkamp
c162dd0e38 nvme_qpair_ut: clean up submitted tracker
This fixes a memory leak in the unit test.

Change-Id: Ie94459e8e46e966c437ad43702a04f1a8f9becdb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-19 13:15:34 -07:00
Daniel Verkamp
639fee5f13 nvme: remove initialization of nvme_reqest::parent
The parent field is in the cache line of nvme_request that is only
supposed to be accessed for split (child) I/Os.  All accesses to parent
are done from child-specific calls now, so it does not need to be
initialized in the common case of a non-split I/O.
nvme_request_add_child() will set parent when splitting occurs.

Change-Id: Ib86c16ba1ea2ce32f62079831101da2a099047af
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-18 15:15:36 -07:00
Daniel Verkamp
7c34c2005d nvme: store SGL callback in nvme_payload
Allows simplification of _nvme_qpair_build_sgl_request(), which does not
need to know whether a request is a child or not.

This also removes a read of req->parent for non-split I/Os; the parent
field is in the section of nvme_request that is only intended to be
initialized for split I/Os, which should be detected by looking at
num_children.

Additionally, this fixes a potential problem if requests were nested
more than one level deep (e.g. req->parent was not the original user
request).

Change-Id: I3ea1dc134bbb1e3b8c6b5a479f5d760bd97ea848
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-18 15:07:18 -07:00
Ziye Yang
d0a543daf6 SPDK: fix the compliation issues in ioat_impl.h
This patch fixes the following issues in ioat_impl.h
while CONFIG_PCIACCESS is set to n:

1 lack of the definition of ioat_pcicfg_unmap_bar.
2 Define a macro to remove the duplicated vendor code
for ioat_driver_id enumration.

Change-Id: I4011098ac296d1ec320bffb5ffa6e098b70a5545
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-18 09:42:58 -07:00
Daniel Verkamp
9322c25808 doc: add spdk/ path to Doxygen indexes
Reference the public headers in the way that apps should include them
(e.g. spdk/nvme.h rather than just nvme.h).

Change-Id: I0489d906e9c55ff00ffa4e8c9532b0941ce69449
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-17 12:18:27 -07:00
Changpeng Liu
6d497a8e88 spdk: add NVMe controller memory buffer definitions
Change-Id: I49de87a502b979e1457bde4e850a081faee31cae
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-02-18 10:31:23 +08:00
Andrey Kuzmin
5bb66add46 Include rte_config.h before other DPDK headers
Building the tip of the spdk master against the dpdk-2.2.0 fails with
inappropriate RTE_CACHE_LINE_SIZE error. The simple reversal of the RTE
include file order below fixed it for me.

Change-Id: I8782b7ee21d7f185e6e678f874fbdab9403117a5
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-16 14:44:41 -07:00
Ziye Yang
accf229829 SPDK: remove the duplicated code in ioat/nvme_impl.h
This patch is used to remove the duplicated code.
As we found the structure in "ifdef and else" are same.

Change-Id: I1717ce3dcc14134ac59c165d801e5e811b987be5
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-16 14:40:12 -07:00
Ziye Yang
3d7bcab40a SPDK: Add spdk_pci_device_cfg read/write for byte, word.
This patch is used to support spdk_pci_device_cfg read/write
with byte and word size.

Change-Id: I49084e231bd6b5f5b22180a3eb36ddad4430b3a4
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-16 09:54:43 -07:00
Daniel Verkamp
04adec22bc Add extern "C" wrapper to all public headers
Also consistently place the extern "C" before any includes or other
declarations in files that already had it.

Change-Id: Ia316d5be3e509ec76c4a98cfa90ed516073351e0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-12 07:52:35 -07:00
Daniel Verkamp
bfd92d4719 nvme: update doc index file with spdk_ prefixes
Change-Id: I5745d1f9af63547b05f4dfdd364f18bcf615f2b0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-11 13:45:58 -07:00
Daniel Verkamp
5cab054f4e ioat: add spdk_ prefixes
Change-Id: Ic42f204a9a4e65661931af2220e8f8ac77242a69
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-11 12:35:57 -07:00
Daniel Verkamp
c7150a5611 ioat_spec: add spdk_ prefixes
Change-Id: I91444f10b98d7e247af5eb2fea719e283a1156a2
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-11 10:39:28 -07:00
Daniel Verkamp
6ce73aa6e7 nvme: add spdk_ prefix
Shorten commonly-used names:
controller -> ctrlr
namespace -> ns

Change-Id: I64f0ce7c65385bab0283f8a8341a3447792b3312
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-10 11:27:45 -07:00
Daniel Verkamp
ad35d6cd86 nvme_spec: add spdk_ prefixes
Use shorter names for commonly-used objects:
namespace -> ns
controller -> ctrlr
command -> cmd
completion -> cpl

Change-Id: I97d192546b35a6aeb76ad3a709f65631502cde71
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-09 11:06:48 -07:00
Daniel Verkamp
7672976a6a nvme: move class code to pci_ids.h
Move toward collecting PCI IDs, class codes, etc. in pci_ids.h instead
of individual device-specific headers.

Change-Id: Icff162d48ac663db71d0576ceee16a9bd7a751cd
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-09 09:32:49 -07:00
Daniel Verkamp
8332f9e47e nvme_intel: add spdk_ prefixes and tweak names
Change-Id: I7c256bce365c92636f4f183e218117a1d7fe63d9
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 16:53:36 -07:00
Daniel Verkamp
516c37562d vtophys: add spdk_ prefix
vtophys() -> spdk_vtophys()
VTOPHYS_ERROR -> SPDK_VTOPHYS_ERROR

Change-Id: I68ab24fbb48f419ba1d41b78d7c9958cf666b800
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 14:28:56 -07:00
Daniel Verkamp
40c591eac8 string: add spdk_ prefix
sprintf_alloc() -> spdk_sprintf_alloc()

Change-Id: I24970baa37615633572d132abe3e57d0889f1a48
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 14:28:56 -07:00
Daniel Verkamp
8cb09df68e pci_ids: add SPDK_ prefix
PCI_VENDOR_ID_INTEL -> SPDK_PCI_VID_INTEL

Also change the inclusion guard macro to be consistent with the other
SPDK headers.

Change-Id: I29346267172cb8c07cc4289eed4eca2d55e942d6
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 14:28:47 -07:00
Daniel Verkamp
87844a30ef nvme: move struct pci_id into nvme_internal.h
This doesn't need to be part of the public API. It is only used by the
NVMe quirk lookup tables.

Change-Id: I7662e333c70b7c5f814bd7c8a528b6bff1f0732e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 14:00:16 -07:00
Daniel Verkamp
aae6ee8c80 file: add spdk_ prefix to public APIs
file_get_size() -> spdk_file_get_size()
dev_get_blocklen() -> spdk_dev_get_blocklen()

Change-Id: Iba1eb4a22ba331887e2c5c3a16e1c0f9e695d83e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 13:44:34 -07:00
Daniel Verkamp
53e45aee8c barrier: add spdk_ prefix to wmb() and mb()
Change-Id: Ie5e724e34cbcd8ef9feb9054c867fcb0065f5a1d
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 13:32:19 -07:00
Ziye Yang
6a1463deee nvme: add definitions for nvme namespace commands
This patch is used to add enumerations for nvme
namespace attach, detach, create, and delete commands.

Change-Id: I03fb40438ba137affaf81dcf69479c83a4bb9c09
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-08 12:39:01 -07:00
Liang Yan
ee292e4b55 spdk: Add callback NULL check for readv and writev command
Change-Id: I04286c97ff0536127dbbc4bd1d65ee44a128ecf5
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-02-08 12:33:34 -07:00
Liang Yan
7a9b004c65 spdk: Add reservation cmd unit test
Change-Id: I859376e792c2be981d8877b068126f8c9a192abd
Signed-off-by: Liang Yan <liangx.yan@intel.com>
2016-02-08 11:02:30 -07:00
Daniel Verkamp
93933831f7 pci: clean up public pci.h interface
Rename all functions with a spdk_ prefix, and provide enough of an API
to avoid apps needing to #include <pciaccess.h>.

The opaque type used in the public API for a PCI device is now
struct spdk_pci_device *.

Change-Id: I1e7a09bbc5328c624bec8cf5c8a69ab0ea8e8254
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 09:58:13 -07:00
Daniel Verkamp
9a924a066e pci: factor out PCI enumeration into util lib
This is a step toward abstracting PCI access so that libpciaccess can be
swapped out more easily.

Change-Id: I5491459460cbfbd0be471f70f9d07a7eb3175234
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 09:58:13 -07:00
Jim Harris
ff7e2122c7 nvme: reduce number of mmio writes in completion path
Instead of writing the completion doorbell once per completion,
just write it once at the end of the completion while loop.
This reduces the number of mmio writes by coalescing several
writes into one when we get multiple completions at a time.

Signed-off-by: Jim Harris <james.r.harris@intel.com>
Change-Id: I3cc3864dcfe43186bec51be1a732e84ef3be05ae
2016-02-08 09:52:15 -07:00
Daniel Verkamp
047c5aaaa8 ioat: refactor ioat_attach() API into ioat_probe()
Similar to the NVMe API change, this allows better abstraction of the
PCI subsystem.

Change-Id: I2b84d9c3c498a08d4451b4ff27d0865f0456c210
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 09:50:57 -07:00
Daniel Verkamp
53d6437bb5 unbind.sh: fix ioat unbind to work with 'set -e'
Since the grep would fail for most devices and the result wasn't
checked, the script would exit on the first device ID that didn't match.

Also initialize $rootdir so ioat_pci.h can be located.

Change-Id: I6a714772e22692ab002e1971134d2a86edc81d92
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-08 09:50:57 -07:00
Ziye Yang
3c10754a8d SPDK: add the support to use pci functions provided by DPDK
This patch did the work to use pci related functions
provided by DPDK.

Change-Id: I263b79f1b42868ef0c1efcf1bc392a4b3a328e93
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-08 09:33:54 -07:00
Ziye Yang
ff6125195c SPDK: Support configuring libpciaccess library
With CONFIG_PCIACCESS=y in CONFIG file, we can
use libpciaccess library; With CONFIG_PCIACCESS=n
in CONFIG file, we use pciaccess functions provided
in DPDK.

Change-Id: I786c5589b8e7909ba2e59d222938dd5ba45bf92d
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-04 17:30:09 -07:00
Daniel Verkamp
8374a727a9 nvme: refactor nvme_attach() into nvme_probe()
The new probing API will find all NVMe devices on the system and ask the
caller whether to attach to each one.  The caller will then receive a
callback once each controller has finished initializing and has been
attached to the driver.

This will enable cleanup of the PCI abstraction layer (enabling us to
use DPDK PCI functionality) as well as allowing future work on parallel
NVMe controller startup and PCIe hotplug support.

Change-Id: I3cdde7bfab0bc0bea1993dd549b9b0e8d36db9be
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-03 11:15:31 -07:00
Haomai Wang
20c767e796 nvme: add support for write zeroes command
Change-Id: I07ae5805c434d3c06ac24c1a8e09c761b5506ff4
Signed-off-by: Ben Walker <benjamin.walker@intel.com>
2016-02-03 10:28:35 -07:00
Daniel Verkamp
5578f74dd4 test/nvme/sgl: fix dereference before NULL check
Change-Id: I7976f4598d61df501dc917bc77bec225c907b9b1
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-01 15:45:22 -07:00
Daniel Verkamp
b235c942fa examples/ioat: fix initialization error handling
Previously, ioat_init() in the example programs was returning 0 even if
initialization failed.

Change-Id: I96b2ec5646f7051ab881611acff424fb8547d5eb
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-02-01 15:45:16 -07:00
Ziye Yang
2c94318f92 Update bind.sh to bind nvme/ioat to uio_pci_generic driver
This patch is used to detect the nvme and ioat devices
in the tested platform and bind it to uio_pci_generic
driver. Thus, when we can smoothly use pci device functions
provided by DPDK instead of using libpciaccess.

Change-Id: Ibfe504fd2864c9b4980eacd0adc5ff2a029b5eaf
Signed-off-by: Ziye Yang <ziye.yang@intel.com>
2016-02-01 15:37:00 -07:00
Changpeng Liu
8aa497f083 spdk: Add block fill API to ioat driver
For those Crystal Beach DMA channels which support block fill capability,
we add a fill API here that can zero out pages or fill them with a
fixed pattern.

Change-Id: I8a57337702b951c703d494004b111f6d206279fb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
2016-02-02 13:29:07 +08:00
Daniel Verkamp
0ca6afb850 Add .gitignore for test/lib/nvme/sgl/sgl
Change-Id: I0bd68c19382111542e83498af3e4d070a1a1c076
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-01-29 15:05:04 -07:00
Daniel Verkamp
26b1964c8f pci: fix spdk_pci_device_get_dev() accessor
spdk_pci_device_get_dev() needs to access the 'dev' member of a struct
pci_device, but the pointer to the struct is also called 'dev' in the
macro argument list.  Change it to something else so the 'dev' member
doesn't get replaced by the macro argument.

This is the minimal bug fix; more cleanup will happen later as the PCI
interface is refactored.

Change-Id: I37b4a303f980d40c5b1528293ecfd22ad0be9dc7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
2016-01-29 14:26:58 -07:00