The IOAT "ring" is actually just a circular linked list of descriptors;
the descriptors do not need to be in a single physically contiguous
region. This can be accomodated by calling spdk_vtophys() on each
descriptor rather than assuming they are all in a single contiguous
region.
Also store the physical address of each descriptor in its associated
software descriptor context to avoid the need to call spdk_vtophys()
during runtime.
Change-Id: Ic8636bbc61deb496a0c6d0ea56b75d298f5f426c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-on: https://review.gerrithub.io/417782
Reviewed-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Reviewed-by: Changpeng Liu <changpeng.liu@intel.com>
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
This code was created when spdk_pci_device wasn't
available yet. Now we can use spdk_pci_device* instead
of void* for extra code clarity.
Change-Id: I81d440720b22a484ae3d2739e0510a021bebbafe
Signed-off-by: Dariusz Stojaczyk <dariuszx.stojaczyk@intel.com>
Reviewed-on: https://review.gerrithub.io/416995
Tested-by: SPDK Automated Test System <sys_sgsw@intel.com>
Reviewed-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Daniel Verkamp <daniel.verkamp@intel.com>
Reviewed-by: Jim Harris <james.r.harris@intel.com>
Add a parameter to each I/OAT library function that requires a channel
instead of implicitly using the thread-local channel registration model.
I/OAT channels are already reported by the spdk_ioat_probe() attach
callback, so no infrastructure for channel allocation is necessary.
Change-Id: I8731126fcaea9fe2bafc41a3f75c969a100ef8f0
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Similar to the NVMe API change, this allows better abstraction of the
PCI subsystem.
Change-Id: I2b84d9c3c498a08d4451b4ff27d0865f0456c210
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
For those Crystal Beach DMA channels which support block fill capability,
we add a fill API here that can zero out pages or fill them with a
fixed pattern.
Change-Id: I8a57337702b951c703d494004b111f6d206279fb
Signed-off-by: Changpeng Liu <changpeng.liu@intel.com>
Also add a space between Copyright and (c).
The copyright year can be determined using git metadata.
Also remove the duplicated "All rights reserved." - every instance of
this line already has a corresponding "All rights reserved" immediately
below it, except for examples/ioat/kperf/kmod/dma_perf.c, where I have
added it manually.
Performed using this command:
git ls-files | xargs sed -i -e 's/Copyright(c) \(.*\) Intel Corporation. All rights reserved./Copyright (c) Intel Corporation./'
Change-Id: I3779f404966800709024eb1eb66a50068af2716c
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The ioat library currently only supports DMA copy operations, but the
hardware can do other types of transfers. Add a union of the hardware
descriptor structures to enable support for the other operations in the
future.
Also add a generic hardware descriptor type to allow access to the parts
of the descriptor that are common between all types.
Change-Id: I3b54421ce771f58b78910e790b53026f311f918e
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
Rather than individually allocating each ring entry, use two large
allocations, one for the hardware descriptors and one for the software
descriptor contexts.
This allows the use of simple array indexing on the rings and also
allows the removal of most of the software descriptor structure,
since the necessary information can be retrieved based on the ring
index now.
Change-Id: I73ef24450f69ca0fc35e350286282c6b1c77a207
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
NVMe doesn't require the specific 64-bit MMIO ordering on 32-bit
platforms performed in spdk_mmio_read_8(), but it doesn't hurt.
We have to pick one of the two possible orderings, so pick the one
required by I/OAT.
Change-Id: I2b909d64d0c077b797d0f64a11d78d1ecc55eec7
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
The ioat driver supports DMA engine copy offload hardware available on
Intel Xeon platforms.
Change-Id: Ida0b17b25816576948ddb1b0443587e0f09574d4
Signed-off-by: Daniel Verkamp <daniel.verkamp@intel.com>
