Update ioat probe function to set the driver name for
secondary process.
Fixes: bc230b569e7f ("raw/ioat: support secondary process")
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Add support for secondary processes in ioat devices. The update
allocates a memzone for a primary process or returns it in a
secondary process.
Signed-off-by: Kumar Amber <kumar.amber@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Coverity flags a possible problem where the 8-bit wq_idx value may have
errors when shifted and sign-extended to pointer size. Since this can
only occur if the shift index is larger than any expected value from
hardware, it's unlikely to cause any real problems, but we can eliminate
any possible errors, and the coverity issue, by explicitly typecasting
the uint8_t value to uintptr_t before any shift operations occur.
Coverity issue: 363695
Fixes: a33969462135 ("raw/ioat: fix work-queue config size")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
A lot of drivers export headers, reproduce the same facility than for
libraries.
Note: this change fixes an issue with the crypto scheduler headers which
were not installed properly. A separate backport will be sent to stable
branches.
Suggested-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Since each version map file is contained in the subdirectory of the library
it refers to, there is no need to include the library name in the filename.
This makes things simpler in case of library renaming.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
There is a null pointer check in 'idxd_vdev_parse_params()' which is
causing a coverity issue. This check is redundant as the same check is
being done in 'rte_kvargs_free()', so it is simply removed in this patch.
In addition, kvlist was only being free'd on one path in this function.
This is fixed by always free'ing kvlist before returning.
Coverity issue: 363049
Fixes: 777edf43ae4f ("raw/ioat: introduce vdev probe for DSA/idxd device")
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
The 'idxd' pointer in 'idxd_rawdev_destroy()' is being dereferenced before
it is checked. To fix this, the null pointer check was moved to occur
earlier in the code.
Coverity issue: 363040
Fixes: ff06fa2cf3ba ("raw/ioat: probe idxd PCI")
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Add fill operation enqueue support for IOAT and IDXD. The fill enqueue is
similar to the copy enqueue, but takes a 'pattern' rather than a source
address to transfer to the destination address. This patch also includes an
additional test case for the new operation type.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Now that all devices can pass the same set of unit tests, eliminate the
temporary idxd_rawdev_test function and move the prototype for
ioat_rawdev_test to the proper internal header file, to be used by all
device instances.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add update of the relevant stats for the data path functions and point the
overall device struct xstats function pointers to the existing ioat
functions.
At this point, all necessary hooks for supporting the existing unit tests
are in place so call them for each device.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
The xstats functions can be used by all ioat devices so move them from the
ioat_rawdev.c file to ioat_common.c, and add the function prototypes to the
internal header file.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Rather than having the xstats as fields inside the main driver structure,
create a separate structure type for them.
As part of the change, when updating the stats functions referring to the
stats by the old path, we can simplify them to use the id to directly index
into the stats structure, making the code shorter and simpler.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add the info get function for DSA devices, returning just the ring size
info about the device, same as is returned for existing IOAT/CBDMA devices.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add support for doing copies using DSA hardware. This is implemented by
just switching on the device type field at the start of the inline
functions. Since there is no hardware which will have both device types
present this branch will always be predictable after the first call,
meaning it has little to no perf penalty.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add the start and stop functions for DSA hardware devices using the
vfio/uio kernel drivers. For vdevs using the idxd kernel driver, the device
must be started using sysfs before the device node appears for vdev use -
making start/stop functions in the driver unnecessary.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add configure function for idxd devices, taking the same parameters as the
existing configure function for ioat. The ring_size parameter is used to
compute the maximum number of bursts to be supported by the driver, given
that the hardware works on individual bursts of descriptors at a time.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add in the relevant data structures for the data path for DSA devices. Also
include a device dump function to output the status of each device.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
For each vdev (DSA work queue) instance, create a rawdev instance.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
When a matching device is found via PCI probe create a rawdev instance for
each queue on the hardware. Use empty self-test function for these devices
so that the overall rawdev_autotest does not report failures.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Devices managed by the idxd kernel driver must be configured for DPDK use
before it can be used by the ioat driver. This example script serves both
as a quick way to get the driver set up with a simple configuration, and as
the basis for users to modify it and create their own configuration
scripts.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
The Intel DSA devices can be exposed to userspace via kernel driver, so can
be used without having to bind them to vfio/uio. Therefore we add support
for using those kernel-configured devices as vdevs, taking as parameter the
individual HW work queue to be used by the vdev.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add in the basic probe/remove skeleton code for DSA devices which are bound
directly to vfio or uio driver. The kernel module for supporting these uses
the "idxd" name, so that name is used as function and file prefix to avoid
conflict with existing "ioat" prefixed functions.
Since we are adding new files to the driver and there will be common
definitions shared between the various files, we create a new internal
header file ioat_private.h to hold common macros and function prototypes.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Only a few definitions from the hardware spec are actually used in the
driver runtime, so we can copy over those few and make the rest of the spec
a private header in the driver.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Rather than having the fence signalled via a flag on a descriptor - which
requires reading the docs to find out whether the flag needs to go on the
last descriptor before, or the first descriptor after the fence - we can
instead add a separate fence API call. This becomes unambiguous to use,
since the fence call explicitly comes between two other enqueue calls. It
also allows more freedom of implementation in the driver code.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Since the hardware supported by the ioat driver is capable of operations
other than just copies, we can rename the doorbell and completion-return
functions to not have "copies" in their names. These functions are not
copy-specific, and so would apply for other operations which may be added
later to the driver.
Also add a suitable warning using deprecation attribute for any code using
the old functions names.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Rather than having a single long complicated header file for general use we
can split things so that there is one header with all the publicly needed
information - data structs and function prototypes - while the rest of the
internal details are put separately. This makes it easier to read,
understand and use the APIs.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
Add a flag which controls whether rte_ioat_enqueue_copy and
rte_ioat_completed_copies function should process handle parameters. Not
doing so can improve the performance when handle parameters are not
necessary.
Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
In case of any failures, include the function name and the line number of
the error message in the message, to make tracking down the failure easier.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
To allow the header file to be used from C++ code we need to ensure all
typecasts are explicit, and include an 'extern "C"' guard.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
When rte_rawdev_pmd_release() is called, rte_rawdev_close() looks for a
dev_close function for the device causing a segmentation fault when no
close() function is implemented for a driver.
This patch resolves the issue by adding a stub function ioat_dev_close().
Fixes: f687e842e328 ("raw/ioat: introduce IOAT driver")
Cc: stable@dpdk.org
Reported-by: Sunil Pai G <sunil.pai.g@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Sunil Pai G <sunil.pai.g@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
The current selftest function uses a single global variable to track state
which implies that only a single instance can have the selftest function
called on it. Change this to an array to allow multiple instances to be
tested.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Currently with the rawdev API there is no way to check that the structure
passed in via the dev_private pointer in the structure passed to configure
API is of the correct type - it's just checked that it is non-NULL. Adding
in the length of the expected structure provides a measure of typechecking,
and can also be used for ABI compatibility in future, since ABI changes
involving structs almost always involve a change in size.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Since we now allow some parameter checking inside the driver info_get()
functions, it makes sense to allow error return from those functions to the
caller. Therefore we change the driver callback return type from void to
int.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Currently with the rawdev API there is no way to check that the structure
passed in via the dev_private pointer in the dev_info structure is of the
correct type - it's just checked that it is non-NULL. Adding in the length
of the expected structure provides a measure of typechecking, and can also
be used for ABI compatibility in future, since ABI changes involving
structs almost always involve a change in size.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
A decision was made [1] to no longer support Make in DPDK, this patch
removes all Makefiles that do not make use of pkg-config, along with
the mk directory previously used by make.
[1] https://mails.dpdk.org/archives/dev/2020-April/162839.html
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Start a new release cycle with empty release notes.
The ABI version becomes 21.0.
The ABI major is back to normal, having only one number (21 vs 20.0).
The map files are updated to the new ABI major number (21).
The ABI exceptions are dropped.
Travis ABI check is disabled because compatibility is not preserved.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Introduce the RTE_LOG_REGISTER macro to avoid the code duplication
in the logtype registration process.
It is a wrapper macro for declaring the logtype, registering it and
setting its level in the constructor context.
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Add support for Ice Lake IOAT DMA engine PCI Device ID.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
There is a common macro __rte_packed for packing structs,
which is now used where appropriate for consistency.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Merge all versions in linker version script files to DPDK_20.0.
This commit was generated by running the following command:
:~/DPDK$ buildtools/update-abi.sh 20.0
Signed-off-by: Pawel Modrak <pawelx.modrak@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Since the library versioning for both stable and experimental ABI's is
now managed globally, the LIBABIVER and version variables no longer
serve any useful purpose, and can be removed.
The replacement in Makefiles was done using the following regex:
^(#.*\n)?LIBABIVER\s*:=\s*\d+\n(\s*\n)?
(LIBABIVER := numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
The replacement for meson files was done using the following regex:
^(#.*\n)?version\s*=\s*\d+\n(\s*\n)?
(version = numbers, optionally preceded by a comment and optionally
succeeded by an empty line)
[David]: those variables are manually removed for the files:
- drivers/common/qat/Makefile
- lib/librte_eal/meson.build
[David]: the LIBABIVER is restored for the external ethtool example
library.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
The rawdev xstats_reset function is now enabled. It is called when the
ioat autotest completes, to reset all xstat values after they have been
modified during testing.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
When running the ioat_rawdev_autotest, the ring size was initially required
to be 0. This prevented the test being run multiple times, as the ring size
is modified to a non-zero value when running the first test.
A global variable is now used to track the expected ring size, and is used
in the initial value check. This enables the test to be run more than
once.
Fixes: 849470d522ed ("raw/ioat: add configure, start and stop functions")
Cc: stable@dpdk.org
Reported-by: Yinan Wang <yinan.wang@intel.com>
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The ioat rawdev was initially started during the test, however was never
stopped. This would cause issues when running the test again, as the device
cannot be configured when in a running state.
The device is now stopped after the test has completed, or if the test is
terminated due to error.
Fixes: 849470d522ed ("raw/ioat: add configure, start and stop functions")
Cc: stable@dpdk.org
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Some drivers were missing reasons text for their disabling in meson.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Luca Boccassi <bluca@debian.org>
The driver names for rawdevs were both different in make and meson builds
and were non-standard in the make version in that some included "rawdev" in
the name while others didn't.
Therefore, for global consistency of naming, we can use "rte_rawdev" rather
than "rte_pmd" for the prefix for the libraries. While most other driver
categories use "rte_pmd" as a prefix, there is precedent for this in the
mempool drivers use "rte_mempool" as a prefix.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Some builds with clang report an error because '<>' rather than '""' were
used for including the ioat spec header file.
Target: x86_64-native-bsdapp-clang
error: 'rte_ioat_spec.h' file not found with <angled> include; use "quotes" instead
#include <rte_ioat_spec.h>
^~~~~~~~~~~~~~~~~
"rte_ioat_spec.h"
1 error generated.
Since this file should always be in the same directory as the main header,
we can safely change the include line to fix this error.
Fixes: abff4333ec20 ("raw/ioat: create device on probe and destroy on release")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The incriminated commit broke the use of RTE_PCI_DRV_IOVA_AS_VA which
was intended to mean "driver only supports VA" but had been understood
as "driver supports both PA and VA" by most net drivers and used to let
dpdk processes to run as non root (which do not have access to physical
addresses on recent kernels).
The check on physical addresses actually closed the gap for those
drivers. We don't need to mark them with RTE_PCI_DRV_IOVA_AS_VA and this
flag can retain its intended meaning.
Document explicitly its meaning.
We can check that a driver requirement wrt to IOVA mode is fulfilled
before trying to probe a device.
Finally, document the heuristic used to select the IOVA mode and hope
that we won't break it again.
Fixes: 703458e19c16 ("bus/pci: consider only usable devices for IOVA mode")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jerin Jacob <jerinj@marvell.com>
Tested-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
When using _mm_set_epi64() rather than _mm_set_epi64x() intrinsic, ICC
tries to use the x87 floating point registers, leading to warnings about
not properly clearing value when switching between x87 and SSE/AVX modes.
error #13203: No EMMS instruction before call to function
Fix this by using the set64x() intrinsic.
Fixes: 0a92e63fc4cd ("raw/ioat: add local API to perform copies")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Tested-by: Anatoly Burakov <anatoly.burakov@intel.com>
