Add an extra parameter to the ring dequeue burst/bulk functions so that
those functions can optionally return the amount of remaining objs in the
ring. This information can be used by applications in a number of ways,
for instance, with single-consumer queues, it provides a max
dequeue size which is guaranteed to work.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Add an extra parameter to the ring enqueue burst/bulk functions so that
those functions can optionally return the amount of free space in the
ring. This information can be used by applications in a number of ways,
for instance, with single-producer queues, it provides a max
enqueue size which is guaranteed to work. It can also be used to
implement watermark functionality in apps, replacing the older
functionality with a more flexible version, which enables apps to
implement multiple watermark thresholds, rather than just one.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
When a C file for a library/driver is including the public header files for
that library, those need to be included as local includes using quotes
rather than angle-brackets. Without doing so, parallel builds can fail, as
the compiler will only look for those headers in the global include folder
rather than locally, and the build system does not enforce that the headers
for a lib are installed before the rest of the lib is compiled.
Fixes: 097ab0bac0 ("crypto/scheduler: add API")
Fixes: 503e9c5afb ("crypto/scheduler: register as vdev driver")
Fixes: 31439ee72b ("crypto/scheduler: add API implementations")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Rather than reading the size directly from the ring structure, use the
dedicated ring function for that purpose.
Previous commits to do this only did so for the null crypto driver which
was the only one compiled in by default, but all other drivers need to be
similarly updated.
Fixes: b11c78a2e0 ("crypto/null: use ring size function")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Before this patch, the management of dependencies between directories
had several issues:
- the generation of .depdirs, done at configuration is slow: it can take
more than one minute on some slow targets (usually ~10s on a standard
PC without -j).
- for instance, it is possible to express a dependency like:
- app/foo depends on lib/librte_foo
- and lib/librte_foo depends on app/bar
But this won't work because the directories are traversed with a
depth-first algorithm, so we have to choose between doing 'app' before
or after 'lib'.
- the script depdirs-rule.sh is too complex.
- we cannot use "make -d" for debug, because the output of make is used for
the generation of .depdirs.
This patch moves the DEPDIRS-* variables in the upper Makefile, making
the dependencies much easier to calculate. A DEPDIRS variable is still
used to process library dependencies in LDLIBS.
After this commit, "make config" is almost immediate.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Tested-by: Robin Jarry <robin.jarry@6wind.com>
Tested-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Rather than reading the size directly from the ring structure,
use the dedicated function for that purpose.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Fixes the missed session backup during enqueue.
Fixes: 100e4f7e44 ("crypto/scheduler: add round-robin mode")
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Fixes the wrong slave initialization issue on start-up
Fixes: 100e4f7e44 ("crypto/scheduler: add round-robin mode")
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When ciphering and authenticating in the same operation
(cipher-then-auth or auth-then-cipher),
the cipher key and authentication key were set with the same
key, in SNOW3G, KASUMI and ZUC PMDs.
They were using the key of the first transform structure,
instead of using the keys of the two different transform
structures.
This is not a big issue, since usually, the same key is
used for ciphering and authentication, but keys may be different.
Fixes: 3aafc423cf ("snow3g: add driver for SNOW 3G library")
Fixes: 2773c86d06 ("crypto/kasumi: add driver for KASUMI library")
Fixes: cf7685d68f ("crypto/zuc: add driver for ZUC library")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
AESNI MB now supports authentication only operations,
but the array containing all the possible digest lengths
was not updated to reflect the new algorithm NULL_HASH,
causing an "Out-of-bounds access".
Coverity issue: 140977
Fixes: 8772c3f713 ("crypto/aesni_mb: add single operation functionality")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Adds Makefile for scheduler cryptodev PMD, and updates existing
Makefiles. Different than other cryptodev PMDs, scheduler PMD
is required to be built as shared libraries.
Adds scheduler PMD enable and debug flags to config/common_base.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Implements all standard operations required for cryptodev,
and register them to cryptodev operation function pointer table.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Adds crypto scheduler's PMD's probe and remove function and the device's
enqueue and dequeue burst functions. A cryptodev scheduler PMD is
then registered in the end.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Implements round-robin scheduling mode and register into cryptodev
scheduler ops structure. This mode enqueues a burst of operation
to one of its slaves, and iterates the next burst to the other
slave. Same procedure is done on dequeueing operations.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Adds the implementations of the APIs for scheduler cryptodev PMD.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Adds a number of internal structures for the cryptodev scheduler PMD. The
structures include the scheduler context, slave, queue pair context,
and session.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Adds APIs and function prototypes for the scheduler PMD to perform extra
operations other than standard cryptodev APIs.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This makes struct rte_cryptodev independent of struct rte_pci_device by
replacing it with a pointer to the generic struct rte_device.
This is inline with the recent changes in ethdev
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: John Griffin <john.griffin@intel.com>
Reviewed-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Add two new feature flags:
* RTE_CRYPTODEV_FF_CPU_NEON
represents ARM NEON (TM) instructions
* RTE_CRYPTODEV_FF_CPU_ARM_CE
represents ARM crypto extensions
Add them to both cryptodev library, documentation and relevant
PMD driver for ARMv8.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
This patch introduces crypto poll mode driver
using ARMv8 cryptographic extensions.
CPU compatibility with this driver is detected in
run-time and virtual crypto device will not be
created if CPU doesn't provide:
AES, SHA1, SHA2 and NEON.
This PMD is optimized to provide performance boost
for chained crypto operations processing,
such as encryption + HMAC generation,
decryption + HMAC validation. In particular,
cipher only or hash only operations are
not provided.
The driver currently supports AES-128-CBC
in combination with: SHA256 HMAC and SHA1 HMAC
and relies on the external armv8_crypto library:
https://github.com/caviumnetworks/armv8_crypto
Build ARMv8 crypto PMD if compiling for ARM64
and CONFIG_RTE_LIBRTE_PMD_ARMV8_CRYPTO option
is enable in the configuration file.
ARMV8_CRYPTO_LIB_PATH environment variable will
point to the appropriate library directory.
Signed-off-by: Zbigniew Bodek <zbigniew.bodek@caviumnetworks.com>
Reviewed-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Current Cryptodev AES-NI GCM PMD is implemented using Multi Buffer
Crypto library.This patch reimplement the device using ISA-L Crypto
library: https://github.com/01org/isa-l_crypto.
The migration entailed the following additional support for:
* GMAC algorithm.
* 256-bit cipher key.
* Session-less mode.
* Out-of place processing
* Scatter-gatter support for chained mbufs (only out-of place and
destination mbuf must be contiguous)
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
This patch adds a user defined name initializing parameter to cryptodev
library.
Originally, for software cryptodev PMD, the vdev name parameter is
treated as the driver identifier, and will create an unique name for each
device automatically, which is not necessarily as same as the vdev
parameter.
This patch allows the user to either create a unique name for his software
cryptodev, or by default, let the system creates a unique one. This should
help the user managing the created cryptodevs easily.
Examples:
CLI command fragment 1: --vdev "crypto_aesni_gcm_pmd"
The above command will result in creating a AESNI-GCM PMD with name of
"crypto_aesni_gcm_X", where postfix X is the number assigned by the system,
starting from 0. This fragment can be placed in the same CLI command
multiple times, resulting the postfixs incremented by one for each new
device.
CLI command fragment 2: --vdev "crypto_aesni_gcm_pmd,name=gcm1"
The above command will result in creating a AESNI-GCM PMD with name of
"gcm1". This fragment can be placed in the same CLI command multiple
times, as long as each having a unique name value.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
This patch introduces RTE_CRYPTODEV_FF_MBUF_SCATTER_GATHER feature flag
informing that selected crypto device supports segmented mbufs natively
and doesn't need to be coalesced before crypto operation.
While using segmented buffers in crypto devices may have unpredictable
results, for PMDs which doesn't support it natively, additional check is
made for debug compilation.
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
EVP_CIPHER_CTX_set_padding() function always returns 1, so the check is
unneeded.
Fixes: d61f70b4c9 ("crypto/libcrypto: add driver for OpenSSL library")
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Tested-by: Zhaoyan Chen <zhaoyan.chen@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
This patch sets iv size in qat PMD to 12 bytes to be
conformant with nist SP800-38D.
Fixes: 26c2e4ad5a ("cryptodev: add capabilities discovery")
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch sets iv size in aesni gcm PMD to 12 bytes to be
conformant with nist SP800-38D.
Fixes: eec136f3c5 ("aesni_gcm: add driver for AES-GCM crypto operations")
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
This commit fixes pre-counter block (J0) padding by clearing
four most significant bytes before setting initial counter value.
Fixes: b2bb359747 ("crypto/aesni_gcm: move pre-counter block to driver")
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Release v0.44 of Intel(R) Multi-Buffer Crypto for IPsec library adds
support for AVX512 instructions. This patch enables the new AVX512
accelerated functions from the aesni_mb_pmd crypto poll mode driver.
This patch set requires that the aesni_mb_pmd is linked against the
version 0.44 or greater of the Multi-Buffer Crypto for IPsec library.
Signed-off-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Update driver to use new AESNI Multibuffer IPSec library single
operation functionality (cipher only and authentication only).
This patch also adds tests for this new feature.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
When using sessionless crypto operations, crypto session
is obtained from a pool of sessions, when processing the
operation. Once the operation is processed, the session
is put back in the pool, but for the AESNI MB PMD, this
session was not being saved in the operation and therefore,
it did not return to the session pool.
Fixes: 924e84f873 ("aesni_mb: add driver for multi buffer based crypto")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Extra bytes are being written at end of data while process standard
openssl cipher encryption. This behaviour is unexpected.
This patch disable the padding feature in openssl library, which is
causing the problem.
Fixes: d61f70b4c9 ("crypto/libcrypto: add driver for OpenSSL library")
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
In out-of-place operation, data is DMAed from source mbuf
to destination mbuf. To avoid header data in dest mbuf being
overwritten, the minimal data-set should be DMAed.
Fixes: 39e0bee48e ("crypto/qat: rework request builder for performance")
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>
The cryptodev API had specified that if the digest address field was
left empty on an authentication operation, then the PMD would assume
the digest was appended to the source or destination data.
This case was not handled at all by most PMDs and incorrectly handled
by the QAT PMD.
As no bugs were raised, it is assumed to be not needed, so this patch
removes it, rather than add handling for the case on all PMDs.
The digest can still be appended to the data, but its
address must now be provided in the op.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: John Griffin <john.griffin@intel.com>
Replace the raw I/O device memory read/write access with eal
abstraction for I/O device memory read/write access to fix portability
issues across different architectures.
CC: John Griffin <john.griffin@intel.com>
CC: Fiona Trahe <fiona.trahe@intel.com>
CC: Deepak Kumar Jain <deepak.k.jain@intel.com>
Signed-off-by: Santosh Shukla <santosh.shukla@caviumnetworks.com>
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
QAT PMD adjusts the buffer start address and offsets passed
to the device so that the DMAs in and out of the device are
64-byte aligned.
This gives more consistent throughput, which had been
variable depending on how the application set up the mbuf.
The message builder code had to be considerably re-factored
to do this efficiently.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
This patch replaces name "libcrypto" to "openssl" from file directories,
symbol prefixes and sub-names connected with old name.
Renamed poll mode driver files, test files, and documentations.
It is done to better name association with library because
the cryptography operations are using Openssl library crypto API.
Fixes: d61f70b4c9 ("crypto/libcrypto: add driver for OpenSSL library")
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
This registers the legacy names of the driver being renamed in
commit 2f45703c17 ("drivers: make driver names consistent").
Signed-off-by: Jan Blunck <jblunck@infradead.org>
Tested-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
All init/uninit function names in the virtual devices have been renamed,
so they finish with probe or remove, so to keep consistency,
same thing should be done in this PMD.
Fixes: d61f70b4c9 ("crypto/libcrypto: add driver for OpenSSL library")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
All init/uninit function names in the virtual devices have been renamed,
so they finish with probe or remove, so to keep consistency,
same thing should be done in this PMD.
Fixes: cf7685d68f ("crypto/zuc: add driver for ZUC library")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
All macros related to driver registeration renamed from DRIVER_*
to RTE_PMD_*
This includes:
DRIVER_REGISTER_PCI -> RTE_PMD_REGISTER_PCI
DRIVER_REGISTER_PCI_TABLE -> RTE_PMD_REGISTER_PCI_TABLE
DRIVER_REGISTER_VDEV -> RTE_PMD_REGISTER_VDEV
DRIVER_REGISTER_PARAM_STRING -> RTE_PMD_REGISTER_PARAM_STRING
DRIVER_EXPORT_* -> RTE_PMD_EXPORT_*
Fix PMDINFOGEN tool to look for matches of RTE_PMD_REGISTER_*.
Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Free ring memory on queue_pair release, else
releasing and setting up queue-pair of a different size fails.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
This code provides the initial implementation of the libcrypto
poll mode driver. All cryptography operations are using Openssl
library crypto API. Each algorithm uses EVP_ interface from
openssl API - which is recommended by Openssl maintainers.
This patch adds libcrypto poll mode driver support to librte_cryptodev
library.
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Michal Kobylinski <michalx.kobylinski@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Added new SW PMD which makes use of the libsso SW library,
which provides wireless algorithms ZUC EEA3 and EIA3
in software.
This PMD supports cipher-only, hash-only and chained operations
("cipher then hash" and "hash then cipher") of the following
algorithms:
- RTE_CRYPTO_SYM_CIPHER_ZUC_EEA3
- RTE_CRYPTO_SYM_AUTH_ZUC_EIA3
The ZUC hash and cipher algorithms, which are enabled
by this crypto PMD are implemented by Intel's libsso software
library.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
This commit fixes a compilation error on icc,
due to unallowed conversion from int to enum:
drivers/crypto/snow3g/rte_snow3g_pmd.c(155):
error #188: enumerated type mixed with another type
sess->op = mode;
^
drivers/crypto/kasumi/rte_kasumi_pmd.c(155):
error #188: enumerated type mixed with another type
sess->op = mode;
^
Fixes: 3aafc423cf ("snow3g: add driver for SNOW 3G library")
Fixes: 2773c86d06 ("crypto/kasumi: add driver for KASUMI library")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
This patch moves computing of pre-counter block into the AESNI-GCM
driver so it can be moved from test files.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
