For files that already have rte_string_fns.h included in them, we can
do a straight replacement of snprintf(..."%s",...) with strlcpy. The
changes in this patch were auto-generated via command:
spatch --sp-file devtools/cocci/strlcpy-with-header.cocci --dir . --in-place
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The existing cocci script for coccinelle replaces all matching instances
of snprintf() with strlcpy() without regards to header inclusion. To allow
changes without build errors, we create a safer version of this script
that only makes changes when the rte_string_fns.h header is already
included.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The original coccinelle script worked by replacing instances of
snprintf(.."%s",...) with strlcpy(), but only where the source and dest
parameters were plain identifiers. Allowing expressions for those params
opens up a wide range of other possible changes.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Using the size of the source string is incorrect when printing using
snprintf. Instead pass in the buffer size to be used appropriately.
Fixes: 457ecf2953 ("bond: add debug info for mode 6")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
When creating files on disk, e.g. for EAL configuration or shared memory
locks, etc., there is no need to grant any permissions on those files to
other users. All directories are already created with 0700 permissions, so
we should create all files with 0600 permissions.
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
'/**<' style comments apply to the previous member, which caused doxygen to
emit the RTE_RING_NAMESIZE documentation for RTE_RING_MZ_PREFIX.
Fixes: 38c9817ee1 ("mempool: adjust name size in related data types")
Cc: stable@dpdk.org
Signed-off-by: Gage Eads <gage.eads@intel.com>
This commit adds support for lock-free (linked list based) stack mempool
handler.
In mempool_perf_autotest the lock-based stack outperforms the
lock-free handler for certain lcore/alloc count/free count
combinations*, however:
- For applications with preemptible pthreads, a standard (lock-based)
stack's worst-case performance (i.e. one thread being preempted while
holding the spinlock) is much worse than the lock-free stack's.
- Using per-thread mempool caches will largely mitigate the performance
difference.
*Test setup: x86_64 build with default config, dual-socket Xeon E5-2699 v4,
running on isolcpus cores with a tickless scheduler. The lock-based stack's
rate_persec was 0.6x-3.5x the lock-free stack's.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit adds lock-free stack variants of stack_autotest
(stack_lf_autotest) and stack_perf_autotest (stack_lf_perf_autotest), which
differ only in that the lock-free versions pass the RTE_STACK_F_LF flag to
all rte_stack_create() calls.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit adds an implementation of the lock-free stack push, pop, and
length functions that use __atomic builtins, for systems that benefit from
the finer-grained memory ordering control.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
This commit adds support for a lock-free (linked list based) stack to the
stack API. This behavior is selected through a new rte_stack_create() flag,
RTE_STACK_F_LF.
The stack consists of a linked list of elements, each containing a data
pointer and a next pointer, and an atomic stack depth counter.
The lock-free push operation enqueues a linked list of pointers by pointing
the tail of the list to the current stack head, and using a CAS to swing
the stack head pointer to the head of the list. The operation retries if it
is unsuccessful (i.e. the list changed between reading the head and
modifying it), else it adjusts the stack length and returns.
The lock-free pop operation first reserves num elements by adjusting the
stack length, to ensure the dequeue operation will succeed without
blocking. It then dequeues pointers by walking the list -- starting from
the head -- then swinging the head pointer (using a CAS as well). While
walking the list, the data pointers are recorded in an object table.
This algorithm stack uses a 128-bit compare-and-swap instruction, which
atomically updates the stack top pointer and a modification counter, to
protect against the ABA problem.
The linked list elements themselves are maintained in a lock-free LIFO
list, and are allocated before stack pushes and freed after stack pops.
Since the stack has a fixed maximum depth, these elements do not need to be
dynamically created.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
stack_perf_autotest tests the following with one lcore:
- Cycles to attempt to pop an empty stack
- Cycles to push then pop a single object
- Cycles to push then pop a burst of 32 objects
It also tests the cycles to push then pop a burst of 8 and 32 objects with
the following lcore combinations (if possible):
- Two hyperthreads
- Two physical cores
- Two physical cores on separate NUMA nodes
- All available lcores
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
stack_autotest performs positive and negative testing of the stack API, and
exercises the push and pop datapath functions with all available lcores.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
The new rte_stack library is derived from the mempool handler, so this
commit removes duplicated code and simplifies the handler by migrating it
to this new API.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
The rte_stack library provides an API for configuration and use of a
bounded stack of pointers. Push and pop operations are MT-safe, allowing
concurrent access, and the interface supports pushing and popping multiple
pointers at a time.
The library's interface is modeled after another DPDK data structure,
rte_ring, and its lock-based implementation is derived from the stack
mempool handler. An upcoming commit will migrate the stack mempool handler
to rte_stack.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
When enabling pedantic compilation with CONFIG_RTE_LIBRTE_MLX5_DEBUG,
the compiler complains about non standard 128-bit integer type:
include/rte_atomic_64.h:223:3: error:
ISO C does not support ‘__int128’ types [-Werror=pedantic]
It must be marked as an extension of the standard C language
to be accepted in pedantic compilation.
Fixes: 640c5f09ef ("eal/x86: add 128-bit atomic compare exchange")
Cc: gage.eads@intel.com
Reported-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Gage Eads <gage.eads@intel.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
This patch updates the ipsec library programmer's guide with
the additional algorithms which are now supported.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
This patch adds the deprecation notice of changing Cryptodev
symmetric xform structure. The proposed change is to making
key pointers in the crypto xforms (cipher, auth, aead) to
indicate neither the library or the drivers will not change
the content of the key buffer.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Add librte_ipsec into 'Shared Library Versions' list in the release notes.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
For sym_crypto_op prepare move common code into a separate function(s).
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Change the order of operations for esp inbound post-process:
- read mbuf metadata and esp tail first for all packets in the burst
first to minimize stalls due to load latency.
- move code that is common for both transport and tunnel modes into
separate functions to reduce code duplication.
- add extra check for packet consitency
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Right now check for packet length and padding is done inside cop_prepare().
It makes sense to have all necessary checks in one place at early stage:
inside pkt_prepare().
That allows to simplify (and later hopefully) optimize cop_prepare() part.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
sa.c becomes too big, so decided to split it into 3 chunks:
- sa.c - control path related functions (init/fini, etc.)
- esp_inb.c - ESP inbound packet processing
- esp_outb.c - ESP outbound packet processing
Plus few changes in internal function names to follow the same
code convention.
No functional changes introduced.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
As was pointed in one of previous reviews - we can avoid updating
contents of mbuf array for successfully processed packets.
Instead store indexes of failed packets, to move them beyond the good
ones later.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Right now we first fill crypto_sym_op part of crypto_op,
then in a separate cycle we fill crypto op fields.
It makes more sense to fill whole crypto-op in one go instead.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Operations to set/update bit-fields often cause compilers
to generate suboptimal code. To avoid such negative effect,
use tx_offload raw value and mask to update l2_len and l3_len
fields within mbufs.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Operations to set/update bit-fields often cause compilers
to generate suboptimal code.
To help avoid such situation for tx_offload fields:
introduce new enum for tx_offload bit-fields lengths and offsets,
and new function to generate raw tx_offload value.
Add new test-case into UT for introduced function.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Currently, compress-perf doesn't respect incompressible
data inside one operation.
This patch adds such a functionality. Now the output buffer
in one operation is big enough to store such a data after
compression. Also added segment size checking to pass
values in right range.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch adds query about CPU features
Fixes: 53a9baa98c ("compress/isal: add basic PMD ops")
Cc: stable@dpdk.org
Signed-off-by: Tomasz Cel <tomaszx.cel@intel.com>
Acked-by: Lee Daly <lee.daly@intel.com>
This patch assigns QAT queue pair resources to the correct NUMA nodes.
Any DMA'able memory should use NUMA node of QAT device
rather than socket_id of the initializing process.
Fixes: 98c4a35c73 ("crypto/qat: move common qat files to common dir")
Fixes: a795248d74 ("compress/qat: add configure and clear functions")
Cc: stable@dpdk.org
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch will change the operation status when ISA-L returns because
of a recoverable out of space error, rather than a just generic fail.
Signed-off-by: Lee Daly <lee.daly@intel.com>
Tested-by: Tomasz Cel <tomaszx.cel@intel.com>
crypto pmds are queried to check if Sign and Decrypt
with CRT keys or exponent is supported, thus call
operation with relevant key type.
Signed-off-by: Ayuj Verma <ayverma@marvell.com>
Signed-off-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
openssl PMD support RSA private key operation
using both qt and exp key type.
Set rsa key type feature flag
Signed-off-by: Ayuj Verma <ayverma@marvell.com>
Signed-off-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Add feature flag to reflect RSA private key
operation support using quintuple (crt) or
exponent type key. if PMD support both,
then it should set both.
App should query cryptodev feature flag to check
if Sign and Decryt with CRT keys or exponent is
supported, thus call operation with relevant
key type.
Signed-off-by: Ayuj Verma <ayverma@marvell.com>
Signed-off-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
- fle pool allocations should be done for each process.
- cryptodev->data is shared across muliple processes but
cryptodev itself is allocated for each process. So any
information which needs to be shared between processes,
should be kept in cryptodev->data.
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
session inq and qp are assigned for each core from which the
packets arrive. This was not correctly handled while supporting
multiple sessions per queue pair.
This patch fixes the attach and detach of queues for each core.
Fixes: e79416d10f ("crypto/dpaa_sec: support multiple sessions per queue pair")
Cc: stable@dpdk.org
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
SEC HW descriptor sharing mode can now be controlled
during Session preparation by the respective drivers
shared descriptors in case of non-protocol offload does not need
any sync between the subsequent jobs. Thus, changing it to
SHR_NEVER from SHR_SERIAL for cipher_only, auth_only, and gcm.
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
In case of gcm, output buffer should have aad space
before the actual buffer which needs to be written.
CAAM will not write into the aad anything, it will skip
auth_only_len (aad) and write the buffer afterwards.
Fixes: 37f96eb01b ("crypto/dpaa2_sec: support scatter gather")
Cc: stable@dpdk.org
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
private data should be cleared instead of the complete session
Fixes: 8d1f3a5d75 ("crypto/dpaa2_sec: support crypto operation")
Cc: stable@dpdk.org
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch adds dynamic SGL allocation instead of static one.
The number of element in SGL can be adjusted in each operation
depend of the request.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
As stated in 19.02 deprecation notice, this patch updates the
aesni_mb PMD to remove the support of older Intel-ipsec-mb
library version earlier than 0.52.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This patch updates the unit test to enable AESNI-MB PMD
out-of-place tests. A special test type that swap both
the source and destination buffer is added for a more
comprehensive test set to take place.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Add out-of-place processing, i.e. different source and
destination m_bufs, plus related capability update, tests
and documentation.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This patch adds new test structure for modexp
and modinv for asymmetric cryptography.
Signed-off-by: Damian Nowak <damianx.nowak@intel.com>
Acked-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This commit adds modular multiplicative inverse to Intel
QuickAssist Technology driver. For capabilities or limitations
please refer to qat.rst or qat_asym_capabilities.h.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This commit adds modular exponentiation to Intel QuickAssist
Technology driver. For capabilities or limitations please refer to
qat.rst or qat_asym_capabilities.h.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch adds Poll Mode Driver for asymmetric crypto
functions of Intel QuickAssist Technology hardware.
It contains plain driver with no functions implemented, specific
algorithms will be introduced in separate patches.
This patch depends on a QAT PF driver for device initialization. See
the file docs/guides/cryptodevs/qat.rst for configuration details.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This commit adds headers to be used in conjunction with asymmetric
cryptography operations using Intel QuickAssist Technology driver
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Add test case for encryption, decryption for snow3g when digest
is encrypted
Signed-off-by: Lukasz Krakowiak <lukaszx.krakowiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
