According to the API, AES-CCM has special requirements
when setting IV and AAD fields.
The L2fwd-crypto app is updated to set the nonce (IV)
and AAD in the right positions in these two fields
(1 byte after start of IV field and 18 bytes after start
of AAD).
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
In order to improve memory utilization, a single mempool
is created, containing the crypto operation and mbufs
(one if operation is in-place, two if out-of-place).
This way, a single object is allocated and freed
per operation, reducing the amount of memory in cache,
which improves scalability.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Add support for multiple queue pairs, when there are
more logical cores available than crypto devices enabled.
For instance, if there are 4 cores available and
2 crypto devices, each device will have two queue pairs.
This is useful to have multiple logical cores using
a single crypto device, without needing to initialize
a crypto device per core.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
For throughput and latency tests, it is not required
to populate the mbufs with any test vector.
For verify test, there is already a function that rewrites
the mbufs every time they are going to be used with
crypto operations.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
When running the verify test, mbufs in the pool were
populated with the test vector loaded from a file.
To avoid limiting the number of operations to the pool size,
mbufs will be rewritten with the test vector, before
linking them to the crypto operations.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Instead of parsing number of segments, from the command line,
parse segment size, as it is a more usual case to have
the segment size fixed and then different packet sizes
will require different number of segments.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Since DPDK 17.08, there is specific parameters
for AEAD algorithm, like AES-GCM. When verifying
crypto operations with test vectors, the parser
was not reading AEAD data (such as IV or key).
Fixes: 8a5b494a7f ("app/test-crypto-perf: add AEAD parameters")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Instead of prepending the AAD (Additional Authenticated Data)
in the mbuf, it is easier to set after the crypto operation,
as it is a read-only value, like the IV, and then it is not
restricted to the size of the mbuf headroom.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Currently, there is some duplication in all the test types,
in the crypto performance application.
In order to improve maintainability of this code,
and ease future work on it, common functions have been separated
in a different file that gets included in all the tests.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
When using DES-CBC, packet size has to be multiple
of 8 bytes, but if a list of packets is provided.
the check was not correct.
Fixes: fc4600fb25 ("app/crypto-perf: add extra option checks")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
This patch adds a new benchmarking mode, which is intended for
microbenchmarking individual parts of the cryptodev framework,
specifically crypto ops alloc-build-free, cryptodev PMD enqueue
and cryptodev PMD dequeue.
It works by first benchmarking crypto operation alloc-build-free
loop (no enqueues/dequeues happening), and then benchmarking
enqueue and dequeue separately, by first completely filling up the
TX queue, and then completely draining the RX queue.
Results are shown as cycle counts per alloc/build/free, PMD enqueue
and PMD dequeue.
One new test mode is added: "pmd-cyclecount"
(called with --ptest=pmd-cyclecount)
New command-line argument is also added:
--pmd-cyclecount-delay-ms: this is a pmd-cyclecount-specific parameter
that controls the delay between enqueue and dequeue. This is
useful for benchmarking hardware acceleration, as hardware may
not be able to keep up with enqueued packets. This parameter
can be increased if there are large amounts of dequeue
retries.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This parameter makes number of cryptodev descriptors adjustable
and defaults to earlier hardcoded default of 2048.
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The compilation with gcc-6.3.0 and EXTRA_CFLAGS=-Og gives the following
error:
CC cperf_test_verify.o
cperf_test_verify.c: In function ‘cperf_verify_op’:
cperf_test_verify.c:382:5: error: ‘auth’ may be used uninitialized
in this function
[-Werror=maybe-uninitialized]
if (auth == 1) {
^
cperf_test_verify.c:371:5: error: ‘cipher’ may be used uninitialized
in this function
[-Werror=maybe-uninitialized]
if (cipher == 1) {
^
cperf_test_verify.c:384:11: error: ‘auth_offset’ may be used
uninitialized in this function
[-Werror=maybe-uninitialized]
res += memcmp(data + auth_offset,
^~~~~~~~~~~~~~~~~~~~~~~~~~
vector->digest.data,
~~~~~~~~~~~~~~~~~~~~
options->digest_sz);
~~~~~~~~~~~~~~~~~~~
cperf_test_verify.c:377:11: error: ‘cipher_offset’ may be used
uninitialized in this function
[-Werror=maybe-uninitialized]
res += memcmp(data + cipher_offset,
^~~~~~~~~~~~~~~~~~~~~~~~~~~~
vector->plaintext.data,
~~~~~~~~~~~~~~~~~~~~~~~
options->test_buffer_size);
~~~~~~~~~~~~~~~~~~~~~~~~~~
There is no default case in the switch statement, so if options->op_type
is an unknown value, the function will use uninitialized values. Fix it
by adding a default.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
data is allocated but never freed.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
In crypto latency test, when some crypto operations cannot be
enqueued, they are returned to its mempool.
However, instead of freeing the operations in the array passed
to the enqueue function, the app was freeing the operations
in the array passed to the dequeue function, leading to incorrect
results.
Fixes: 15b55dd751 ("app/crypto-perf: return crypto ops to mempool in bulk")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Memory is reserved after each crypto operation
for the necessary IV(s), which could be for cipher,
authentication or AEAD algorithms.
However, for AEAD algorithms (such as AES-GCM), this
memory was not being reserved, leading to potential
memory overflow.
Fixes: 8a5b494a7f ("app/test-crypto-perf: add AEAD parameters")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When using CSV, the output of the throughput performance numbers
was not correct, representing decimal numbers incorrectly.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
If the crypto operation pool size is too small,
the pool might run out of operations, if all the crypto operations
are still being used and have not been freed.
To inform the user about this, the application should display
an error message, asking the user to increase the pool size
through the app parameters.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Call of rte_cryptodev_stop() function from test destructors
is added.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Kirill Rybalchenko <kirill.rybalchenko@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Additional Authenticated Data (AAD) is called "aad" in most
places of cryptodev, but it was called "add_auth_data"
in the AEAD transform transform (aead_xform).
This field is renamed to "aad_length" in order to keep
consistency.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Additional Authenticated Data (AAD) was removed from the
authentication parameters, but still the supported size
was part of the authentication capabilities of a PMD.
Fixes: 4428eda8bb ("cryptodev: remove AAD from authentication structure")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
After test is finished the main thread waits only for lcores
where runners were started because, in case of using the
multicore scheduler, more cores are launched that do not need
to be waited for.
Signed-off-by: Kirill Rybalchenko <kirill.rybalchenko@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The session mempool pointer is needed in each queue pair,
if session-less operations are being handled.
Therefore, the API is changed to accept this parameter,
as the session mempool is created outside the
device configuration function, similar to what ethdev
does with the rx queues.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Change crypto device's session management to make it
device independent and simplify architecture when session
is intended to be used on more than one device.
Sessions private data is agnostic to underlying device
by adding an indirection in the sessions private data
using the crypto driver identifier.
A single session can contain indirections to multiple device types.
New function rte_cryptodev_sym_session_init has been created,
to initialize the driver private session data per driver to be
used on a same session, and rte_cryptodev_sym_session_clear
to clear this data before calling rte_cryptodev_sym_session_free.
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Instead of creating the session mempool while configuring
the crypto device, apps will create the mempool themselves.
This way, it gives flexibility to the user to have a single
mempool for all devices (as long as the objects are big
enough to contain the biggest private session size) or
separate mempools for different drivers.
Also, since the mempool is now created outside the
device configuration function, now it needs to be passed
through this function, which will be eventually passed
when setting up the queue pairs, as ethernet devices do.
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Now that AAD is only used in AEAD algorithms,
there is no need to keep AAD in the authentication
structure.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Since there is a new operation type (AEAD), add parameters
for this in the application.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
AES-GMAC is an authentication algorithm, based on AES-GCM
without encryption. To simplify its usage, now it can be used
setting the authentication parameters, without requiring
to concatenate a ciphering transform.
Therefore, it is not required to set AAD, but authentication
data length and offset, giving the user the option
to have Scatter-Gather List in the input buffer,
as long as the driver supports it.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Digest length was duplicated in the authentication transform
and the crypto operation structures.
Since digest length is not expected to change in a same
session, it is removed from the crypto operation.
Also, the length has been shrunk to 16 bits,
which should be sufficient for any digest.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Additional authenticated data (AAD) information was duplicated
in the authentication transform and in the crypto
operation structures.
Since AAD length is not meant to be changed in a same session,
it is removed from the crypto operation structure.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
For wireless algorithms (SNOW3G, KASUMI, ZUC),
the IV for the authentication algorithms (F9, UIA2 and EIA3)
was taken from the AAD parameter, as there was no IV parameter
in the authentication structure.
Now that IV is available for all algorithms, there is need
to keep doing this, so AAD is not used for these algorithms
anymore.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Authentication algorithms, such as AES-GMAC or the wireless
algorithms (like SNOW3G) use IV, like cipher algorithms.
So far, AES-GMAC has used the IV from the cipher structure,
and the wireless algorithms have used the AAD field,
which is not technically correct.
Therefore, authentication IV parameters have been added,
so API is more correct. Like cipher IV, auth IV is expected
to be copied after the crypto operation.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Since IV parameters (offset and length) should not
change for operations in the same session, these parameters
are moved to the crypto transform structure, so they will
be stored in the sessions.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Since IV now is copied after the crypto operation, in
its private size, IV can be passed only with offset
and length.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Usually, IV will change for each crypto operation.
Therefore, instead of pointing at the same location,
IV is copied after each crypto operation.
This will let the IV to be passed as an offset from
the beginning of the crypto operation, instead of
a pointer.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Storing a pointer to the user data is unnecessary,
since user can store additional data, after the crypto operation.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Declan Doherty <declan.doherty@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Use rte_mempool_put_bulk for both latency and throughput tests instead
of rte_crypto_op_free to improve application performance.
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
For AEAD algorithms, additional authenticated data (AAD)
can be passed, but it is optional, so its size can be zero.
Therefore, test can be run if no memory is allocated.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When SNOW3G/KASUMI/ZUC algorithms are used, ciphering
and authentication lengths have to be passed as bits
and not as bytes.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This patch fixes the crypto operation resubmission problem in crypto
perferformance test. Originally, when needed crypto ops amount is
smaller than the enqueued crypto ops in the last round, one or more
processed crypto operations will be re-enqueued.
Fixes: f8be1786b1 ("app/crypto-perf: introduce performance test application")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
In the latency test, when number of enqueued operations
is less than the burst size, the timestamp value of the
non-enqueued operations was being stored, even though
those operations were being freed.
This could cause an array overflow, since it could store
more values than the total number of operations.
Fixes: 5d75fb09d3 ("app/crypto-perf: fix invalid latency for QAT")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Reorganize options structure, to have all the parameters
used in a single run in the first bytes of the structure,
so all fit in a single cache line.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
When using the verify test, test name is necessary
to be passed when digest is needed.
Also, when using an block cipher algorithm (CBC, ECB),
the buffer size has to be aligned to the block size.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
So far, the crypto performance application was only able to
test one buffer size and one burst size.
With this commit, multiple sizes can be passed, either as a range
of values or as a list of values.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Instead of printing the test results in the destructor
function, print them just after each run.
This will be useful for after the commit that allows
a range of buffer and burst sizes.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
When testing cipher only operations, there is no need to append digest
at the end of the buffer.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
