For all trace on the data-path move to dynamic logging.
Add new QAT_DP_LOG macro to separate from non-dp trace.
Use the overall RTE_LOG_DP_LEVEL to override and compile out
debug data-path trace.
Add "unlikely" in two appropriate error checks.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
For all trace not on the data-path move to dynamic logging.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Removed INIT_FUNC trace and other unused macros.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Other services, apart from symmetric crypto,
such as compression, will be added in future patches.
Therefore, the assumption that only symmetric crypto
operations are processed will not be valid anymore,
and service type needs to be checked.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Optimize the dequeue function by inlining the response
processing function, assuming only symmetric
operations are supported.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Process response function is only implemented
for crypto symmetric operations, which do not require
some of the arguments.
Therefore, these arguments can be removed from the
function prototype.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
For every QAT PCI device probed, populate a local rte_device
containing an rte_driver. The rte_driver was created in a previous
patch to provide a crypto-specific driver name: "crypto_qat".
This was previously only used for driver registration, now it's also
used in device creation.
This allows applications to find devices driven by "crypto_qat".
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
QAT_64_BTYE_ALIGN_MASK macro is not specific to
symmetric crypto, but it is common for other
future services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Removed crypto reference in common debug message.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Renamed sgl_cookie to cookie and qp_gen_config
to qat_gen_config, as it is intended to hold
more than just queue pair data.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
As bundle_num is included in qat_gen1_qps static array
there shouldn't be a multiplier used in qat_qps_per_service()
Then removed ADF_NUM_BUNDLES_PER_DEV as no longer used.
Also renamed ADF_MAX_QPS_PER_BUNDLE to ADF_MAX_QPS_ON_ANY_SERVICE
and reduced from 4 to 2 which is enough for all current devices.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Add lock around accesses to the arbiter CSR
and use & instead of ^ as ^ not safe if
arb_disable called when already disabled.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Move all code into appropriate files, no actual code changes. Specifically:
- Rename rte_qat_cryptodev.c to qat_sym_pmd.c
- Create qat_sym_pmd.h and populate with fn prototypes for qat_sym_pmd.c
- Create qat_comp_pmd.c/.h and populate with placeholder functions
- Create qat_asym_pmd.c/.h and populate with placeholder functions
- Rename qat_crypto_capabilities.h to qat_sym_capabilities.h
- Move CRYPTODEV_NAME_QAT_SYM_PMD from qat_common.h to qat_sym_pmd.h
- Move qat_sym_dev_private from qat_device.h to qat_sym_pmd.h
- Move prototype for qat_sym_dev_info_get frm qat_device.h 2 qat_sym_pmd.h
- Move all qat_device.c sym dev_ops fns to qat_sym_pmd.c file
- Move all qat_sym.c dev_ops fns to qat_sym_pmd.c file
- Remove unused header file #includes from all files.
- Move pci_id_qat_map, probe/release/register from
rte_qat_cryptodev.c to qat_device.c
- Moved stray comment for bpi_cipher_ctx_init() from qat_sym.c
to qat_sym_session.c
- Changed all files to use SPDX license header
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Split qat_sym_stats_get/reset into 2 functions, a wrapper function calling
a new qat_stats_get/reset function which can be called per service.
Remove cryptodev stats struct from qat_qp, replace with qat_common_stats.
Add links for qat_qp into qat_pci_device using an array per service to
avoid need for a lock and so qp_id for the service can be used as index.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Struct qat_pmd_private held the data needed by cryptodev, common code
now gets most data from struct qat_pci_device instead. qat_pmd_private
is trimmed to hold only sym crypto data and renamed qat_sym_private
to reflect its usage.
Specifically
- remove max_nb_queue_pairs from qat_pmd_private, get from qp_hw_data
- remove max_nb_sesssions from qat_pmd_private as not needed.
- remove qat_gen from qat_pmd_private, get from qat_pci_device instead.
- use qat_pci_device throughout common code instead of qat_pmd_private
- rename qat_pmd_private to qat_sym_dev_private - this now holds only
sym-specific data for the cryptodev API
- extend pci device name to <bdf>_qat for clarity, was just <bdf>
- update qp mem and cookiepool names to reflect the appropriate device,
service and qp.
- rename qat_dev_info_get() to qat_sym_dev_info_get() as mostly sym,
not enough common info to warrant a generic fn.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
The QAT PMD used to register with PCI using the name "crypto_qat".
Keep this name for the driver registered with cryptodev
and use a more generic name "qat" for the PCI registration.
This paves the way for the PCI device to host other services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
- Added struct qat_pci_device to use internally in QAT PMD
to avoid dependencies on rte_cryptodev or rte_compressdev
- Added a global array of these
- Restructured probe/release to separate QAT common init/clear
from crypto pmd create/destroy.
- In QAT common part allocated a qat_pci_device and populated it
- Removed meaningless check in probe for RTE_PROC_PRIMARY
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Create data structures to support different generations
of qat hardware supplying services through different queue pairs.
- Add two new structs qat_gen_hw_data and qat_qp_hw_dat
- Add a qat_service_type enum
An array of qat_qp_hw_data elements is
initialised with constants, these are arranged so that the qp_id used
on the API can be used as an index to pick up the qp data to use.
The constants are common to current generations,
new arrays will be added for future generations.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Move defines related to coalescing from sym header file to qp header
file as these will be common for all services.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Remove dependence on rte_cryptodev from common qp code
to facilitate being used by other device types in future.
Transferred required data into qat-specific structures.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Extracted all sym-specific code from qp setup fns, leaving
generic qat_qp_setup fn and helper fns. Created a new
meta-data struct qat_qp_config to hold all the data needed
to create a qp, filled this out in the sym-specific code
and passed to the generic qp_setup fn.
No need now for rx and tx queue_create fns, one generic
queue_create fn replaces these.
Included the service name (e.g. "sym") in the qp memzone
and cookie pool names.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Move the generic enqueue and dequeue fns from
the qat_sym.c file to the qat_qp.c file
Move generic qp structs to a new qat_qp.h file
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Queue-handling code in dequeue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic dequeue with the sym-specific dequeue
called through the API.
- extracting the sym-specific response processing into a new fn.
- Setting a fn ptr for process_response in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_process_response cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Queue-handling code in enqueue is made generic, so it can
be used by other services in future. This is done by
- Removing all sym-specific refs in input params - replace with void ptrs.
- Wrapping this generic enqueue with the sym-specific enqueue
called through the API.
- Setting a fn ptr for build_request in qp on qp creation
- Passing void * params to this, in the service-specific implementation
qat_sym_build_request cast back to sym structs.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
This commit renames qat_crypto.c/h to qat_sym.c/h
And makes a few whitespace changes to resolve line-length
issues.
Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
This commit adds qat_sym_session.c/h files and moves objects
from qat_algs_build_desc and qat_algs.h
Following objects were moved:
qat_adf/qat_algs_build_desc.c => qat_sym_session.c
- all objects -
qat_adf/qat_algs.h => qat_sym_session.h
- enum qat_crypto_proto_flag
- struct qat_alg_cd
- struct qat_session
- int qat_get_inter_state_size()
- int qat_alg_aead_session_create_content_desc_cipher()
- int qat_alg_aead_session_create_content_desc_auth()
- void qat_alg_init_common_hdr()
- int qat_alg_validate_aes_key()
- int qat_alg_validate_aes_docsisbpi_key()
- int qat_alg_validate_snow3g_key()
- int qat_alg_validate_kasumi_key()
- int qat_alg_validate_3des_key()
- int qat_alg_validate_des_key()
- int qat_cipher_get_block_size()
- int qat_alg_validate_zuc_key()
-- all macros
qat_crypto.h => qat_sym_session.h
int qat_crypto_sym_configure_session()
int qat_crypto_set_session_parameters()
int qat_crypto_sym_configure_session_aead()
int qat_crypto_sym_configure_session_cipher()
int qat_crypto_sym_configure_session_auth()
int qat_alg_aead_session_create_content_desc_cipher()
int qat_alg_aead_session_create_content_desc_auth()
static struct rte_crypto_auth_xform qat_get_auth_xform()
static struct rte_crypto_cipher_xform qat_get_cipher_xform()
Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
This commit removes unused includes from qat_crypto.c
Signed-off-by: Arkadiusz Kusztal <arkadiuszx.kusztal@intel.com>
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
For HMAC algorithms (SHAx-HMAC), the supported digest sizes are not a
fixed value, but a range between 1 and the maximum digest size for those
algorithms.
Fixes: 0f89def76127 ("crypto/armv8: fix HMAC supported key sizes")
Cc: stable@dpdk.org
Signed-off-by: Anoob Joseph <anoob.joseph@caviumnetworks.com>
Signed-off-by: Sachin Saxena <sachin.saxena@nxp.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
rte_cryptodev_get_header_session_size() and
rte_cryptodev_get_private_session_size() functions are
targeting symmetric sessions.
With the future addition of asymmetric operations,
these functions need to be renamed from *cryptodev_*_session_*
to *cryptodev_sym_*_session_* to be symmetric specific.
The two original functions are marked as deprecated
and will be removed in 18.08, so applications can still
use the functions in 18.05.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Shally Verma <shally.verma@caviumnetworks.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Deepak Kumar Jain <deepak.k.jain@intel.com>
The list of workers cores was represented by 64-bit bitmask.
It doesn't work if system has cores with id higher than 63.
This fix changes list of workers cores to array of uint16_t.
The size of array equals to RTE_MAX_LCORE.
Fixes: 4c07e0552f0a ("crypto/scheduler: add multicore scheduling mode")
Cc: stable@dpdk.org
Signed-off-by: Kirill Rybalchenko <kirill.rybalchenko@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
qp->temp_digest is already an array of uint8_t,
so no need to cast to uint8_t*.
Fixes: baf1e63bfd65 ("crypto/aesni_gcm: do not append digest")
Cc: stable@dpdk.org
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Missing AES-CMAC supported parameters
in PMD capabilities.
Fixes: 6491dbbecebb ("crypto/aesni_mb: support AES CMAC")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Marko Kovacevic <marko.kovacevic@intel.com>
This patch fixes the possible duplicated ring names in multi-core
scheduler. Originally two or more multi-core schedulers may have
same worker ring names thus will cause initialization error.
Fixes: 4c07e0552f0a ("crypto/scheduler: add multicore scheduling mode")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When scheduler mode changed from multicore to roundrobin and
back to multicore, scheduler tries to create memory rings with
the same name and fails. The fix allows to lookup and re-use
previously allocated memory rings.
Fixes: 4c07e0552f0a ("crypto/scheduler: add multicore scheduling mode")
Cc: stable@dpdk.org
Signed-off-by: Kirill Rybalchenko <kirill.rybalchenko@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Tested-by: Reshma Pattan <reshma.pattan@intel.com>
CCP PMD supports authentication offload to either of CCP or CPU.
The earlier version of patch provides this option as compile time.
This patch changes this option from compile time to run time.
User can pass "ccp_auth_opt=1" as an additional arguments to vdev
parameter to enable authentication operations on CPU.
Signed-off-by: Ravi Kumar <ravi1.kumar@amd.com>
