In virtio_dev_extbuf_alloc(), the shinfo structure used to store
the reference counter and the free callback of the external buffer
is by default stored inside the mbuf data.
This is wrong because the mbuf (and its data) can be freed before
the external buffer, for instance in the following situation:
pkt2 = rte_pktmbuf_alloc(mp);
rte_pktmbuf_attach(pkt2, pkt);
rte_pktmbuf_free(pkt);
After this, pkt is freed, but it still contains shinfo, which is
referenced by pkt2.
Fix this by always storing the shinfo beside the external buffer.
Fixes: c3ff0ac70a ("vhost: improve performance by supporting large buffer")
Cc: stable@dpdk.org
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch fixes the feature negotiation for vhost crypto during
initialization. The patch uses the newly created driver start
function to inform the driver type with the fixed vhost features.
In addition the patch provides a new API specifically used by
the application to start a vhost-crypto driver.
Fixes: 939066d965 ("vhost/crypto: add public function implementation")
Cc: stable@dpdk.org
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When testing on some x86 platforms, code compiled with meson was observed
running at a different power-license level to that compiled with make. This
is due to the fact that meson auto-detects the instruction sets available
on the system and enabled AVX512 rte_memcpy when AVX512 was available,
while on make, a build time AVX-512 flag needed to be explicitly set to
enable that AVX512 rte_memcpy code path.
In the absence of runtime path selection for rte_memcpy - which is
complicated by it being a static inline function in a header file - we can
fix this behaviour regression by similarly having a build-time option which
must be set to enable the AVX-512 memcpy path.
Fixes: a25a650be5 ("build: add infrastructure for meson and ninja builds")
Fixes: 3e1bb55fd6 ("build/x86: add SSE flags")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Yingya Han <yingyax.han@intel.com>
When running Sphinx through ninja, the wrapper configured in meson
redirects stdout to a log file.
It makes more important to print issues on stderr.
Some warnings generated by the conf.py were hidden because
printed on stdout. The first improvement is to print them on stderr.
The second measure is to stop processing if meson was configured
with --werror.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The DPDK version should appear in the top left corner of the HTML guides.
When dropping make, the variable version has been removed,
so Sphinx stopped integrating the version number.
Fixes: a4362f1502 ("doc: build without using make")
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
When reading bus and segment values using SPDRP_BUSNUMBER
bits 24-31 are undefined.
They are cleared to verify we read the segment number correctly.
Fixes: c3adf8144a ("bus/pci: support segment as address domain on Windows")
Cc: stable@dpdk.org
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
rte_cldemote is similar to a prefetch hint - in reverse.
On x86, cldemote(addr) enables software to hint to hardware that line is
likely to be shared. This is quite useful in core-to-core communications
where cache-line is likely to be shared.
ARM and PPC implementation is provided with NOP and can be added if any
equivalent instructions could be used for implementation on those
architectures.
Signed-off-by: Omkar Maslekar <omkar.maslekar@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: David Christensen <drc@linux.vnet.ibm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Updating export files (supposed to disappear at some point, but still
there) might be missed when removing symbols in the API / map files.
Add a check for this case.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
The incriminated commit forgot to clean the Windows export file.
Fixes: 3cd73a1a1c ("eal: simplify exit functions")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
There is a potential race condition in 'service_attr_get' which will cause
test failures since the service core thread is still running while the
values are being retrieved/reset.
This patch fixes the race condition by waiting for the service core thread
to stop before continuing with the unit test checks.
Fixes: 4d55194d76 ("service: add attribute get function")
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Add new internal wrapper function for use by pci drivers as a
.probe function to attach to an event interface. Same as
rte_event_pmd_pci_probe, except the caller can specify the name.
Updated rte_event_pmd_pci_probe so as to not duplicate
code.
Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
This commit implements the eventdev ABI changes required by
the DLB/DLB2 PMDs. Several data structures and constants are modified
or added in this patch, thereby requiring modifications to the
dependent apps and examples.
The DLB/DLB2 hardware does not conform exactly to the eventdev interface.
1) It has a limit on the number of queues that may be linked to a port.
2) Some ports a further restricted to a maximum of 1 linked queue.
3) DLB does not have the ability to carry the flow_id as part
of the event (QE) payload. Note that the DLB2 hardware is capable of
carrying the flow_id.
Following is a detailed description of the changes that have been made.
1) Add new fields to the rte_event_dev_info struct. These fields allow
the device to advertise its capabilities so that applications can take
the appropriate actions based on those capabilities.
struct rte_event_dev_info {
uint32_t max_event_port_links;
/**< Maximum number of queues that can be linked to a single event
* port by this device.
*/
uint8_t max_single_link_event_port_queue_pairs;
/**< Maximum number of event ports and queues that are optimized for
* (and only capable of) single-link configurations supported by this
* device. These ports and queues are not accounted for in
* max_event_ports or max_event_queues.
*/
}
2) Add a new field to the rte_event_dev_config struct. This field allows
the application to specify how many of its ports are limited to a single
link, or will be used in single link mode.
/** Event device configuration structure */
struct rte_event_dev_config {
uint8_t nb_single_link_event_port_queues;
/**< Number of event ports and queues that will be singly-linked to
* each other. These are a subset of the overall event ports and
* queues; this value cannot exceed *nb_event_ports* or
* *nb_event_queues*. If the device has ports and queues that are
* optimized for single-link usage, this field is a hint for how many
* to allocate; otherwise, regular event ports and queues can be used.
*/
}
3) Replace the dedicated implicit_release_disabled field with a bit field
of explicit port capabilities. The implicit_release_disable functionality
is assigned to one bit, and a port-is-single-link-only attribute is
assigned to other, with the remaining bits available for future assignment.
* Event port configuration bitmap flags */
#define RTE_EVENT_PORT_CFG_DISABLE_IMPL_REL (1ULL << 0)
/**< Configure the port not to release outstanding events in
* rte_event_dev_dequeue_burst(). If set, all events received through
* the port must be explicitly released with RTE_EVENT_OP_RELEASE or
* RTE_EVENT_OP_FORWARD. Must be unset if the device is not
* RTE_EVENT_DEV_CAP_IMPLICIT_RELEASE_DISABLE capable.
*/
#define RTE_EVENT_PORT_CFG_SINGLE_LINK (1ULL << 1)
/**< This event port links only to a single event queue.
*
* @see rte_event_port_setup(), rte_event_port_link()
*/
#define RTE_EVENT_PORT_ATTR_IMPLICIT_RELEASE_DISABLE 3
/**
* The implicit release disable attribute of the port
*/
struct rte_event_port_conf {
uint32_t event_port_cfg;
/**< Port cfg flags(EVENT_PORT_CFG_) */
}
This patch also removes the depreciation notice and announce
the new eventdev ABI changes in release note.
Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Add minimum burst throughout the scheduler pipeline and a flush counter.
Use a single threaded ring implementation for the reorder buffer free list.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Since we are not holding the mbufs or creating any references
in the app, hence mbuf fast free offload can be enabled.
Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Issue has been observed in case of multi segments where mbuf
data gets corrupted due to missing barriers. Changes made to
mbuf just before LMTST by one core gets updatded when the
same mbuf is in use by another core, leading to corruption.
It should be ensured that all changes made to mbuf should be
written before LMTST.
Fixes: cbd5710db4 ("net/octeontx2: add Tx multi segment version")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: Harman Kalra <hkalra@marvell.com>
Improve single flow performance by moving the point of coherence
to the end of transmit sequence.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Add SWTAG flush operation at the end of transmit sequence to
immediately release the tag held by the core.
Reuse Tag address to check SWTAG completion status.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
In the op new mode of crypto adapter, the completed crypto operation
is submitted to the event device by the OCTEON TX2 crypto PMD.
During event device dequeue the result of crypto operation is checked.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
The crypto adapter callback functions and associated data structures
are added.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Some functions are common across cryptodev pmd and the event
crypto adapter. This patch moves them into a helper file.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
The resources held by crypto adapter should be freed when the
test suite exits.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
The arguments passed to rte_event_crypto_adapter_caps_get() and
rte_event_crypto_adapter_create() are incorrect.
In the rte_event_crypto_adapter_caps_get(), event device id should
be the first argument and cryptodev id should be the second argument.
In the rte_event_crypto_adapter_create(), the event device id should
be the second argument.
Fixes: 3c2c535ecf ("test: add event crypto adapter auto-test")
Cc: stable@dpdk.org
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
The capability of a hardware event device should be checked before
creating a event crypto adapter in a particular mode. The test case
returns error if the mode is not supported.
Signed-off-by: Ankur Dwivedi <adwivedi@marvell.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
Allows creation of net_null if vdev EAL option is not specified and
uninit vdev created in the test. The change also adds error checks
for vdev init and uninit.
Signed-off-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
adapter_multi_eth_add_del() does vdev init but doesn't uninit them.
This causes issues when running event_eth_rx_adapter_autotest multiple
times.
The fix does vdev uninit before exiting the test.
Signed-off-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
In rte_event_crypto_adapter_create_ext() allocated memory for
adapter, we should free it when error happens, otherwise it
will lead to memory leak.
Fixes: 7901eac340 ("eventdev: add crypto adapter implementation")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Coverity flags that 'portal' variable is used before
it's checked for NULL. This patch fixes this issue.
Coverity issue: 323516
Fixes: 4ab57b042e ("event/dpaa2: affine portal at runtime during I/O")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Acked-by: Nipun Gupta <nipun.gupta@nxp.com>
Validate events configured in ssopf against the total number of
events configured across all the RX/TIM event adapters.
Events available to ssopf can be reconfigured by passing the required
amount to kernel bootargs and are only limited by DRAM size.
Example:
ssopf.max_events= 2097152
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
The function rte_zmalloc() could return NULL, the return value
need to be checked.
Fixes: a3bbf2e097 ("eventdev: add eth Tx adapter implementation")
Cc: stable@dpdk.org
Signed-off-by: Yunjian Wang <wangyunjian@huawei.com>
Reviewed-by: Nikhil Rao <nikhil.rao@intel.com>
The telemetry library is connected with eventdev xstats and
port link info. The following new telemetry commands are added:
/eventdev/dev_list
/eventdev/port_list,DevID
/eventdev/queue_list,DevID
/eventdev/dev_xstats,DevID
/eventdev/port_xstats,DevID,PortID
/eventdev/queue_xstats,DevID,PortID
/eventdev/queue_links,DevID,PortID
queue_links command displays a list of queues linked with a specified
eventdev port and a service priority associated with each link.
Signed-off-by: Mike Ximing Chen <mike.ximing.chen@intel.com>
Reviewed-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
Currently, the rte_flow functions are not defined as thread safe.
DPDK applications either call the functions in single thread or
protect any concurrent calling for the rte_flow operations using
a lock.
For PMDs support the flow operations thread safe natively, the
redundant protection in application hurts the performance of the
rte_flow operation functions.
And the restriction of thread safe is not guaranteed for the
rte_flow functions also limits the applications' expectation.
This feature is going to change the rte_flow functions to be thread
safe. As different PMDs have different flow operations, some may
support thread safe already and others may not. For PMDs don't
support flow thread safe operation, a new lock is defined in ethdev
in order to protects thread unsafe PMDs from rte_flow level.
A new RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE device flag is added to
determine whether the PMD supports thread safe flow operation or not.
For PMDs support thread safe flow operations, set the
RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE flag, rte_flow level functions will
skip the thread safe helper lock for these PMDs. Again the rte_flow
level thread safe lock only works when PMD operation functions are
not thread safe.
For the PMDs which don't want the default mutex lock, just set the
flag in the PMD, and add the prefer type of lock in the PMD. Then
the default mutex lock is easily replaced by the PMD level lock.
The change has no effect on the current DPDK applications. No change
is required for the current DPDK applications. For the standard posix
pthread_mutex, if no lock contention with the added rte_flow level
mutex, the mutex only does the atomic increasing in
pthread_mutex_lock() and decreasing in
pthread_mutex_unlock(). No futex() syscall will be involved.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add pthread mutex lock as it is needed for the thread safe rte_flow
functions.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Tested-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Narcisa Vasile <navasile@linux.microsoft.com>
Clarify the documentation of QinQ flags, and extend the meaning of the
flag: if PKT_RX_QINQ_STRIPPED is set and PKT_RX_VLAN_STRIPPED is unset,
only the outer VLAN is removed from packet data, but both tci are saved
in mbuf->vlan_tci (inner) and mbuf->vlan_tci_outer (outer).
Signed-off-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reduce number of source BDs to submit a request to crypto engine.
This improves the performance as crypto engine fetches all the BDs in
single cycle. Adjust optional metadata (OMD) in continuation of
fixed meta data (FMD).
Signed-off-by: Vikas Gupta <vikas.gupta@broadcom.com>
Signed-off-by: Raveendra Padasalagi <raveendra.padasalagi@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
A new functions which uses the structure of the test vectors for SDAP
is added and call a functions responsible to call the test_pdcp_proto
with the test vector both for encapsulation and decapsulation.
Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
The function test_pdcp_proto was relying to heavily on the structure
of test vectors for PDCP making it difficult to be reusable.
The function is changed to take all the test parameters as input and
does not need access to the tests vectors anymore.
Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
The test vectors are structured in a more readable way compared
to test vector for PDCP. This structure allows to have all the
information about a test vector at the same place.
Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Based on the new field in PDCP xform, a decision is made
to create a PDCP session with or without SDAP enabled.
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Based on the new field in PDCP xform, a decision is made
to create a PDCP session with or without SDAP enabled.
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Various xform structures are being copied in
rte_security guide which can be referred from the
API documentation generated by Doxygen. The security guide
does not talk about specific details of these xforms and
thus are removed from the security guide.
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
The SDAP is a protocol in the LTE stack on top of PDCP for
QOS. A particular PDCP session may or may not have
SDAP enabled. But if it is enabled, SDAP header should be
authenticated but not encrypted if both confidentiality and
integrity is enabled. Hence, the driver should be intimated
from the xform so that it skip the SDAP header while encryption.
A new field is added in the PDCP xform to specify SDAP is enabled.
The overall size of the xform is not changed, as hfn_ovrd is just
a flag and does not need uint32. Hence, it is converted to uint8_t
and a 16 bit reserved field is added for future.
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
The SDAP is a protocol in the LTE stack on top of PDCP. It is
dedicated to QoS.
The difficulty of implementing this protocol is because the
PDCP change behavior regarding encryption and authentication
of the SDU it receives. In effect PDCP shall not encrypt the
SDAP SDU but must authenticate it (when encryption and
authentication is enabled).
The current version of SEC does not support the SDAP and the
change of behavior of PDCP prevent the use of the PDCP
protocol command available.
The way to do it is to reuse the PDCP implementation but to
not use the PDCP protocol and to have descriptors which
performs the PDCP protocol.
It is implemented by doing small changes of code:
#ifdef SDAP_SUPPORT
length += SDAP_BYTE_SIZE;
offset -= SDAP_BYTE_SIZE;
#endif
after having computed the size of the SN to read from the
input data, then
#ifdef SDAP_SUPPORT
MATHI(p, MATH0, LSHIFT, 8, MATH1, 8, 0);
MATHB(p, MATH1, AND, sn_mask, MATH1, 8, IFB | IMMED2);
#else
MATHB(p, MATH0, AND, sn_mask, MATH1, 8, IFB | IMMED2);
#endif
It will keep the SN and the SDAP header in MATH0, then shift
it to remove the SDAP header and store the result in MATH1.
Signed-off-by: Franck Lenormand <franck.lenormand@nxp.com>
Signed-off-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch removes enumerators RTE_CRYPTO_CIPHER_LIST_END,
RTE_CRYPTO_AUTH_LIST_END, RTE_CRYPTO_AEAD_LIST_END to prevent
ABI breakage that may arise when adding new crypto algorithms.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch updates the xform with right configuration.
For session based ops, sym session pool is created with
valid userdata size.
Fixes: 24054e3640 ("test/crypto: use separate session mempools")
Cc: stable@dpdk.org
Signed-off-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch adds the cryptodev raw API test support to unit test.
In addition a new test-case for QAT PMD for the test type is
enabled.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch updates QAT PMD to add raw data-path API support.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
This patch adds raw data-path APIs for enqueue and dequeue
operations to cryptodev. The APIs support flexible user-define
enqueue and dequeue behaviors.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Signed-off-by: Piotr Bronowski <piotrx.bronowski@intel.com>
Acked-by: Adam Dybkowski <adamx.dybkowski@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch updates ``rte_crypto_sym_vec`` structure to add
support for both cpu_crypto synchronous operation and
asynchronous raw data-path APIs. The patch also includes
AESNI-MB and AESNI-GCM PMD changes, unit test changes and
documentation updates.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
