This commit adds support for lock-free (linked list based) stack mempool
handler.
In mempool_perf_autotest the lock-based stack outperforms the
lock-free handler for certain lcore/alloc count/free count
combinations*, however:
- For applications with preemptible pthreads, a standard (lock-based)
stack's worst-case performance (i.e. one thread being preempted while
holding the spinlock) is much worse than the lock-free stack's.
- Using per-thread mempool caches will largely mitigate the performance
difference.
*Test setup: x86_64 build with default config, dual-socket Xeon E5-2699 v4,
running on isolcpus cores with a tickless scheduler. The lock-based stack's
rate_persec was 0.6x-3.5x the lock-free stack's.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
This commit adds support for a lock-free (linked list based) stack to the
stack API. This behavior is selected through a new rte_stack_create() flag,
RTE_STACK_F_LF.
The stack consists of a linked list of elements, each containing a data
pointer and a next pointer, and an atomic stack depth counter.
The lock-free push operation enqueues a linked list of pointers by pointing
the tail of the list to the current stack head, and using a CAS to swing
the stack head pointer to the head of the list. The operation retries if it
is unsuccessful (i.e. the list changed between reading the head and
modifying it), else it adjusts the stack length and returns.
The lock-free pop operation first reserves num elements by adjusting the
stack length, to ensure the dequeue operation will succeed without
blocking. It then dequeues pointers by walking the list -- starting from
the head -- then swinging the head pointer (using a CAS as well). While
walking the list, the data pointers are recorded in an object table.
This algorithm stack uses a 128-bit compare-and-swap instruction, which
atomically updates the stack top pointer and a modification counter, to
protect against the ABA problem.
The linked list elements themselves are maintained in a lock-free LIFO
list, and are allocated before stack pushes and freed after stack pops.
Since the stack has a fixed maximum depth, these elements do not need to be
dynamically created.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
The rte_stack library provides an API for configuration and use of a
bounded stack of pointers. Push and pop operations are MT-safe, allowing
concurrent access, and the interface supports pushing and popping multiple
pointers at a time.
The library's interface is modeled after another DPDK data structure,
rte_ring, and its lock-based implementation is derived from the stack
mempool handler. An upcoming commit will migrate the stack mempool handler
to rte_stack.
Signed-off-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
This patch updates the ipsec library programmer's guide with
the additional algorithms which are now supported.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
This patch adds the deprecation notice of changing Cryptodev
symmetric xform structure. The proposed change is to making
key pointers in the crypto xforms (cipher, auth, aead) to
indicate neither the library or the drivers will not change
the content of the key buffer.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Anoob Joseph <anoobj@marvell.com>
Add librte_ipsec into 'Shared Library Versions' list in the release notes.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
openssl PMD support RSA private key operation
using both qt and exp key type.
Set rsa key type feature flag
Signed-off-by: Ayuj Verma <ayverma@marvell.com>
Signed-off-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Add feature flag to reflect RSA private key
operation support using quintuple (crt) or
exponent type key. if PMD support both,
then it should set both.
App should query cryptodev feature flag to check
if Sign and Decryt with CRT keys or exponent is
supported, thus call operation with relevant
key type.
Signed-off-by: Ayuj Verma <ayverma@marvell.com>
Signed-off-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
This patch adds dynamic SGL allocation instead of static one.
The number of element in SGL can be adjusted in each operation
depend of the request.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
As stated in 19.02 deprecation notice, this patch updates the
aesni_mb PMD to remove the support of older Intel-ipsec-mb
library version earlier than 0.52.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Add out-of-place processing, i.e. different source and
destination m_bufs, plus related capability update, tests
and documentation.
Signed-off-by: Fiona Trahe <fiona.trahe@intel.com>
Signed-off-by: Paul Luse <paul.e.luse@intel.com>
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
This commit adds modular multiplicative inverse to Intel
QuickAssist Technology driver. For capabilities or limitations
please refer to qat.rst or qat_asym_capabilities.h.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This commit adds modular exponentiation to Intel QuickAssist
Technology driver. For capabilities or limitations please refer to
qat.rst or qat_asym_capabilities.h.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch adds Poll Mode Driver for asymmetric crypto
functions of Intel QuickAssist Technology hardware.
It contains plain driver with no functions implemented, specific
algorithms will be introduced in separate patches.
This patch depends on a QAT PF driver for device initialization. See
the file docs/guides/cryptodevs/qat.rst for configuration details.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Add new field ff_disable in rte_cryptodev_config. This enables
applications to control the features enabled on the crypto device.
Proposed new layout:
/** Crypto device configuration structure */
struct rte_cryptodev_config {
int socket_id; /**< Socket to allocate resources on */
uint16_t nb_queue_pairs;
/**< Number of queue pairs to configure on device */
+ uint64_t ff_disable;
+ /**< Feature flags to be disabled. Only the following features are
+ * allowed to be disabled,
+ * - RTE_CRYPTODEV_FF_SYMMETRIC_CRYPTO
+ * - RTE_CRYPTODEV_FF_ASYMMETRIC_CRYPTO
+ * - RTE_CRYTPODEV_FF_SECURITY
+ */
};
For eth devices, rte_eth_conf.rx_mode.offloads and
rte_eth_conf.tx_mode.offloads fields are used by applications to
control the offloads enabled on the eth device. This proposal adds a
similar ability for the crypto device.
Signed-off-by: Anoob Joseph <anoobj@marvell.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Add option to provide a global dequeue timeout that is used to create
the eventdev.
The dequeue timeout provided will be common across all the worker
ports. If the eventdev hardware supports power management through
dequeue timeout then this option can be used for verifying power
demands at various packet rates.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Due to internal glibc limitations [1], DPDK may exhaust internal
file descriptor limits when using smaller page sizes, which results
in inability to use system calls such as select() by user
applications.
Single file segments option stores lock files per page to ensure
that pages are deleted when there are no more users, however this
is not necessary because the processes will be holding onto the
pages anyway because of mmap(). Thus, removing pages from the
filesystem is safe even though they may be used by some other
secondary process. As a result, single file segments mode no
longer stores inordinate amounts of segment fd's, and the above
issue with fd limits is solved.
However, this will not work for legacy mem mode. For that, simply
document that using bigger page sizes is the only option.
[1] https://mails.dpdk.org/archives/dev/2019-February/124386.html
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Testpmd can generate multiple flows without taking much cost and this
could be a simple traffic generator for developer's quick tests.
If "--txonly-multi-flow" is specified in the command line, IP source
address is varied to generate multiple flows.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Bernard Iremonger <bernard.iremonger@intel.com>
This addresses the usability issue raised by OVS at DPDK Userspace
summit. It adds general min/max MTU into device info. For compatibility,
and to save space, it fits in a hole in existing structure.
The initial version sets max MTU to normal Ethernet, it is up to
PMD to set larger value if it supports Jumbo frames.
Also remove the deprecation notice introduced in 18.11 regarding this
change and bump ethdev ABI version.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Ian Stokes <ian.stokes@intel.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
This patch enabled RSS for UPD/TCP/SCTP+IPV4/IPV6 packets.
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
If E810 download package failed, driver need to go to safe mode.
In the safe mode, some advanced features will not be supported.
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
This patch enables package downloading to the device. The package is
to be in the /lib/firmware/intel/ice/ddp directory and named ice.pkg.
The package is shared by the kernel driver and the DPDK PMD.
There is no per device package be supported so far, all the
devices can only download the same package. This limitation will
be removed in the future.
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Paul M Stillwell Jr <paul.m.stillwell.jr@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Add a new "show/clear fwd stats all" command to display fwd and port
statistics on the fly.
To be able to do so, the (testpmd only) rte_port structure can't be used
to maintain any statistics.
Moved the stats dump parts from stop_packet_forwarding() and merge with
fwd_port_stats_display() into fwd_stats_display().
fwd engine statistics are then aggregated into a local per port array.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
As those should be replaced by rte_dev_dma_map and rte_dev_dma_unmap
APIs.
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Rami Rosen <ramirose@gmail.com>
Acked-by: Gaetan Rivet <gaetan.rivet@6wind.com>
The DPDK APIs expose 3 different modes to work with memory used for DMA:
1. Use the DPDK owned memory (backed by the DPDK provided hugepages).
This memory is allocated by the DPDK libraries, included in the DPDK
memory system (memseg lists) and automatically DMA mapped by the DPDK
layers.
2. Use memory allocated by the user and register to the DPDK memory
systems. Upon registration of memory, the DPDK layers will DMA map it
to all needed devices. After registration, allocation of this memory
will be done with rte_*malloc APIs.
3. Use memory allocated by the user and not registered to the DPDK memory
system. This is for users who wants to have tight control on this
memory (e.g. avoid the rte_malloc header).
The user should create a memory, register it through rte_extmem_register
API, and call DMA map function in order to register such memory to
the different devices.
The scope of the patch focus on #3 above.
Currently the only way to map external memory is through VFIO
(rte_vfio_dma_map). While VFIO is common, there are other vendors
which use different ways to map memory (e.g. Mellanox and NXP).
The work in this patch moves the DMA mapping to vendor agnostic APIs.
Device level DMA map and unmap APIs were added. Implementation of those
APIs was done currently only for PCI devices.
For PCI bus devices, the pci driver can expose its own map and unmap
functions to be used for the mapping. In case the driver doesn't provide
any, the memory will be mapped, if possible, to IOMMU through VFIO APIs.
Application usage with those APIs is quite simple:
* allocate memory
* call rte_extmem_register on the memory chunk.
* take a device, and query its rte_device.
* call the device specific mapping function for this device.
Future work will deprecate the rte_vfio_dma_map and rte_vfio_dma_unmap
APIs, leaving the rte device APIs as the preferred option for the user.
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Enable users the option to call rte_vfio_dma_map with request to map
to the default vfio fd.
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Gaetan Rivet <gaetan.rivet@6wind.com>
- mbuf_size and mtu are now being calculated according
to the given mb-pool.
- max_mtu is now being set according to the given mtu
the above two changes provide the ability to work with jumbo frames
Signed-off-by: Liron Himi <lironh@marvell.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
The type of value parameter to rte_service_attr_get
should be uint64_t *, since the attributes
are of type uint64_t.
Fixes: 4d55194d76a4 ("service: add attribute get function")
Signed-off-by: Nikhil Rao <nikhil.rao@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
Reviewed-by: Rami Rosen <ramirose@gmail.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
The spinlock implementation is unfair, some threads may take locks
aggressively while leaving the other threads starving for long time.
This patch introduces ticketlock which gives each waiting thread a
ticket and they can take the lock one by one. First come, first serviced.
This avoids starvation for too long time and is more predictable.
Suggested-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Define '__rte_deprecated' usage process.
Suggests keeping old API with '__rte_deprecated' marker including
next LTS, they will be removed just after the LTS release.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Initial process requires oncoming changes described in deprecation
notice should be implemented in a RTE_NEXT_ABI gated way.
This has been discussed in technical board, and since this can cause a
multiple #ifdef blocks in multiple locations of the code, can be
confusing specially for the modifications that requires data structure
changes. Anyway this was not happening in practice.
Making RTE_NEXT_ABI usage more optional based on techboard decision:
http://mails.dpdk.org/archives/dev/2019-January/123519.html
The intention with using RTE_NEXT_ABI was to provide more information
to the user about planned changes, and force developer to think more in
coding level. Since RTE_NEXT_ABI become optional, now the preferred way
to do this is, if possible, sending changes, described in deprecation
notice, as a separate patch and reference it in deprecation notice.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
The original document written from the point of ABI versioning but later
additions make document confusing, convert document into a ABI/API
policy documentation and organize the document in subsections:
- ABI/API Deprecation
- Experimental APIs
- Library versioning
- ABI versioning
Aim to clarify confusion between deprecation versioned ABI and overall
ABI/API deprecation, also ABI versioning and Library versioning by
organizing the sections.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Kevin Traynor <ktraynor@redhat.com>
Support for 16.11 has ended. 17.11 and 18.11 are the current LTSs.
Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Update cross build tool links as newer cross build tools
version are provided on Linaro, and attempts to download
the old one give permission denied.
Fixes: 01add9da25cd ("doc: add cross compiling guide")
Cc: stable@dpdk.org
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Acked-by: Stephen Hemminger <sthemmin@microsoft.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
GitHub is a service used by developers to store repositories. GitHub
provides service integrations that allow 3rd party services to access
developer repositories and perform actions. One of these services is
Travis-CI, a simple continuous integration platform.
This series introduces the ability for any github mirrors of the DPDK
project, including developer mirrors, to kick off builds under the
travis CI infrastructure. For now, this just means compilation - no
other kinds of automated run exists yet. In the future, this can be
expanded to execute and report results for any test-suites that might
exist.
This is a simple initial implementation of a travis build for the DPDK
project. It doesn't require any changes from individual developers to
enable, but will allow those developers who opt-in to GitHub and the
travis service to get automatic builds for every push they make.
The files added under .ci/ exist so that in the future, other CI
support platforms (such as cirrus, appveyor, etc.) could have a common
place to put their requisite scripts without polluting the main tree.
Signed-off-by: Aaron Conole <aconole@redhat.com>
Signed-off-by: Michael Santana <msantana@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
This patch adds the functional test scripts to ipsec-secgw
sample application for both transport and tunnel working
mode.
Updated a bit on common_defs to use "mktemp" instead of "tempfile"
as Fedora does not like the command.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
This patch adds AES-CTR cipher algorithm support to ipsec
library.
Signed-off-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
This patch adds XTS capabilities and enables
XTS cipher mode on QAT.
It also updates the documentation for newly
supported AES XTS mode on QAT.
Signed-off-by: Damian Nowak <damianx.nowak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
This patch adds fallback to fixed compression
feature during dynamic compression, when the input data size
is greater than IM buffer size / 1.1. This feature doesn't
stop compression proccess when IM buffer can be too small
to handle produced data.
Signed-off-by: Tomasz Jozwiak <tomaszx.jozwiak@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
in 18.08 new cache-aligned structure rte_crypto_asym_op was introduced.
As it also was included into rte_crypto_op, it caused implicit change
in rte_crypto_op layout and alignment: now rte_crypto_op is cache-line
aligned has a hole of 40/104 bytes between phys_addr and sym/asym op.
It looks like unintended ABI breakage, plus such change can cause
negative performance effects:
- now status and sym[0].m_src lies on different cache-lines, so
post-process code would need extra cache-line read.
- new alignment causes grow of the space requirements and cache-line
reads/updates for structures that contain rte_crypto_op inside.
As there seems no actual need to have rte_crypto_asym_op cache-line
aligned, and rte_crypto_asym_op is not intended to be used on it's own -
the simplest fix is just to remove cache-line alignment for it.
As the immediate positive effect: on IA ipsec-secgw performance increased
by 5-10% (depending on the crypto-dev and algo used).
My guess that on machines with 128B cache-line and lookaside-protocol
capable crypto devices the impact will be even more noticeable.
Fixes: 26008aaed14c ("cryptodev: add asymmetric xform and op definitions")
Cc: stable@dpdk.org
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Fiona Trahe <fiona.trahe@intel.com>
Acked-by: Shally Verma <shallyv@marvell.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
Improved MAC swap performance for ARM platform.
The improvement was achieved by using neon intrinsics
to save CPU cycles and doing swap for four packets
at a time.
The optimization had 15% - 20% throughput boost
in testpmd MAC swap mode.
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
EventDev i.e consumer needs to be started before starting the
event producers.
Update documentation of EventDev and EventDev adapters.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Reviewed-by: Erik Gabriel Carrillo <erik.g.carrillo@intel.com>
Reviewed-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
