The comments in rte_rib6 were cut-and-pasted from rte_rib
and because of that some references to rte_rib_node were
not updated.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Found by nullfree.cocci.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
[David: for lpm parts:]
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
[David: for vdpa/mlx5 parts:]
Acked-by: Matan Azrad <matan@nvidia.com>
[David: for dma/dpaa2, raw/ifpga, vdpa/mlx5:]
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
[David: reran cocci.sh and updated common/mlx5 and cryptodev asym test]
Signed-off-by: David Marchand <david.marchand@redhat.com>
Make sure all functions which use the convention that XXX_free(NULL)
is a nop are all documented.
The wording is chosen to match the documentation of free(3).
"If ptr is NULL, no operation is performed."
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
[David: squashed with other series updates, unified wording]
Remove extraneous phrase "This API is used to" and use
active instead of passive voice when describing a function.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
[David: for raw/ioat and dmadev parts:]
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Conor Walsh <conor.walsh@intel.com>
Add support for using hugepages for worker lcore stack memory. The
intent is to improve performance by reducing stack memory related TLB
misses and also by using memory local to the NUMA node of each lcore.
EAL option '--huge-worker-stack[=stack-size-in-kbytes]' is added to allow
the feature to be enabled at runtime. If the size is not specified,
the system pthread stack size will be used.
Signed-off-by: Don Wallwork <donw@xsightlabs.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
GCC 12 raises warnings on usage of rte_memcpy with IPv4 options handling
in fragments for both the ip_frag library and unit tests.
For example in the library:
In function ‘_mm256_storeu_si256’,
inlined from ‘rte_mov32’ at
../lib/eal/x86/include/rte_memcpy.h:347:2,
inlined from ‘rte_mov128’ at
../lib/eal/x86/include/rte_memcpy.h:369:2,
inlined from ‘rte_memcpy_generic’
at ../lib/eal/x86/include/rte_memcpy.h:445:4,
inlined from ‘rte_memcpy’
at ../lib/eal/x86/include/rte_memcpy.h:851:10,
inlined from ‘__create_ipopt_frag_hdr’
at ../lib/ip_frag/rte_ipv4_fragmentation.c:68:4,
inlined from ‘rte_ipv4_fragment_packet’
at ../lib/ip_frag/rte_ipv4_fragmentation.c:242:16:
/usr/lib/gcc/x86_64-redhat-linux/12/include/avxintrin.h:935:8: error:
array subscript ‘__m256i_u[1]’ is partly outside array bounds of
‘uint8_t[60]’ {aka ‘unsigned char[60]’} [-Werror=array-bounds]
935 | *__P = __A;
| ~~~~~^~~~~
../lib/ip_frag/rte_ipv4_fragmentation.c: In function
‘rte_ipv4_fragment_packet’:
../lib/ip_frag/rte_ipv4_fragmentation.c:122:17: note: at offset [52, 60]
into object ‘ipopt_frag_hdr’ of size 60
122 | uint8_t ipopt_frag_hdr[IPV4_HDR_MAX_LEN];
| ^~~~~~~~~~~~~~
To resolve the compilation warning, replace the rte_memcpy with memcpy.
Fixes: b50a14a853 ("ip_frag: add IPv4 options fragment")
Signed-off-by: Huichao Cai <chcchc88@163.com>
If called to allocate memory of size is between multiple of hugepage
size minus malloc_header_len and hugepage size, rte_malloc fails.
This fix replaces malloc_elem_trailer_len with malloc_elem_overhead in
try_expand_heap() to include malloc_elem_header_len when calculating
n_seg.
Bugzilla ID: 800
Fixes: 07dcbfe010 ("malloc: support multiprocess memory hotplug")
Cc: stable@dpdk.org
Signed-off-by: Fidaullah Noonari <fidaullah.noonari@emumba.com>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
rte_dump_stack() needs to be usable in situations when a bug is
encountered and from signal handlers (such as SEGV).
Glibc backtrace_symbols() calls malloc which makes it
dangerous in a signal handler that is handling errors that maybe
due to memory corruption. Additionally, rte_log() is unsafe because
syslog() is not signal safe; printf() is also documented as
not being safe.
This version formats message and uses writev for each line in a manner
similar to what glibc version of backtrace_symbols_fd() does. The
FreeBSD version of backtrace_symbols_fd() is not signal safe.
Sample output:
0: ./build/app/dpdk-testpmd (rte_dump_stack+0x2b) [560a6e9c002b]
1: ./build/app/dpdk-testpmd (main+0xad) [560a6decd5ad]
2: /lib/x86_64-linux-gnu/libc.so.6 (__libc_start_main+0xcd) [7fd43d3e27fd]
3: ./build/app/dpdk-testpmd (_start+0x2a) [560a6e83628a]
Bugzilla ID: 929
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: David Marchand <david.marchand@redhat.com>
copy_data was returning a pointer to an increased (off by one) descriptor.
Subsequent calls to copy_data in the library were then failing.
Fix this by incrementing the descriptor only if there is some left data
to copy.
Fixes: 4414bb6701 ("vhost/crypto: fix build with GCC 12")
Cc: stable@dpdk.org
Reported-by: Jakub Poczatek <jakub.poczatek@intel.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Tested-by: Jakub Poczatek <jakub.poczatek@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
GCC 12 raises the following warning:
In file included from ../lib/mempool/rte_mempool.h:46,
from ../lib/mbuf/rte_mbuf.h:38,
from ../lib/vhost/vhost_crypto.c:7:
../lib/vhost/vhost_crypto.c: In function ‘rte_vhost_crypto_fetch_requests’:
../lib/eal/x86/include/rte_memcpy.h:371:9: warning: array subscript 1 is
outside array bounds of ‘struct virtio_crypto_op_data_req[1]’
[-Warray-bounds]
371 | rte_mov32((uint8_t *)dst + 3 * 32, (const uint8_t *)src + 3 * 32);
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../lib/vhost/vhost_crypto.c:1178:42: note: while referencing ‘req’
1178 | struct virtio_crypto_op_data_req req;
| ^~~
Split this function and separate the per descriptor copy.
This makes the code clearer, and the compiler happier.
Note: logs for errors have been moved to callers to avoid duplicates.
Fixes: 3c79609fda ("vhost/crypto: handle virtually non-contiguous buffers")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Since the commit 02798b0735 ("vhost: improve virtio-net layer logs"),
vhost logs contain the socket path as a prefix.
Async dequeue path was copied from the sync dequeue path but a log
was incorrect.
Fixes: 84d5204310 ("vhost: support async dequeue for split ring")
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patchs renames the local variables free_entries to
avail_entries in the dequeue path.
Indeed, this variable represents the number of new packets
available in the Virtio transmit queue, so these entries
are actually used, not free.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
This patch implements packed ring dequeue data path
for asynchronous vhost.
Signed-off-by: Cheng Jiang <cheng1.jiang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
rte_vhost_clear_queue_thread_unsafe() supports to clear
in-flight packets for async enqueue only. But after
supporting async dequeue, this API should support async dequeue too.
This patch also adds the thread-safe version of this API,
the difference between the two API is that thread safety uses lock.
These APIs maybe used to clean up packets in the async channel
to prevent packet loss when the device state changes or
when the device is destroyed.
Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Jiayu Hu <jiayu.hu@intel.com>
The Virtio specification requires that in case of checksum
offloading, the pseudo-header checksum must be set in the
L4 header.
When received from another Vhost-user port, the packet
checksum might already contain the pseudo-header checksum
but we have no way to know it. So we have no other choice
than doing the pseudo-header checksum systematically.
This patch handles this using the rte_net_intel_cksum_prepare()
helper.
Fixes: 859b480d5a ("vhost: add guest offload setting")
Cc: stable@dpdk.org
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
During adapter create, memory is allocated for storing event port
configuration which is freed during adapter free. The following
error is seen during free "EAL: Error: Invalid memory"
The service data pointer storage for txa_service_data_array is
allocated during adapter create with incorrect size which is less
than the required size.
Initialization of this memory causes buffer overflow and result in
metadata overwrite of event port config memory allocated above
and results in the above error message during free.
Allocating the correct size of memory for txa_service_data_array
prevents overwriting other memory areas like event port config
memory.
Fixes: a3bbf2e097 ("eventdev: add eth Tx adapter implementation")
Cc: stable@dpdk.org
Signed-off-by: Naga Harish K S V <s.v.naga.harish.k@intel.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>
Fix the UDP header fields, wrong byte order used for src and dst port
and wrong offset used when updating UDP datagram length.
Fixes: 01eef5907f ("ipsec: support NAT-T")
Cc: stable@dpdk.org
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
ALIGNMENT_MASK is only used internally.
Besides it lacks a DPDK-related prefix.
Hide it from external eyes.
Fixes: f5472703c0 ("eal: optimize aligned memcpy on x86")
Cc: stable@dpdk.org
Reported-by: Morten Brørup <mb@smartsharesystems.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
The function rte_pie_drop was attempting to do a random probability
drop, but because of incorrect usage of fixed point divide
it would always return 1.
Change to use new rte_drand() instead.
Fixes: 44c730b0e3 ("sched: add PIE based congestion management")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Jasvinder Singh <jasvinder.singh@intel.com>
The qdelay variable is derived from and compared to 64 bit
value so it doesn't have to be floating point.
Fixes: 44c730b0e3 ("sched: add PIE based congestion management")
Cc: stable@dpdk.org
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Jasvinder Singh <jasvinder.singh@intel.com>
The PIE code and other applications can benefit from having a
fast way to get a random floating point value. This new function
is equivalent to drand() in the standard library.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
These header includes have been flagged by the iwyu_tool
and removed.
Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>
Acked-by: Chenbo Xia <chenbo.xia@intel.com>
These header includes have been flagged by the iwyu_tool
and removed.
Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Bug scenario:
1. start testpmd:
$ dpdk-testpmd -l 4-6 -a 0000:7d:00.0 --trace=.* \
--file-prefix=trace_autotest -- -i
2. then observed:
EAL: eal_trace_init():93 failed to initialize trace [File exists]
EAL: FATAL: Cannot init trace
EAL: Cannot init trace
EAL: Error - exiting with code: 1
The root cause it that the offset set wrong with long file-prefix and
then lead the strftime return failed.
At the same time, trace_session_name_generate() uses errno as the return
value, but the errno was not set if strftime returned zero.
A previously set errno (EEXIST or ENOENT from call to mkdir for creating
the runtime configuration directory) was returned in this case.
This is fragile and may lead to incorrect logic if errno was set
to 0 previously.
This also resulted in inaccurate prompting.
Set errno to ENOSPC if strftime return zero.
Fixes: 321dd5f8fa ("trace: add internal init and fini interface")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Bug scenario:
1. start testpmd:
$ dpdk-testpmd -l 4-6 -a 0000:7d:00.0 --trace=.* -- -i
2. quit testpmd and then observed segment fault:
Bye...
Segmentation fault (core dumped)
The root cause is that rte_trace_save() and eal_trace_fini() access
the huge pages which were cleanup by rte_eal_memory_detach().
This patch moves rte_trace_save() and eal_trace_fini() before
rte_eal_memory_detach() to fix the bug.
Fixes: dfbc61a2f9 ("mem: detach memsegs on cleanup")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Tested-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
The P4 language requires marking a header as valid before any of the
header fields are written as opposed to after the writes are done.
Hence, the optimization of replacing the sequence of instructions to
generate a header by reading it from the table action data with a
single DMA internal instruction are reworked from "mov all + validate
-> dma" to "validate + mov all -> dma".
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Fix comparison used to check against the maximum number of learner
table timeouts.
Fixes: e2ecc53582 ("pipeline: improve learner table timers")
Signed-off-by: Harshad Narayane <harshad.suresh.narayane@intel.com>
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Fix segmentation fault due to null pointer dereferencing inside the
"mirror" instruction when number of mirroring slots is set to 0. This
was taking place when the "mirror" instruction was used without the
mirror feature being properly configured, i.e. the API function
rte_swx_pipeline_mirroring_config was not called at initialization.
Fixes: dac0ecd909 ("pipeline: support packet mirroring")
Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
rte_xmm_t is a union type which wraps around xmm_t and maps its contents
to scalar structures. Since C++ has stricter type conversion rules than
C, the rte_xmm_t::x has to be used instead of C-casting.
The generated assembly is identical to the code without the fix (checked
both on x86 and RISC-V).
Fixes: 406937f89f ("lpm: add scalar version of lookupx4")
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
rte_xmm_t is a union type which wraps around xmm_t and maps its contents
to scalar structures. Since C++ has stricter type conversion rules than
C, the rte_xmm_t::x has to be used instead of C-casting.
Fixes: f22e705ebf ("eal/riscv: support RISC-V architecture")
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
The cmdline library cmdline_parse() function parses a command and
executes the action automatically too. The cmdline_valid_buffer function
also uses this function to validate commands, meaning that there is no
function to validate a command as ok without executing it.
To fix this omission, we extract the body of cmdline_parse into a new
static inline function with an extra parameter to indicate whether the
action should be performed or not. Then we create two wrappers around
that - a replacement for the existing cmdline_parse function where the
extra parameter is "true" to execute the command, and a new function
"cmdline_parse_check" which passes the parameter as "false" to perform
cmdline validation only.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Weiyuan Li <weiyuanx.li@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
A new event RTE_ETH_EVENT_RX_AVAIL_THRESH should be generated by HW
when number of available descriptors in Rx queue goes below the
threshold.
The threshold is defined as a percentage of an Rx queue size with valid
values from 0 to 99 (inclusive). Zero (default) value disables it.
There is no capability reporting for the feature. Application should
simply try to set required threshold value and handle result.
Add testpmd commands to control the threshold:
set port <port_id> rxq <rxq_id> avail_thresh <avail_thresh_num>
Signed-off-by: Spike Du <spiked@nvidia.com>
Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
MCS lock, PF lock and Ticket lock have no arch specific implementation,
there is no need for the extra redirection in headers.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Stanislaw Kardach <kda@semihalf.com>
Add all necessary elements for DPDK to compile and run EAL on SiFive
Freedom U740 SoC which is based on SiFive U74-MC (ISA: rv64imafdc)
core complex.
This includes:
- EAL library implementation for rv64imafdc ISA.
- meson build structure for 'riscv' architecture. RTE_ARCH_RISCV define
is added for architecture identification.
- xmm_t structure operation stubs as there is no vector support in the
U74 core.
Compilation was tested on Ubuntu and Arch Linux using riscv64 toolchain.
Clang compilation currently not supported due to issues with missing
relocation relaxation.
Two rte_rdtsc() schemes are provided: stable low-resolution using rdtime
(default) and unstable high-resolution using rdcycle. User can override
the scheme by defining RTE_RISCV_RDTSC_USE_HPM=1 during compile time of
both DPDK and the application. The reasoning for this is as follows.
The RISC-V ISA mandates that clock read by rdtime has to be of constant
period and synchronized between all hardware threads within 1 tick
(chapter 10.1 in version 20191213 of RISC-V spec).
However this clock may not be of high-enough frequency for dataplane
uses. I.e. on HiFive Unmatched (FU740) it is 1MHz.
There is a high-resolution alternative in form of rdcycle which is
clocked at the core clock frequency. The drawbacks are that it may be
disabled during sleep (WFI), its frequency might change due to DVFS and
it is core-local and therefore cannot be used as a wall-clock. It can
however be used for micro-benchmarking user applications, similarly to
Aarch64's PMCCNTR PMU counter.
The platform is currently marked as linux-only because rte_cycles
implementation uses the timebase-frequency device-tree node read through
the proc file system. Such approach was chosen because Linux kernel
depends on the presence of this device-tree node.
The i40e PMD driver is disabled on RISC-V as the rv64gc ISA has no vector
operations.
The compilation of following modules has been disabled by this commit
and will be re-enabled in later commits as fixes are introduced:
net/ixgbe, net/memif, net/tap, example/l3fwd.
Sponsored-by: Frank Zhao <frank.zhao@starfivetech.com>
Sponsored-by: Sam Grove <sam.grove@sifive.com>
Signed-off-by: Michal Mazurek <maz@semihalf.com>
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
To allow other projects to easily use DPDK as a subproject, add in the
necessary dependency definitions. Slightly different definitions are
necessary for static and shared builds, since for shared builds the
drivers should not be linked in, and the internal meson dependency
objects are more complete.
To use DPDK as a subproject fallback i.e. use installed DPDK if present,
otherwise the shipped one, the following meson statement can be used:
libdpdk = dependency('libdpdk', fallback: ['dpdk', 'dpdk_dep'])
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Ben Magistro <koncept1@gmail.com>
Tested-by: Ben Magistro <koncept1@gmail.com>
This patch replaces instances of zero-sized arrays i.e. those at the end
of structures with "[0]" with the more standard syntax of "[]".
Replacement was done using coccinelle script, with some revert and
cleanup of whitespace afterwards.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Add functions for setting and getting the priority of a thread.
Priorities on multiple platforms are similarly determined by a priority
value and a priority class/policy.
Currently in DPDK most threads operate at the OS-default priority level
but there are cases when increasing the priority is useful. For
example, high performance applications may require elevated priority
levels.
For these reasons, EAL will expose two priority levels which are named
suggestively "normal" and "realtime_critical" and are computed as
follows:
On Linux, the following mapping is created:
RTE_THREAD_PRIORITY_NORMAL corresponds to
* policy SCHED_OTHER
* priority value: (sched_get_priority_min(SCHED_OTHER) +
sched_get_priority_max(SCHED_OTHER))/2;
RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to
* policy SCHED_RR
* priority value: sched_get_priority_max(SCHED_RR);
On Windows, the following mapping is created:
RTE_THREAD_PRIORITY_NORMAL corresponds to
* class NORMAL_PRIORITY_CLASS
* priority THREAD_PRIORITY_NORMAL
RTE_THREAD_PRIORITY_REALTIME_CRITICAL corresponds to
* class REALTIME_PRIORITY_CLASS (when running with privileges)
* class HIGH_PRIORITY_CLASS (when running without privileges)
* priority THREAD_PRIORITY_TIME_CRITICAL
Note that on Linux the resulting priority value will be 0, in
accordance to the documentation that mention the value should be 0 for
SCHED_OTHER policy.
Signed-off-by: Narcisa Vasile <navasile@linux.microsoft.com>
Signed-off-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
A sequence lock (seqlock) is a synchronization primitive which allows
for data-race free, low-overhead, high-frequency reads, suitable for
data structures shared across many cores and which are updated
relatively infrequently.
A seqlock permits multiple parallel readers. A spinlock is used to
serialize writers. In cases where there is only a single writer, or
writer-writer synchronization is done by some external means, the
"raw" sequence counter type (and accompanying rte_seqcount_*()
functions) may be used instead.
To avoid resource reclamation and other issues, the data protected by
a seqlock is best off being self-contained (i.e., no pointers [except
to constant data]).
One way to think about seqlocks is that they provide means to perform
atomic operations on data objects larger than what the native atomic
machine instructions allow for.
DPDK seqlocks (and the underlying sequence counters) are not
preemption safe on the writer side. A thread preemption affects
performance, not correctness.
A seqlock contains a sequence number, which can be thought of as the
generation of the data it protects.
A reader will
1. Load the sequence number (sn).
2. Load, in arbitrary order, the seqlock-protected data.
3. Load the sn again.
4. Check if the first and second sn are equal, and even numbered.
If they are not, discard the loaded data, and restart from 1.
The first three steps need to be ordered using suitable memory fences.
A writer will
1. Take the spinlock, to serialize writer access.
2. Load the sn.
3. Store the original sn + 1 as the new sn.
4. Perform load and stores to the seqlock-protected data.
5. Store the original sn + 2 as the new sn.
6. Release the spinlock.
Proper memory fencing is required to make sure the first sn store, the
data stores, and the second sn store appear to the reader in the
mentioned order.
The sn loads and stores must be atomic, but the data loads and stores
need not be.
The original seqlock design and implementation was done by Stephen
Hemminger. This is an independent implementation, using C11 atomics.
For more information on seqlocks, see
https://en.wikipedia.org/wiki/Seqlock
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
musl lacks __ppc_get_timebase() but has __builtin_ppc_get_timebase()
Signed-off-by: Duncan Bellamy <dunk@denkimushi.com>
Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>
Telemetry commands are now registered through the dmadev library
for the gathering of DSA stats. The corresponding callback
functions for listing dmadevs and providing info and stats for a
specific dmadev are implemented in the dmadev library.
An example usage can be seen below:
Connecting to /var/run/dpdk/rte/dpdk_telemetry.v2
{"version": "DPDK 22.03.0-rc2", "pid": 2956551, "max_output_len": 16384}
Connected to application: "dpdk-dma"
--> /
{"/": ["/", "/dmadev/info", "/dmadev/list", "/dmadev/stats", ...]}
--> /dmadev/list
{"/dmadev/list": [0, 1]}
--> /dmadev/info,0
{"/dmadev/info": {"name": "0000:00:01.0", "nb_vchans": 1, "numa_node": 0,
"max_vchans": 1, "max_desc": 4096, "min_desc": 32, "max_sges": 0,
"capabilities": {"mem2mem": 1, "mem2dev": 0, "dev2mem": 0, ...}}}
--> /dmadev/stats,0,0
{"/dmadev/stats": {"submitted": 0, "completed": 0, "errors": 0}}
Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Conor Walsh <conor.walsh@intel.com>
Tested-by: Sunil Pai G <sunil.pai.g@intel.com>
Tested-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Chengwen Feng <fengchengwen@huawei.com>
Clarify that once an operation has completed, the output of that
operation is visible to all cores.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Vhost backend of different devices have different features.
Add an API to get vDPA device type, net device or blk device
currently, so users can set different features for different
kinds of devices.
Signed-off-by: Andy Pei <andy.pei@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Add support for VHOST_USER_GET_CONFIG and VHOST_USER_SET_CONFIG.
VHOST_USER_GET_CONFIG and VHOST_USER_SET_CONFIG message is only
supported by virtio blk VDPA device.
Signed-off-by: Andy Pei <andy.pei@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Get_config and set_config are necessary ops for blk device.
Add get_config and set_config ops to vDPA ops.
Signed-off-by: Andy Pei <andy.pei@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
In vhost_user_msg_handler(), if vhost message handling
failed, we should check whether the queue is locked and
release the lock before returning. Or, it will cause a
deadlock later.
Fixes: 7f31d4ea05 ("vhost: fix lock on device readiness notification")
Cc: stable@dpdk.org
Signed-off-by: Wenwu Ma <wenwux.ma@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Tested-by: Wei Ling <weix.ling@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
This patch implements asynchronous dequeue data path for vhost split
ring, a new API rte_vhost_async_try_dequeue_burst() is introduced.
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Signed-off-by: Yuan Wang <yuanx.wang@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch refactors copy_desc_to_mbuf() used by the sync
path to support both sync and async descriptor to mbuf filling.
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch refactors vhost async enqueue path and dequeue path to use
the same function async_fill_seg() for preparing batch elements,
which simplifies the code without performance degradation.
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch extracts the descriptors to buffers filling from
copy_desc_to_mbuf() into a dedicated function. Besides, enqueue
and dequeue path are refactored to use the same function
sync_fill_seg() for preparing batch elements, which simplifies
the code without performance degradation.
Signed-off-by: Xuan Ding <xuan.ding@intel.com>
Tested-by: Yvonne Yang <yvonnex.yang@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch adds runtime checks in unsafe Vhost async APIs,
to ensure the access lock is taken.
The detection won't work every time, as another thread
could take the lock, but it would help to detect misuse
of these unsafe API.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Jiayu Hu <jiayu.hu@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
This patch adds statistics for packets in-flight submission
and completion, when Vhost async mode is used.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
This patch adds statistics for IOTLB hits and misses.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
This patch adds a new virtqueue statistic for guest
notifications. It is useful to deduce from hypervisor side
whether the corresponding guest Virtio device is using
Kernel Virtio-net driver or DPDK Virtio PMD.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
This patch introduces new APIs for the application
to query and reset per-virtqueue statistics. The
patch also introduces generic counters.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
vq->async accesses must be protected with vq->access_lock.
Fixes: eb666d2408 ("vhost: fix async unregister deadlock")
Fixes: 0c0935c5f7 ("vhost: allow to check in-flight packets for async vhost")
Cc: stable@dpdk.org
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Sunil Pai G <sunil.pai.g@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The port ownership concept was introduced in ethdev in DPDK 18.02.
Not sure it is used by applications except those using failsafe or netvsc.
It can also be used by libraries or applications to sort out
how ports are controlled.
Hiding sub-ports controlled by failsafe or netvsc look to be enough
justification to promote this API as stable.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
This patch introduces the IPv4/IPv6 ECN modify field support, and
adds the testpmd CLI commands support.
Usage:
modify_field op set dst_type ipv4_ecn src_type ...
For example:
flow create 0 ingress group 1 pattern eth / ipv4 / end actions
modify_field op set dst_type ipv4_ecn src_type value src_value
0x03 width 2 / queue index 0 / end
Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Secondary process needs to close device to release process private
resources. But secondary process should not be obliged to wait for
device stop before closing ethdev.
Fixes: febc855b35 ("ethdev: forbid closing started device")
Cc: stable@dpdk.org
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add support for module EEPROM information format defined in
SFF-8636 Rev 2.7.
Signed-off-by: Robin Zhang <robinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add support for module EEPROM information format defined in
SFF-8472 Rev 12.0
Signed-off-by: Robin Zhang <robinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add support for module EEPROM information format defined in
SFF-8079 Rev 1.7.
Signed-off-by: Robin Zhang <robinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add support for SFF-8024 Rev 4.0 of pluggable I/O configuration
and some common utilities for SFF-8436/8636 and SFF-8472/8079.
Signed-off-by: Robin Zhang <robinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add a new telemetry command /ethdev/module_eeprom to dump the module
EEPROM of each port. The format of module EEPROM information follows
the SFF(Small Form Factor) Committee specifications.
Signed-off-by: Robin Zhang <robinx.zhang@intel.com>
Signed-off-by: Kevin Liu <kevinx.liu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Currently, 'dev_started' is always set to be 0 when dev stop, whether
it succeeded or failed. This is unreasonable and this patch fixed it.
Fixes: 62024eb827 ("ethdev: change stop operation callback to return int")
Cc: stable@dpdk.org
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@xilinx.com>
Whether it is allowed to call Rx/Tx functions for a stopped queue
was undocumented. Some PMDs make this behavior a no-op
either by explicitly checking the queue state
or by the way how their routines are implemented or HW works.
No-op behavior may be convenient for application developers.
But it also means that pollers of stopped queues
would go all the way down to PMD Rx/Tx routines, wasting cycles.
Some PMDs would do a check for the queue state on data path,
even though it may never be needed for a particular application.
Also, use cases for stopping queues or starting them deferred
do not logically require polling stopped queues.
Use case 1: a secondary that was polling the queue has crashed,
the primary is doing a recovery to free all mbufs.
By definition the queue to be restarted is not polled.
Use case 2: deferred queue start or queue reconfiguration.
The polling thread must be synchronized anyway,
because queue start and stop are non-atomic.
Prohibit calling Rx/Tx functions on stopped queues.
Fixes: 0748be2cf9 ("ethdev: queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
The rte_tel_data_alloc() may return NULL, so the caller should add
judgement for it.
Fixes: 083b0b310b ("ethdev: add common stats for telemetry")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
When xstats location is null in rte_eth_xstats_get() the return value
is not clearly specified. Some PMDs (eg. hns3/ipn3ke/mvpp2/axgbe) return
zero while others return the required number of elements.
In this patch, special parameter combinations are restricted:
1. highlight that xstats location may be null if and only if n is 0.
2. amend n parameter description to specify that if n is lower than
the required number of elements, the function returns the required
number of elements.
3. specify that if n is zero, the xstats must be NULL, the function
returns the required number of elements (a duplicate which should
help to not very attentive readers).
Add sanity check for null xstats and non-zero n case on API level to
make it unnecessary to care about it in drivers.
Fixes: ce757f5c9a ("ethdev: new method to retrieve extended statistics")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Currently, meter object supports only DSCP based on input color table,
The patch enhance that to support VLAN based input color table,
color table based on inner field for the tunnel use case, and
support for fallback color per meter if packet based on a different field.
All of the above features are exposed through capability and added
additional capability to specify the implementation supports
more than one input color table per ethdev port.
Suggested-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Add new get/set API to allow the user or application to set the minimum
and maximum frequencies to use when scaling.
Previously, the frequency range was determined by the HW capabilities of
the CPU. With this new API, the user or application can constrain this
if required.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: David Hunt <david.hunt@intel.com>
Add new get/set API for configuring 'pause_duration' which used to adjust
the pause mode callback duration.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Tested-by: David Hunt <david.hunt@intel.com>
Add new get/set APIs to configure emptypoll max which is used to
determine when a queue can go into sleep state.
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Tested-by: David Hunt <david.hunt@intel.com>
Add an implementation of the rte_lpm_lookupx4() function for platforms
without support for vector operations.
This will be useful in the upcoming RISC-V port as well as any platform
which may want to start with a basic level of LPM support.
Signed-off-by: Michal Mazurek <maz@semihalf.com>
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
All other rte_lpm_lookup* functions take lpm argument as a const. As the
basic rte_lpm_lookup() performs the same function, it should also do
that.
Signed-off-by: Stanislaw Kardach <kda@semihalf.com>
Acked-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
- Added salt length and optional label.
Common parameters to PSS and OAEP padding for RSA.
- Changed RSA hash padding fields names.
Now it corresponds to the RSA documents.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Clarified where should output be stored of signature
decryption with padding none.
PMD is not able to know what padding algorithm was used,
therefore decrypted signature should be returned to the user.
- Removed incorrect big-endian constraints.
Not all data in RSA can be treated as big endian integer,
therefore some of the constraints were lifted.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- move RSA padding into separate struct.
More padding members should be added into padding,
therefore having separate struct for padding parameters will
make this more readable.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Clarified usage of RSA padding hash.
It was not specified how to use hash for PKCS1_5
padding. This could lead to incorrect implementation.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Added flags to rte_crypto_asym_op struct.
It may be shared between different algorithms.
- Added Diffie-Hellman padding flags.
Diffie-Hellman padding is used in certain protocols,
in others, leading zero bytes need to be stripped.
Even same protocol may use a different approach - most
glaring example is TLS1.2 - TLS1.3.
For ease of use, and to avoid additional copy
on certain occasions, driver should be able to return both.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Added key exchange public key verify option.
For some elliptic curves public point in DH exchange
needs to be checked, if it lays on the curve.
Modular exponentiation needs certain checks as well,
though mathematically much easier.
This commit adds verify option to asym_op operations.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Added elliptic curve Diffie-Hellman parameters.
Point multiplication allows the user to process every phase of
ECDH, but for phase 1, user should not really care about the generator.
The user does not even need to know what the generator looks like,
therefore setting ec xform would make this work.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Moved DH operation type to DH operation struct.
Operation type (PUBLIC_KEY_GENERATION, SHARED_SECRET) should
be free to choose for any operation. One xform/session should
be enough to perform both DH operations, if op_type would be xform
member, session would have to be created twice for the same
group. Similar problem would be observed in sessionless case.
Additionally, it will help extend DH to support Elliptic Curves.
- Changed order of Diffie-Hellman operation phases.
Now it corresponds with the order of operations.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Clarified usage of private key in Diffie-Hellman.
CSRNG capable device should generate private key and then
use it for public key generation.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
Removed comment that stated DSA can be used with Diffie
Hellman ephemeral key.
DH and DSA integration allowed to use ephemeral keys for
random integer, but not for private keys.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- Separated key exchange enum from asym op type.
Key exchange and asymmetric crypto operations like signatures,
encryption/decryption should not share same operation enum as
its use cases are unrelated and mutually exclusive.
Therefore op_type was separate into:
1) operation type
2) key exchange operation type
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
- EC enum was renamed to rte_crypto_curve_id.
Elliptic curve enum name was incorrectly associated
with a group (it comes from the current tls registry name).
- Clarified comments about TLS deprecation.
Some curves included are deprecated with TLS 1.3.
Comments to address it were added.
- Clarified FFDH groups usage.
Elliptic curves IDs in TLS are placed in the same registry
as FFDH. Cryptodev does not assign specific groups, and
if specific groups would be assigned by DPDK, it cannot be
TLS SupportedGroups registry, as it would conflict with
other protocols like IPSec.
- Added IANA reference.
Only few selected curves are included in previously
referenced rfc8422. IANA reference is added instead.
- Removed UNKNOWN ec group.
There is no default value, and there is no UNKNOWN
elliptic curve.
Signed-off-by: Arek Kusztal <arkadiuszx.kusztal@intel.com>
Acked-by: Fan Zhang <roy.fan.zhang@intel.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
