Commit Graph

158 Commits

Author SHA1 Message Date
Mattias Rönnblom
0bee070907 eal: add seqlock
A sequence lock (seqlock) is a synchronization primitive which allows
for data-race free, low-overhead, high-frequency reads, suitable for
data structures shared across many cores and which are updated
relatively infrequently.

A seqlock permits multiple parallel readers. A spinlock is used to
serialize writers. In cases where there is only a single writer, or
writer-writer synchronization is done by some external means, the
"raw" sequence counter type (and accompanying rte_seqcount_*()
functions) may be used instead.

To avoid resource reclamation and other issues, the data protected by
a seqlock is best off being self-contained (i.e., no pointers [except
to constant data]).

One way to think about seqlocks is that they provide means to perform
atomic operations on data objects larger than what the native atomic
machine instructions allow for.

DPDK seqlocks (and the underlying sequence counters) are not
preemption safe on the writer side. A thread preemption affects
performance, not correctness.

A seqlock contains a sequence number, which can be thought of as the
generation of the data it protects.

A reader will
  1. Load the sequence number (sn).
  2. Load, in arbitrary order, the seqlock-protected data.
  3. Load the sn again.
  4. Check if the first and second sn are equal, and even numbered.
     If they are not, discard the loaded data, and restart from 1.

The first three steps need to be ordered using suitable memory fences.

A writer will
  1. Take the spinlock, to serialize writer access.
  2. Load the sn.
  3. Store the original sn + 1 as the new sn.
  4. Perform load and stores to the seqlock-protected data.
  5. Store the original sn + 2 as the new sn.
  6. Release the spinlock.

Proper memory fencing is required to make sure the first sn store, the
data stores, and the second sn store appear to the reader in the
mentioned order.

The sn loads and stores must be atomic, but the data loads and stores
need not be.

The original seqlock design and implementation was done by Stephen
Hemminger. This is an independent implementation, using C11 atomics.

For more information on seqlocks, see
https://en.wikipedia.org/wiki/Seqlock

Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Ola Liljedahl <ola.liljedahl@arm.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Mattias Rönnblom <mattias.ronnblom@ericsson.com>
2022-06-07 13:33:14 +02:00
Dmitry Kozlyuk
53bb9a073f doc: rewrite shell scripts in Python
Shell used in documentation generation could not run on Windows.
Rewrite scripts in Python.
New scripts use proper path separators and handle paths with spaces.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
2022-06-01 11:53:35 +02:00
Dmitry Kozlyuk
a9d84ead56 doc: fix API index Markdown syntax
API documentation index had spaces between link caption and URL,
which may be unsupported by some Markdown implementations.
That is, "[caption](URL)" is valid but "[caption] (URL)" is not.
The problematic behavior is observed with Doxygen on Windows.
Remove the spaces.
Unfortunately, Markdown syntax is not formally specified.

Fixes: 9bf486e606 ("doc: generate HTML for API with doxygen")

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
2022-06-01 11:53:05 +02:00
Dmitry Kozlyuk
e4c2241419 doc: simplify CSS customization for Doxygen
CSS for API documentation was customized by a shell script
modifying the file that Doxygen produces.
This way CSS code is kept in a script and an extra build step is added.
Move custom style to a plain CSS file.
Use Doxygen capability to attach this extra stylesheet.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
2022-06-01 11:26:11 +02:00
Nipun Gupta
97f9032f5a dma/dpaa2: add driver-specific configuration API
Add additional PMD APIs for DPAA2 QDMA driver for configuring
RBP, Ultra Short format, and Scatter Gather support

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
2022-06-01 08:58:27 +02:00
Nipun Gupta
33584c19dd raw/dpaa2_qdma: remove driver
With DMA devices supported as a separate flavor of devices,
the DPAA2 QDMA driver is moved in the DMA devices.

This change removes the DPAA2 QDMA driver from raw devices.

Signed-off-by: Nipun Gupta <nipun.gupta@nxp.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
2022-06-01 08:58:27 +02:00
Kiran Kumar K
8efa348e81 net/cnxk: support custom SA index
Adding cnxk device driver support to configure custom SA index.
Custom SA index can be configured as part of the session create
as SPI, and later original SPI can be updated using session update.

Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Jerin Jacob <jerinj@marvell.com>
2022-05-07 11:45:01 +02:00
Bruce Richardson
e16b972b1a build: remove deprecated Meson functions
Starting in meson 0.56, the functions meson.source_root() and
meson.build_root() are deprecated and to be replaced by the [more
descriptive] functions: project_source_root()/global_source_root() and
project_build_root()/global_build_root(). Unfortunately, these new
replacement functions were only added in 0.56 release too, so to use
them we would need version checks for old/new functions to remove the
deprecation warnings.

However, the functions "current_build_dir()" and "current_source_dir()"
remain unaffected by all this, so we can bypass the versioning problem,
by saving off these values to "dpdk_source_root" and "dpdk_build_root"
in the top-level meson.build file

Bugzilla ID: 926
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Jerin Jacob <jerinj@marvell.com>
2022-02-02 18:46:53 +01:00
Bruce Richardson
fee17a1d29 doc: remove dependency on findutils on FreeBSD
Standard "find" on BSD does not support the "-printf" so gfind from
findutils package was used to enable full doc builds. We can remove this
extra dependency by using "sed" and "tr" to adjust the output from
regular find instead.

Fixes: 8260f4f98c ("mk: use script to generate examples.dox")
Fixes: 499fe9dfcf ("doc: add dependency on examples for API doxygen")
Fixes: 897e55c8d2 ("doc: fix Doxygen examples build on FreeBSD")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2022-01-12 18:21:03 +01:00
Timothy Redaelli
92eb2c3465 doc: strip build artefacts for examples file list
examples.dox is built inside builddir/doc/api and so doxygen generates
some dir_HASH.html that includes the builddir name and this may prevent
DPDK documentation to be correctly generated in some distributions, for
example CentOS Stream 9 and RHEL9, since the builddir includes the
architecture.

This commit adds builddir/doc/api (the path where examples.dox is
generated) to STRIP_FROM_PATH, so the generated documentation doesn't
change if builddir changes.

Fixes: a6090630f4 ("doc: automate examples file list for API")
Cc: stable@dpdk.org

Signed-off-by: Timothy Redaelli <tredaelli@redhat.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Tested-by: David Marchand <david.marchand@redhat.com>
2021-11-24 15:11:42 +01:00
Bruce Richardson
897e55c8d2 doc: fix Doxygen examples build on FreeBSD
On FreeBSD, "find" does not support the "printf" flag, so we need to
use "gfind" from the "findutils" package.

Fixes: 8260f4f98c ("mk: use script to generate examples.dox")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2021-11-17 11:19:31 +01:00
Elena Agostini
8b8036a66e gpudev: introduce GPU device class library
In heterogeneous computing system, processing is not only in the CPU.
Some tasks can be delegated to devices working in parallel.

The new library gpudev is for dealing with GPGPU computing devices
from a DPDK application running on the CPU.

The infrastructure is prepared to welcome drivers in drivers/gpu/.

Signed-off-by: Elena Agostini <eagostini@nvidia.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-11-08 17:20:52 +01:00
Vladimir Medvedkin
4fd8c4cb0d hash: add new Toeplitz hash implementation
This patch add a new Toeplitz hash implementation using
Galios Fields New Instructions (GFNI).

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
2021-11-04 11:19:10 +01:00
Jie Wang
3a929df1f2 ethdev: support L2TPv2 and PPP procotol
Added flow pattern items and header formats of L2TPv2 and PPP.

Signed-off-by: Wenjun Wu <wenjun1.wu@intel.com>
Signed-off-by: Jie Wang <jie1x.wang@intel.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-10-21 14:15:59 +02:00
Stephen Hemminger
8d23ce8f5e pcapng: add new library for writing pcapng files
This is utility library for writing pcapng format files
used by Wireshark family of utilities. Older tcpdump
also knows how to read (but not write) this format.

See
  https://github.com/pcapng/pcapng/

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
2021-10-22 17:19:07 +02:00
Chengwen Feng
b36970f2e1 dmadev: introduce DMA device library
The 'dmadev' is a generic type of DMA device.

This patch introduce the 'dmadev' device allocation functions.

The infrastructure is prepared to welcome drivers in drivers/dma/

Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Reviewed-by: Conor Walsh <conor.walsh@intel.com>
2021-10-17 20:49:57 +02:00
Bruce Richardson
99a2dd955f lib: remove librte_ prefix from directory names
There is no reason for the DPDK libraries to all have 'librte_' prefix on
the directory names. This prefix makes the directory names longer and also
makes it awkward to add features referring to individual libraries in the
build - should the lib names be specified with or without the prefix.
Therefore, we can just remove the library prefix and use the library's
unique name as the directory name, i.e. 'eal' rather than 'librte_eal'

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2021-04-21 14:04:09 +02:00
Bruce Richardson
8dcb898c65 build: change indentation in infrastructure files
Switch from using tabs to 4 spaces for meson.build indentation, for the
basic infrastructure and tooling files, as well as doc and kernel
directories.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2021-04-21 14:04:09 +02:00
Stephen Hemminger
9667d97c25 pflock: add phase-fair reader writer locks
This is a new type of reader-writer lock that provides better fairness
guarantees which better suited for typical DPDK applications.
A pflock has two ticket pools, one for readers and one
for writers.

Phase-fair reader writer locks ensure that neither reader nor writer will
be starved.
Neither reader or writer are preferred, they execute in alternating
phases.
All operations of the same type (reader or writer) that acquire the lock
are handled in FIFO order.
Write operations are exclusive, and multiple read operations can be run
together (until a write arrives).

A similar implementation is in Concurrency Kit package in FreeBSD.
For more information see:
   "Reader-Writer Synchronization for Shared-Memory Multiprocessor
    Real-Time Systems",
    http://www.cs.unc.edu/~anderson/papers/ecrts09b.pdf

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
2021-04-14 21:59:47 +02:00
Timothy McDaniel
698fa82941 event/dlb: remove driver
Remove event/dlb driver from DPDK code base.
Updated release note's removal section to reflect the same.

Also updated doc/guides/rel_notes/release_20_11.rst to fix the
the missing link issue due to removal of doc/guides/eventdevs/dlb.rst

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
2021-04-12 09:21:30 +02:00
Ed Czeck
6c7f491e7f net/ark: generalize meta data between FPGA and PMD
In this commit we generalize the movement of user-specified
meta data between mbufs and FPGA AXIS tuser fields using
user-defined hook functions.

- Previous use of PMD dynfields are removed
- Remove emptied rte_pmd_ark.h
- Hook function added to ark_user_ext
- Add hook function calls in Rx and Tx paths
- Update guide with example of hook function use

Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
2021-03-22 16:56:27 +01:00
Venkata Suresh Kumar P
e2b8dc5256 port: add file descriptor SWX port
Add the file descriptor input/output port type for the SWX pipeline.
File descriptor port type provides interface with the kernel network
stack. Example file descriptor port is TAP device.

Signed-off-by: Venkata Suresh Kumar P <venkata.suresh.kumar.p@intel.com>
Signed-off-by: Churchill Khangar <churchill.khangar@intel.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2021-03-23 19:50:44 +01:00
Cristian Dumitrescu
66440b7b22 table: add wildcard match table type
Add the widlcard match/ACL table type for the SWX pipeline, which is
used under the hood by the table instruction.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
Signed-off-by: Churchill Khangar <churchill.khangar@intel.com>
2021-03-23 19:47:20 +01:00
Cristian Dumitrescu
77a413017c port: add ring SWX port
Add the ring input/output port type for the SWX pipeline.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2021-03-23 17:22:47 +01:00
Thomas Monjalon
e0473c6d5b eal: fix build with musl
In musl libc, cpu_set_t is defined only if _GNU_SOURCE is defined.
In case _GNU_SOURCE is undefined, as in eal_common_errno.c,
it was not possible to include rte_os.h which uses cpu_set_t.

This limitation is removed: if CPU_SETSIZE is not defined,
cpu_set_t related definitions and functions are skipped.
Note: such definitions are unneeded in eal_common_errno.c.

Applications which do not define _GNU_SOURCE may miss cpu_set_t related
features on musl. Such case is detected by RTE_HAS_CPUSET being undefined,
so functions which depend on rte_cpuset_t will be unavailable.

A missing include of fcntl.h is also added.

Bugzilla ID: 35
Fixes: 11b57c6980 ("eal: fix error string function")
Fixes: 176bb37ca6 ("eal: introduce internal wrappers for file operations")
Cc: stable@dpdk.org

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Natanael Copa <ncopa@alpinelinux.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: David Marchand <david.marchand@redhat.com>
2021-03-23 08:41:05 +01:00
Anatoly Burakov
cb21730561 doc: update power management in doxygen API index
The headers rte_power_intrinsics.h and rte_power_pmd_mgmt.h
were missing from the doxygen API index.

Fixes: cda57d9388 ("eal: add power management intrinsics")
Fixes: 682a645438 ("power: add ethdev power management")
Cc: stable@dpdk.org

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
2021-03-17 12:19:23 +01:00
Wei Huang
a05bd1b40b raw/ifpga: add FPGA RSU APIs
RSU (Remote System Update) depends on secure manager which may be
different on various implementations, so a new secure manager device
is implemented for adapting such difference.
There are five APIs added:
1. rte_pmd_ifpga_get_dev_id() get raw device ID of ifpga device from PCI
   address like 'Domain:Bus:Dev.Func'.
2. rte_pmd_ifpga_update_flash() update flash with specific image file.
3. rte_pmd_ifpga_stop_update() abort flash update process.
4. rte_pmd_ifpga_reboot_try() check current ifpga status and change it
   to reboot status if it is idle.
5. rte_pmd_ifpga_reload() trigger full reconfiguration of ifpga device.

Signed-off-by: Wei Huang <wei.huang@intel.com>
Acked-by: Tianfei Zhang <tianfei.zhang@intel.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
2021-03-05 09:55:55 +01:00
Olivier Matz
44d00a1d12 doc: add missing network layers in API index
Add missing files in doxy-api-index.md and add a short description
for files that hadn't one.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
2020-11-27 01:51:27 +01:00
Jeff Guo
12b435bf8f net/iavf: support flex desc metadata extraction
Enable metadata extraction for flexible descriptors in AVF, that would
allow network function directly get metadata without additional parsing
which would reduce the CPU cost for VFs. The enabling metadata
extractions involve the metadata of VLAN/IPv4/IPv6/IPv6-FLOW/TCP/MPLS
flexible descriptors, and the VF could negotiate the capability of
the flexible descriptor with PF and correspondingly configure the
specific offload at receiving queues.

Signed-off-by: Jeff Guo <jia.guo@intel.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>
2020-11-03 23:35:06 +01:00
Bing Zhao
a0dd7cc726 doc: add mlx5 header to API index
The file "rte_pmd_mlx5.h" is used to provide mlx5 PMD specific APIs
and it needs to be included in the document generation.

Signed-off-by: Bing Zhao <bingz@nvidia.com>
Reviewed-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ori Kam <orika@nvidia.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-11-03 23:35:06 +01:00
Timothy McDaniel
b287267d62 event/dlb: add token pop API
The PMD uses a public interface to allow applications to
control the token pop mode. Supported token pop modes are
as follows, and they impact core scheduling affinity for
ldb ports.

AUTO_POP: Pop the CQ tokens immediately after dequeueing.
DELAYED_POP: Pop CQ tokens after (dequeue_depth - 1) events
	     are released. Supported on load-balanced ports
	     only.
DEFERRED_POP: Pop the CQ tokens during next dequeue operation.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
2020-11-02 14:46:01 +01:00
Timothy McDaniel
c667583d82 event/dlb2: add token pop API
The PMD uses a public interface to allow applications to
control the token pop mode. Supported token pop modes are
as follows, and they impact core scheduling affinity for
ldb ports.

AUTO_POP: Pop the CQ tokens immediately after dequeueing.
DELAYED_POP: Pop CQ tokens after (dequeue_depth - 1) events
             are released. Supported on load-balanced ports
             only.
DEFERRED_POP: Pop the CQ tokens during next dequeue operation.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
2020-11-02 09:40:22 +01:00
Ed Czeck
1abc7209bb net/ark: switch user data to dynamic mbuf fields
The second field of metadata is reserved for user data
which was using a deprecated mbuf field.
It is moved to dynamic fields in order to allow removal of udata64.

The use of meta data must be enabled with a compile-time flag
RTE_PMD_ARK_{TX,RX}_USERDATA_ENABLE.
User data on Tx and Rx paths can be defined and used separately.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Ed Czeck <ed.czeck@atomicrules.com>
2020-10-31 16:13:11 +01:00
Bruce Richardson
b28a44fd62 doc: add ioat driver to API index
Add the ioat driver to the doxygen documentation.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Radu Nicolau <radu.nicolau@intel.com>
2020-10-08 14:33:20 +02:00
Bruce Richardson
10044a8e01 doc: add doxygen index file to dependencies
The doxygen index file is not printed as a processed file by doxygen so it
does not appear in the output .d (dependency file) list automatically
generated. Therefore, for correct rebuild tracking, we need to explicitly
include it as a dependency of the doxygen job.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2020-10-06 00:27:04 +02:00
Cristian Dumitrescu
d0a0096661 table: add exact match SWX table
Add the exact match table type for the SWX pipeline. Used under the
hood by the SWX pipeline table instruction.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:10 +02:00
Cristian Dumitrescu
394813eb60 port: add source and sink SWX ports
Add the PCAP file-based source (input) and sink (output) port types
for the SWX pipeline. The sink port is typically used to implement the
packet drop pipeline action. Used under the hood by the pipeline rx
and tx instructions.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:10 +02:00
Cristian Dumitrescu
ecc5f43396 port: add ethernet device SWX port
Add the Ethernet device input/output port type for the SWX pipeline.
Used under the hood by the pipeline rx and tx instructions.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:08 +02:00
Cristian Dumitrescu
e9d870dd93 pipeline: add SWX pipeline tables
Add tables to the SWX pipeline. The match fields are flexibly selected
from the headers and meta-data. The set of table actions is flexibly
selected for each table from the set of pipeline actions.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:07 +02:00
Cristian Dumitrescu
1e4c88caea pipeline: add SWX extern objects and funcs
Add extern objects and functions to plug into the SWX pipeline any
functionality that cannot be efficiently implemented with existing
instructions, e.g. special checksum/ECC, crypto, meters, stats arrays,
heuristics, etc. In/out arguments are passed through mailbox with
format defined by struct.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:07 +02:00
Cristian Dumitrescu
6e0ca01c93 pipeline: add SWX pipeline input port
Add input ports to the newly introduced SWX pipeline type. Each port
instantiates a port type that defines the port operations, e.g. ethdev
port, PCAP port, etc. The RX interface is single packet, with packet
batching internally for performance.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:07 +02:00
Cristian Dumitrescu
56492fd536 pipeline: add new SWX pipeline type
Add new improved Software Switch (SWX) pipeline type that supports
dynamically-defined packet headers, meta-data, actions and pipelines.
Actions and pipelines are defined through instructions.

Signed-off-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
2020-10-01 18:43:06 +02:00
Bruce Richardson
1f16199945 doc: make doxygen comply with meson werror option
When the --werror meson build option is set, we can set the WARN_AS_ERRORS
doxygen option in the doxygen config flag to get the same behaviour for API
doc building as for building the rest of DPDK. This can help catch
documentation errors sooner in the development process.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2020-09-30 16:29:38 +02:00
Bruce Richardson
2a66011df4 doc: put doxygen log file in build directory
The meson documentation states that projects should not rely upon the
custom_target build commands are run from any given directory.  Therefore,
rather than writing the standout output from doxygen to the current
directory - which could be anywhere in future, put it into the api
directory, so that it is in a known location.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2020-09-30 15:33:32 +02:00
Bruce Richardson
b1fa45491f doc: align doxygen output folder with sphinx guides
The API docs were output to "<build>/doc/api/api" folder, which was
ugly-looking with the repeated "api", and inconsistent with the sphinx
guides which were written to "<build>/doc/guides/html". Changing the
doxygen output folder to "html" fixes both these issues.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2020-09-30 15:19:52 +02:00
Bruce Richardson
606fb48f3d doc: hide verbose doxygen standard output
The standard output of doxygen is very verbose, and since ninja mixes
stdout and stderr together it makes it difficult to see any warnings from
the doxygen run. Therefore, we can just log the standard output to file,
and only output the stderr to make warnings clear.

Suggested-by: Thomas Monjalon <thomas@monjalon.net>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2020-09-30 15:19:09 +02:00
Vladimir Medvedkin
0b459fa02a doc: add RIB and FIB into the API index
Add RIB/FIB library into the API doxygen index.
Move LPM/LPM6 under separate section "routing".

Fixes: 5a5793a5ff ("rib: add RIB library")
Fixes: f7e861e21c ("rib: support IPv6")
Fixes: 39e9272484 ("fib: add FIB library")
Fixes: 40d41a8a7b ("fib: support IPv6")
Cc: stable@dpdk.org

Signed-off-by: Vladimir Medvedkin <vladimir.medvedkin@intel.com>
2020-07-11 00:45:20 +02:00
Jerin Jacob
bab9497ef7 regexdev: introduce API
As RegEx usage become more used by DPDK applications, for example:
* Next Generation Firewalls (NGFW)
* Deep Packet and Flow Inspection (DPI)
* Intrusion Prevention Systems (IPS)
* DDoS Mitigation
* Network Monitoring
* Data Loss Prevention (DLP)
* Smart NICs
* Grammar based content processing
* URL, spam and adware filtering
* Advanced auditing and policing of user/application security policies
* Financial data mining - parsing of streamed financial feeds
* Application recognition.
* Dmemory introspection.
* Natural Language Processing (NLP)
* Sentiment Analysis.
* Big data database acceleration.
* Computational storage.

Number of PMD providers started to work on HW implementation,
along side with SW implementations.

This lib adds the support for those kind of devices.

The RegEx Device API is composed of two parts:
- The application-oriented RegEx API that includes functions to setup
  a RegEx device (configure it, setup its queue pairs and start it),
  update the rule database and so on.

- The driver-oriented RegEx API that exports a function allowing
  a RegEx poll Mode Driver (PMD) to simultaneously register itself as
  a RegEx device driver.

RegEx device components and definitions:

    +-----------------+
    |                 |
    |                 o---------+    rte_regexdev_[en|de]queue_burst()
    |   PCRE based    o------+  |               |
    |  RegEx pattern  |      |  |  +--------+   |
    | matching engine o------+--+--o        |   |    +------+
    |                 |      |  |  | queue  |<==o===>|Core 0|
    |                 o----+ |  |  | pair 0 |        |      |
    |                 |    | |  |  +--------+        +------+
    +-----------------+    | |  |
           ^               | |  |  +--------+
           |               | |  |  |        |        +------+
           |               | +--+--o queue  |<======>|Core 1|
       Rule|Database       |    |  | pair 1 |        |      |
    +------+----------+    |    |  +--------+        +------+
    |     Group 0     |    |    |
    | +-------------+ |    |    |  +--------+        +------+
    | | Rules 0..n  | |    |    |  |        |        |Core 2|
    | +-------------+ |    |    +--o queue  |<======>|      |
    |     Group 1     |    |       | pair 2 |        +------+
    | +-------------+ |    |       +--------+
    | | Rules 0..n  | |    |
    | +-------------+ |    |       +--------+
    |     Group 2     |    |       |        |        +------+
    | +-------------+ |    |       | queue  |<======>|Core n|
    | | Rules 0..n  | |    +-------o pair n |        |      |
    | +-------------+ |            +--------+        +------+
    |     Group n     |
    | +-------------+ |<-------rte_regexdev_rule_db_update()
    | |             | |<-------rte_regexdev_rule_db_compile_activate()
    | | Rules 0..n  | |<-------rte_regexdev_rule_db_import()
    | +-------------+ |------->rte_regexdev_rule_db_export()
    +-----------------+

RegEx: A regular expression is a concise and flexible means for matching
strings of text, such as particular characters, words, or patterns of
characters. A common abbreviation for this is â~@~\RegExâ~@~].

RegEx device: A hardware or software-based implementation of RegEx
device API for PCRE based pattern matching syntax and semantics.

PCRE RegEx syntax and semantics specification:
http://regexkit.sourceforge.net/Documentation/pcre/pcrepattern.html

RegEx queue pair: Each RegEx device should have one or more queue pair to
transmit a burst of pattern matching request and receive a burst of
receive the pattern matching response. The pattern matching
request/response embedded in *rte_regex_ops* structure.

Rule: A pattern matching rule expressed in PCRE RegEx syntax along with
Match ID and Group ID to identify the rule upon the match.

Rule database: The RegEx device accepts regular expressions and converts
them into a compiled rule database that can then be used to scan data.
Compilation allows the device to analyze the given pattern(s) and
pre-determine how to scan for these patterns in an optimized fashion that
would be far too expensive to compute at run-time. A rule database
contains a set of rules that compiled in device specific binary form.

Match ID or Rule ID: A unique identifier provided at the time of rule
creation for the application to identify the rule upon match.

Group ID: Group of rules can be grouped under one group ID to enable
rule isolation and effective pattern matching. A unique group identifier
provided at the time of rule creation for the application to identify
the rule upon match.

Scan: A pattern matching request through *enqueue* API.

It may possible that a given RegEx device may not support all the
features
of PCRE. The application may probe unsupported features through
struct rte_regexdev_info::pcre_unsup_flags

By default, all the functions of the RegEx Device API exported by a PMD
are lock-free functions which assume to not be invoked in parallel on
different logical cores to work on the same target object. For instance,
the dequeue function of a PMD cannot be invoked in parallel on two logical
cores to operates on same RegEx queue pair. Of course, this function
can be invoked in parallel by different logical core on different queue
pair. It is the responsibility of the upper level application to
enforce this rule.

In all functions of the RegEx API, the RegEx device is
designated by an integer >= 0 named the device identifier *dev_id*

At the RegEx driver level, RegEx devices are represented by a generic
data structure of type *rte_regexdev*.
RegEx devices are dynamically registered during the PCI/SoC device
probing phase performed at EAL initialization time.
When a RegEx device is being probed, a *rte_regexdev* structure and
a new device identifier are allocated for that device. Then, the
regexdev_init() function supplied by the RegEx driver matching the
probed device is invoked to properly initialize the device.

The role of the device init function consists of resetting the hardware
or software RegEx driver implementations.

If the device init operation is successful, the correspondence between
the device identifier assigned to the new device and its associated
*rte_regexdev* structure is effectively registered.
Otherwise, both the *rte_regexdev* structure and the device identifier
are freed.

The functions exported by the application RegEx API to setup a device
designated by its device identifier must be invoked in the following
order:
    - rte_regexdev_configure()
    - rte_regexdev_queue_pair_setup()
    - rte_regexdev_start()

Then, the application can invoke, in any order, the functions
exported by the RegEx API to enqueue pattern matching job, dequeue
pattern matching response, get the stats, update the rule database,
get/set device attributes and so on

If the application wants to change the configuration (i.e. call
rte_regexdev_configure() or rte_regexdev_queue_pair_setup()), it must
call rte_regexdev_stop() first to stop the device and then do the
reconfiguration before calling rte_regexdev_start() again. The enqueue and
dequeue functions should not be invoked when the device is stopped.

Finally, an application can close a RegEx device by invoking the
rte_regexdev_close() function.

Each function of the application RegEx API invokes a specific function
of the PMD that controls the target device designated by its device
identifier.

For this purpose, all device-specific functions of a RegEx driver are
supplied through a set of pointers contained in a generic structure of
type *regexdev_ops*.
The address of the *regexdev_ops* structure is stored in the
*rte_regexdev* structure by the device init function of the RegEx driver,
which is invoked during the PCI/SoC device probing phase, as explained
earlier.

In other words, each function of the RegEx API simply retrieves the
*rte_regexdev* structure associated with the device identifier and
performs an indirect invocation of the corresponding driver function
supplied in the *regexdev_ops* structure of the *rte_regexdev*
structure.

For performance reasons, the address of the fast-path functions of the
RegEx driver is not contained in the *regexdev_ops* structure.
Instead, they are directly stored at the beginning of the *rte_regexdev*
structure to avoid an extra indirect memory access during their
invocation.

RTE RegEx device drivers do not use interrupts for enqueue or dequeue
operation. Instead, RegEx drivers export Poll-Mode enqueue and dequeue
functions to applications.

The *enqueue* operation submits a burst of RegEx pattern matching
request to the RegEx device and the *dequeue* operation gets a burst of
pattern matching response for the ones submitted through *enqueue*
operation.

Typical application utilisation of the RegEx device API will follow the
following programming flow.

- rte_regexdev_configure()
- rte_regexdev_queue_pair_setup()
- rte_regexdev_rule_db_update() Needs to invoke if precompiled rule
  database not
  provided in rte_regexdev_config::rule_db for rte_regexdev_configure()
  and/or application needs to update rule database.
- rte_regexdev_rule_db_compile_activate() Needs to invoke if
  rte_regexdev_rule_db_update function was used.
- Create or reuse exiting mempool for *rte_regex_ops* objects.
- rte_regexdev_start()
- rte_regexdev_enqueue_burst()
- rte_regexdev_dequeue_burst()

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Ori Kam <orika@mellanox.com>
2020-07-07 00:24:38 +02:00
Joyce Kong
7f3aa08639 eal: introduce bit operations API
Bitwise operation APIs are defined and used in a lot of PMDs,
which caused a huge code duplication. To reduce duplication,
this patch consolidates them into a common API family.

Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Gavin Hu <gavin.hu@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
2020-06-16 14:16:56 +02:00
Thomas Monjalon
60814f955c doc: fix API index
With Doxygen 1.8.18, a warning appears when tagging
the main markdown header with {#index}.
That's why the tag has been removed from the API index in DPDK 20.05.
Unfortunately it makes the index page classified as a standard
"related page" instead of being the "main page".

The tag {#mainpage} could be used instead of {#index}.
Another solution, chosen here, is to specify the main page file
in the Doxygen configuration with the variable USE_MDFILE_AS_MAINPAGE.

Fixes: 76fb8fc486 ("doc: fix build with doxygen 1.8.18")
Cc: stable@dpdk.org

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Tested-by: David Marchand <david.marchand@redhat.com>
2020-05-28 11:10:41 +02:00