Commit Graph

683 Commits

Author SHA1 Message Date
Nick Connolly
ecc69f98c7 build: disable Windows warnings for insecure funtions
Microsoft CRT defines Windows-specific secure alternatives to
standard library functions and triggers warnings when "insecure"
functions are used [1]. However, calling code already has all
necessary checks around those functions, so these warnings are not
useful for DPDK. MinGW provides its own CRT without this issue.

[1]:
https://docs.microsoft.com/en-us/cpp/c-runtime-library/security-features-in-the-crt?view=msvc-160

Disable this by defining -D_CRT_SECURE_NO_WARNINGS.

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Acked-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Pallavi Kadam <pallavi.kadam@intel.com>
2020-12-07 21:34:04 +01:00
Conor Walsh
654e893436 build: fix gcc warning requiring Wformat
On some CentOS/RHEL systems using gcc 8.3.1 to compile dpdk, gcc shows a
warning on every build step saying that -Wformat-nonliteral and
-Wformat-security warnings will be ignored unless -Wformat is
also specified as a compiler flag. When the build is run with -werror
the build will fail due to these warnings.

Exact warning returned:
cc1: error: -Wformat-nonliteral ignored without -Wformat
[-Werror=format-nonliteral]
cc1: error: -Wformat-security ignored without -Wformat
[-Werror=format-security]
cc1: all warnings being treated as errors

This patch adds the -Wformat flag to config/meson.build. The warning id
181 has also been suppressed in icc as icc was showing false positives
with -Wformat enabled.

Fixes: 524a0d5d66 ("build: enable extra warnings with meson")
Cc: stable@dpdk.org

Signed-off-by: Conor Walsh <conor.walsh@intel.com>
Tested-by: Lingli Chen <linglix.chen@intel.com>
2020-11-22 19:38:25 +01:00
Maxime Coquelin
048a2aa3f3 config: enable packet prefetching with Meson
With Make build system, RTE_PMD_PACKET_PREFETCH was enabled
by default. It got lost when transitioning to Meson build
system.

In order to avoid performance changes, this patch enables
packet prefetching in rte_config.h.

Fixes: 9314afb68a ("drivers: add infrastructure for meson build")
Cc: stable@dpdk.org

Reported-by: Marvin Liu <yong.liu@intel.com>
Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-11-14 10:56:30 +01:00
Nick Connolly
5c7d869487 build: fix install on Windows
Don't run symlink-drivers-solibs.sh as part of 'install' because
Windows doesn't support shell scripts.

Fixes: 82ba4416dd ("build: add module definition files for Windows")
Cc: stable@dpdk.org

Signed-off-by: Nick Connolly <nick.connolly@mayadata.io>
Tested-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
2020-11-13 15:13:10 +01:00
Andrew Rybchenko
141d287067 net/sfc: support aarch64 architecture
Enable the PMD build on aarch64.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Andy Moreton <amoreton@xilinx.com>
2020-11-03 23:35:06 +01:00
Vimal Chungath
c514018321 config/arm: add Graviton2
Add meson build configuration for Graviton2 platform
with 64-bit Arm Neoverse N1 cores. This patch makes the
following changes to generic Neoverse N1 config:

1. increase lcore limit to 64
2. increase memory support to 1TB
3. remove +crc from -march as that is default when setting armv8.2

For more information about Graviton2 platform, refer to:
https://aws.amazon.com/ec2/graviton/

Signed-off-by: Vimal Chungath <vcchunga@amazon.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
2020-11-03 21:01:02 +01:00
Timothy McDaniel
218be03459 event/dlb: add documentation and build infrastructure
Note that config/rte_config.h contains several configuration
switches, providing for fine control of the PMD's
runtime behaviour.

The meson infrastructure is expanded as additional files are
added to this patchset.

Adds announcement of availability of the new driver
for Intel Dynamic Load Balancer 1.0 hardware.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
2020-11-02 14:46:01 +01:00
Timothy McDaniel
166378a794 event/dlb2: add documentation and build infrastructure
Adds the meson build infrastructure, which includes
compile-time constants in rte_config.h. DLB2 is
only supported on Linux 64 bit X86 platforms at this time.

Adds announcement of availability for the new driver
for Intel Dynamic Load Balancer 2.0 hardware.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
2020-11-02 06:46:12 +01:00
Timothy McDaniel
ecd8749541 eventdev: increase max queues configuration
DLB supports a total of 256 queues, 128 load balanced queues
and 128 directed queues. Increase RTE_EVENT_MAX_QUEUES_PER_DEV
to max possible uint_8_t max value.

Signed-off-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
2020-11-01 10:35:06 +01:00
Bruce Richardson
c46df4c6ec config: fix kni build
When building kernel modules such as kni, the "config" directory is not
passed as a standard path in the EXTRA_CFLAGS value, meaning that the
rte_compatibility_defines.h is not found from rte_config.h. However, since
both headers are in the same directory, we can just use quotes rather than
angle-brackets to ensure the second header is always found if the first is.

Fixes: b0b672aead ("build: add defines for compatibility with make build")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
2020-10-20 12:37:05 +02:00
Bruce Richardson
a8d0d473a0 build: replace use of old build macros
Use the newer macros defined by meson in all DPDK source code, to ensure
there are no errors when the old non-standard macros are removed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2020-10-19 22:15:44 +02:00
Bruce Richardson
b0b672aead build: add defines for compatibility with make build
The defines used to indicate what crypto, compression and eventdev drivers
were being built were different to those used in the make build, with meson
defining them with "_PMD" at the end, while make defined them with "_PMD"
in the middle and the specific driver name at the end. This might cause
compatibility issues for applications which used the older defines, which
switching to build against new DPDK releases.

As well as changing the default to match that of make, meson also
special-cases the crypto/compression/event drivers to have both defines
provided. This ensures compatibility for these macros with both meson and
make from older versions.

For a selection of other libraries and drivers, there were other
incompatibilities between the meson and make-defined macros which were not
previously highlighted in a deprecation notice, so we add per-macro
compatibility defines for these to ease the transition from make to meson.

Fixes: 5b9656b157 ("lib: build with meson")
Fixes: 9314afb68a ("drivers: add infrastructure for meson build")
Fixes: dcadbbde8e ("crypto/null: build with meson")
Fixes: 3c32e89f68 ("compress/isal: add skeleton ISA-L compression PMD")
Fixes: eca504f318 ("drivers/event: build skeleton and SW drivers with meson")

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
2020-10-19 22:12:28 +02:00
Ferruh Yigit
a72cb3e765 doc: announce queue stats moving to xstats
Queue stats will be removed from basic stats to xstats.

It will be PMDs responsibility to fill queue stats based on number of
queues they have.

Until all PMDs implement the xstats, a temporary
'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' device flag created. PMDs switched
to the xstats should clear this flag to bypass the ethdev layer autofill
for queue stats.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
2020-10-16 23:27:15 +02:00
Dmitry Kozlyuk
841dfdd06d cmdline: support Windows
Implement terminal handling, input polling, and vdprintf() for Windows.

Because Windows I/O model differs fundamentally from Unix and there is
no concept of character device, polling is simulated depending on the
underlying input device. Supporting non-terminal input is useful for
automated testing.

Windows emulation of VT100 uses "ESC [ E" for newline instead of
standard "ESC E", so add a workaround.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
2020-10-15 00:39:10 +02:00
Konstantin Ananyev
7c6cca6b60 acl: add infrastructure for AVX512 classify methods
Add necessary changes to support new AVX512 specific ACL classify
algorithm:
 - changes in meson.build to check that build tools
   (compiler, assembler, etc.) do properly support AVX512.
 - run-time checks to make sure target platform does support AVX512.
 - dummy rte_acl_classify_avx512() for targets where AVX512
   implementation couldn't be properly supported.

Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2020-10-14 14:23:00 +02:00
Mairtin o Loingsigh
17a937baed net: add CRC AVX512 implementation
This patch enables the optimized calculation of CRC32-Ethernet and
CRC16-CCITT using the AVX512 and VPCLMULQDQ instruction sets. This CRC
implementation is built if the compiler supports the required instruction
sets. It is selected at run-time if the host CPU, again, supports the
required instruction sets.

Signed-off-by: Mairtin o Loingsigh <mairtin.oloingsigh@intel.com>
Signed-off-by: David Coyle <david.coyle@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Reviewed-by: Jasvinder Singh <jasvinder.singh@intel.com>
Reviewed-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2020-10-13 19:26:15 +02:00
Ruifeng Wang
e9b9739264 config: remap flags used for Arm platforms
RTE_ARCH_xx flags are used to distinguish platform architectures.
These flags can be used to pick different code paths for different
architectures at compile time.
For Arm platforms, there are 3 flags in use: RTE_ARCH_ARM,
RTE_ARCH_ARMv7 and RTE_ARCH_ARM64.
RTE_ARCH_ARM64 is for 64-bit aarch64 platforms,
and RTE_ARCH_ARM & RTE_ARCH_ARMv7 are for 32-bit platforms.
RTE_ARCH_ARMv7 is for ARMv7 platforms as its name suggested.

The issue is meaning of RTE_ARCH_ARM is not clear enough.
Because no info about platform word length is included in the name.
To make the flag names more clear, a naming scheme is proposed.

RTE_ARCH_ARM (all Arm platforms)
    |
    +----RTE_ARCH_32 (New. 32-bit platforms of all architectures)
    |        |
    |        +----RTE_ARCH_ARMv7 (ARMv7 platforms)
    |        |
    |        +----RTE_ARCH_ARMv8_AARCH32 (aarch32 state on aarch64 machine)
    |
    +----RTE_ARCH_64 (64-bit platforms of all architectures)
             |
             +----RTE_ARCH_ARM64 (64-bit Arm platforms)

RTE_ARCH_32 will be explicitly defined for 32-bit platforms.

To fit into the new naming scheme, current usage of RTE_ARCH_ARM in
project is mapped to (RTE_ARCH_ARM && RTE_ARCH_32).

Matching flags for other architectures are:
RTE_ARCH_X86
    |
    +----RTE_ARCH_32
    |        |
    |        +----RTE_ARCH_I686
    |        |
    |        +----RTE_ARCH_X86_X32
    |
    +----RTE_ARCH_64
             |
             +----RTE_ARCH_X86_64

RTE_ARCH_PPC_64 ---- RTE_ARCH_64

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Phil Yang <phil.yang@arm.com>
2020-10-13 16:35:48 +02:00
Juraj Linkeš
36029fd6b1 config: add -moutline-atomics to default Arm build
-moutline-atomics allows LSE instructions to be used if available when
compiling for ARMv8.0 instruction set. It's enabled by default on newer
compilers, such as gcc-10.1. Enable the option in case an earlier
compiler version is used for the default build that lacks either -mcpu
or -mtune which would otherwise enable it.

Signed-off-by: Juraj Linkeš <juraj.linkes@pantheon.tech>
2020-10-13 16:35:48 +02:00
Bruce Richardson
cce5aecf3c build: skip detecting libpcap via pcap-config
When compiling for a slightly different architecture, e.g. 32-bit on 64-bit
systems using CFLAGS rather than a cross-file, the pcap-config utility can
often return parameters that are unusable for the build in question, i.e.
providing the native 64-bit library paths rather than checking for 32-bit
equivalent.

Since many distros now include a version of libpcap with a
pkg-config file, and for those that don't find-library should work ok as a
fallback, we can explicitly just use pkg-config in the dependency search,
causing meson to skip trying to use pcap-config.

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Luca Boccassi <bluca@debian.org>
Tested-by: David Marchand <david.marchand@redhat.com>
2020-10-12 16:59:23 +02:00
Wei Hu (Xavier)
76d794566d net/hns3: maximize queue number
The maximum number of queues for hns3 PF and VF driver is 64 based on
hns3 network engine with revision_id equals 0x21. Based on hns3 network
engine with revision_id equals 0x30, the hns3 PF PMD driver can support
up to 1280 queues, and hns3 VF PMD driver can support up to 128 queues.

The following points need to be modified to support maximizing queue
number and maintain better compatibility:
1) Maximizing the number of queues for hns3 PF and VF PMD driver In
   current version, VF is not supported when PF is driven by hns3 PMD
   driver. If maximum queue numbers allocated to PF PMD driver is less
   than total tqps_num allocated to this port, all remaining number of
   queues are mapped to VF function, which is unreasonable. So we fix
   that all remaining number of queues are mapped to PF function.

   Using RTE_LIBRTE_HNS3_MAX_TQP_NUM_PER_PF which comes from
   configuration file to limit the queue number allocated to PF device
   based on hns3 network engine with revision_id greater than 0x30. And
   PF device still keep the maximum 64 queues based on hns3 network
   engine with revision_id equals 0x21.

   Remove restriction of the macro HNS3_MAX_TQP_NUM_PER_FUNC on the
   maximum number of queues in hns3 VF PMD driver and use the value
   allocated by hns3 PF kernel netdev driver.

2) According to the queue number allocated to PF device, a variable
   array for Rx and Tx queue is dynamically allocated to record the
   statistics of Rx and Tx queues during the .dev_init ops
   implementation function.
3) Add an extended field in hns3_pf_res_cmd to support the case that
   numbers of queue are greater than 1024.
4) Use new base address of Rx or Tx queue if QUEUE_ID of Rx or Tx queue
   is greater than 1024.
5) Remove queue id mask and use all bits of actual queue_id as the
   queue_id to configure hardware.
6) Currently, 0~9 bits of qset_id in hns3_nq_to_qs_link_cmd used to
   record actual qset id and 10 bit as VLD bit are configured to
   hardware. So we also need to use 11~15 bits when actual qset_id is
   greater than 1024.
7) The number of queue sets based on different network engine are
   different. We use it to calculate group number and configure to
   hardware in the backpressure configuration.
8) Adding check operations for number of Rx and Tx queue user configured
   when mapping queue to tc Rx queue numbers under a single TC must be
   less than rss_size_max supported by a single TC. Rx and Tx queue
   numbers are allocated to every TC by average. So Rx and Tx queue
   numbers must be an integer multiple of 2, or redundant queues are not
   available.
9) We can specify which packets enter the queue with a specific queue
   number, when creating flow table rules by rte_flow API. Currently,
   driver uses 0~9 bits to record the queue_id. So it is necessary to
   extend one bit field to record queue_id and configure to hardware, if
   the queue_id is greater than 1024.

Signed-off-by: Huisong Li <lihuisong@huawei.com>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei@huawei.com>
2020-10-08 19:58:10 +02:00
Louise Kilheeney
3f6f83626c support python 3 only
Changed scripts to explicitly use Python 3 only, to avoid
maintaining Python 2.
Removed deprecation notices.

Signed-off-by: Louise Kilheeney <louise.kilheeney@intel.com>
Signed-off-by: Kevin Laatz <kevin.laatz@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Robin Jarry <robin.jarry@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
2020-10-02 13:51:00 +02:00
Radu Nicolau
84fb33fec1 build: remove deprecated cpuflag macros
Replace use of RTE_MACHINE_CPUFLAG macros with regular compiler
macros, which are more complete than those provided by DPDK, and as such
it allows new instruction sets to be leveraged without having to do
extra work to set them up in DPDK.

Signed-off-by: Sean Morrissey <sean.morrissey@intel.com>
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
2020-09-25 11:13:57 +02:00
Christian Ehrhardt
e356aa46ad avoid libfdt checks adding full paths to pkg-config
The checks for libfdt try dependency() first which would only work if
a pkg-config would be present but libfdt has none.
Then it probes for the lib path itself via cc.find_library.

But later it adds the result of either probe to ext_deps which ends up
in build and also the resulting pkg-config to contain toolchain versioned
paths in Libs.private like:
  /usr/lib/gcc/x86_64-linux-gnu/9/../../../x86_64-linux-gnu/libfdt.so
which obviously breaks on toolchain updates.

In general libs used multiple times - ipn3ke + ifpga in this case - are
checked centrally in config/meson.build so move it there and fix the
adding of dependencies to not use the full file path.

The result is libfdt in pkg-config now showing up as:
  Libs.private: -pthread -lm -ldl -lnuma -lfdt -lpcap

Fixes: e1defba4cf ("raw/ifpga/base: support device tree")
Cc: stable@dpdk.org

Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Reviewed-by: Luca Boccassi <bluca@debian.org>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
2020-09-15 15:19:34 +02:00
Ciara Power
ec260aa3ad config: remove default configs used with make
Make is not supported for compiling DPDK, the config files are no
longer needed.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2020-09-08 00:11:30 +02:00
Thomas Monjalon
4f86c0ba19 version: 20.11-rc0
Start a new release cycle with empty release notes.

The ABI version becomes 21.0.
The ABI major is back to normal, having only one number (21 vs 20.0).
The map files are updated to the new ABI major number (21).
The ABI exceptions are dropped.
Travis ABI check is disabled because compatibility is not preserved.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
2020-08-12 11:32:16 +02:00
Randy Schacher
cf4f055a65 net/bnxt: remove EEM system memory support
Remove the memory management scheme for Extended Exact Match
using system memory. Using host memory scheme instead which
was the default anyway.

Fixes: b2da02480c ("net/bnxt: support EEM system memory")

Signed-off-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Peter Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Farah Smith <farah.smith@broadcom.com>
2020-07-30 00:41:24 +02:00
Ruifeng Wang
cc9035ffbb crypto/armv8: remove debug option
Typo in debug log switch macro caused debug log cannot be enabled.
Since no log used in data path, remove the debug option entirely
and have logs always enabled.

Resolved compilation error when debug log is enabled:
rte_armv8_pmd.c: In function ‘process_armv8_chained_op’:
rte_armv8_pmd.c:633:22: error: expected ‘)’ before ‘crypto_func’
  ARMV8_CRYPTO_ASSERT(crypto_func != NULL);
                      ^

Fixes: 169ca3db55 ("crypto/armv8: add PMD optimized for ARMv8 processors")
Cc: stable@dpdk.org

Reported-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Akhil Goyal <akhil.goyal@nxp.com>
2020-07-28 22:09:22 +02:00
Yuval Avnery
cf9b3c36e5 regex/mlx5: introduce driver for BlueField 2
This commit introduce the RegEx poll mode drivers class, and
adds Mellanox RegEx PMD.

Signed-off-by: Yuval Avnery <yuvalav@mellanox.com>
Signed-off-by: Ori Kam <orika@mellanox.com>
2020-07-21 19:04:05 +02:00
Peter Spreadborough
b2da02480c net/bnxt: support EEM system memory
- Select EEM Host or System memory via config parameter
- Add EEM system memory support for kernel memory
- Dependent on DPDK changes that add support for the HWRM_OEM_CMD.

Signed-off-by: Peter Spreadborough <peter.spreadborough@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
2020-07-07 23:38:27 +02:00
David Marchand
7762e0139b build: remove special versioning for non stable libraries
Having a special versioning for experimental/internal libraries put a
additional maintenance cost while this status is already announced in
MAINTAINERS and the library headers/documentation.
Following discussions and vote at 05/20 TB meeting [1], use a single
versioning for all libraries in DPDK.

Note: for the ABI check, an exception [2] had been added when tweaking
this special versioning [3].
Prefer explicit libabigail rules (which will be dropped in 20.11).

1: https://mails.dpdk.org/archives/dev/2020-May/168450.html
2: https://git.dpdk.org/dpdk/commit/?id=23d7ad5db41c
3: https://git.dpdk.org/dpdk/commit/?id=ec2b8cd7ed69

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2020-07-07 12:48:25 +02:00
Ori Kam
f66967d218 regexdev: implement API functions
This commit implements all the RegEx public API.

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Guy Kaneti <guyk@marvell.com>
2020-07-07 00:24:52 +02:00
Ori Kam
b25246beae regexdev: add core functions
This commit introduce the API that is needed by the RegEx devices in
order to work with the RegEX lib.

During the probe of a RegEx device, the device should configure itself,
and allocate the resources it requires.
On completion of the device init, it should call the
rte_regex_dev_register in order to register itself as a RegEx device.

Signed-off-by: Ori Kam <orika@mellanox.com>
Signed-off-by: Parav Pandit <parav@mellanox.com>
Acked-by: Guy Kaneti <guyk@marvell.com>
2020-07-07 00:24:51 +02:00
Jerin Jacob
bab9497ef7 regexdev: introduce API
As RegEx usage become more used by DPDK applications, for example:
* Next Generation Firewalls (NGFW)
* Deep Packet and Flow Inspection (DPI)
* Intrusion Prevention Systems (IPS)
* DDoS Mitigation
* Network Monitoring
* Data Loss Prevention (DLP)
* Smart NICs
* Grammar based content processing
* URL, spam and adware filtering
* Advanced auditing and policing of user/application security policies
* Financial data mining - parsing of streamed financial feeds
* Application recognition.
* Dmemory introspection.
* Natural Language Processing (NLP)
* Sentiment Analysis.
* Big data database acceleration.
* Computational storage.

Number of PMD providers started to work on HW implementation,
along side with SW implementations.

This lib adds the support for those kind of devices.

The RegEx Device API is composed of two parts:
- The application-oriented RegEx API that includes functions to setup
  a RegEx device (configure it, setup its queue pairs and start it),
  update the rule database and so on.

- The driver-oriented RegEx API that exports a function allowing
  a RegEx poll Mode Driver (PMD) to simultaneously register itself as
  a RegEx device driver.

RegEx device components and definitions:

    +-----------------+
    |                 |
    |                 o---------+    rte_regexdev_[en|de]queue_burst()
    |   PCRE based    o------+  |               |
    |  RegEx pattern  |      |  |  +--------+   |
    | matching engine o------+--+--o        |   |    +------+
    |                 |      |  |  | queue  |<==o===>|Core 0|
    |                 o----+ |  |  | pair 0 |        |      |
    |                 |    | |  |  +--------+        +------+
    +-----------------+    | |  |
           ^               | |  |  +--------+
           |               | |  |  |        |        +------+
           |               | +--+--o queue  |<======>|Core 1|
       Rule|Database       |    |  | pair 1 |        |      |
    +------+----------+    |    |  +--------+        +------+
    |     Group 0     |    |    |
    | +-------------+ |    |    |  +--------+        +------+
    | | Rules 0..n  | |    |    |  |        |        |Core 2|
    | +-------------+ |    |    +--o queue  |<======>|      |
    |     Group 1     |    |       | pair 2 |        +------+
    | +-------------+ |    |       +--------+
    | | Rules 0..n  | |    |
    | +-------------+ |    |       +--------+
    |     Group 2     |    |       |        |        +------+
    | +-------------+ |    |       | queue  |<======>|Core n|
    | | Rules 0..n  | |    +-------o pair n |        |      |
    | +-------------+ |            +--------+        +------+
    |     Group n     |
    | +-------------+ |<-------rte_regexdev_rule_db_update()
    | |             | |<-------rte_regexdev_rule_db_compile_activate()
    | | Rules 0..n  | |<-------rte_regexdev_rule_db_import()
    | +-------------+ |------->rte_regexdev_rule_db_export()
    +-----------------+

RegEx: A regular expression is a concise and flexible means for matching
strings of text, such as particular characters, words, or patterns of
characters. A common abbreviation for this is â~@~\RegExâ~@~].

RegEx device: A hardware or software-based implementation of RegEx
device API for PCRE based pattern matching syntax and semantics.

PCRE RegEx syntax and semantics specification:
http://regexkit.sourceforge.net/Documentation/pcre/pcrepattern.html

RegEx queue pair: Each RegEx device should have one or more queue pair to
transmit a burst of pattern matching request and receive a burst of
receive the pattern matching response. The pattern matching
request/response embedded in *rte_regex_ops* structure.

Rule: A pattern matching rule expressed in PCRE RegEx syntax along with
Match ID and Group ID to identify the rule upon the match.

Rule database: The RegEx device accepts regular expressions and converts
them into a compiled rule database that can then be used to scan data.
Compilation allows the device to analyze the given pattern(s) and
pre-determine how to scan for these patterns in an optimized fashion that
would be far too expensive to compute at run-time. A rule database
contains a set of rules that compiled in device specific binary form.

Match ID or Rule ID: A unique identifier provided at the time of rule
creation for the application to identify the rule upon match.

Group ID: Group of rules can be grouped under one group ID to enable
rule isolation and effective pattern matching. A unique group identifier
provided at the time of rule creation for the application to identify
the rule upon match.

Scan: A pattern matching request through *enqueue* API.

It may possible that a given RegEx device may not support all the
features
of PCRE. The application may probe unsupported features through
struct rte_regexdev_info::pcre_unsup_flags

By default, all the functions of the RegEx Device API exported by a PMD
are lock-free functions which assume to not be invoked in parallel on
different logical cores to work on the same target object. For instance,
the dequeue function of a PMD cannot be invoked in parallel on two logical
cores to operates on same RegEx queue pair. Of course, this function
can be invoked in parallel by different logical core on different queue
pair. It is the responsibility of the upper level application to
enforce this rule.

In all functions of the RegEx API, the RegEx device is
designated by an integer >= 0 named the device identifier *dev_id*

At the RegEx driver level, RegEx devices are represented by a generic
data structure of type *rte_regexdev*.
RegEx devices are dynamically registered during the PCI/SoC device
probing phase performed at EAL initialization time.
When a RegEx device is being probed, a *rte_regexdev* structure and
a new device identifier are allocated for that device. Then, the
regexdev_init() function supplied by the RegEx driver matching the
probed device is invoked to properly initialize the device.

The role of the device init function consists of resetting the hardware
or software RegEx driver implementations.

If the device init operation is successful, the correspondence between
the device identifier assigned to the new device and its associated
*rte_regexdev* structure is effectively registered.
Otherwise, both the *rte_regexdev* structure and the device identifier
are freed.

The functions exported by the application RegEx API to setup a device
designated by its device identifier must be invoked in the following
order:
    - rte_regexdev_configure()
    - rte_regexdev_queue_pair_setup()
    - rte_regexdev_start()

Then, the application can invoke, in any order, the functions
exported by the RegEx API to enqueue pattern matching job, dequeue
pattern matching response, get the stats, update the rule database,
get/set device attributes and so on

If the application wants to change the configuration (i.e. call
rte_regexdev_configure() or rte_regexdev_queue_pair_setup()), it must
call rte_regexdev_stop() first to stop the device and then do the
reconfiguration before calling rte_regexdev_start() again. The enqueue and
dequeue functions should not be invoked when the device is stopped.

Finally, an application can close a RegEx device by invoking the
rte_regexdev_close() function.

Each function of the application RegEx API invokes a specific function
of the PMD that controls the target device designated by its device
identifier.

For this purpose, all device-specific functions of a RegEx driver are
supplied through a set of pointers contained in a generic structure of
type *regexdev_ops*.
The address of the *regexdev_ops* structure is stored in the
*rte_regexdev* structure by the device init function of the RegEx driver,
which is invoked during the PCI/SoC device probing phase, as explained
earlier.

In other words, each function of the RegEx API simply retrieves the
*rte_regexdev* structure associated with the device identifier and
performs an indirect invocation of the corresponding driver function
supplied in the *regexdev_ops* structure of the *rte_regexdev*
structure.

For performance reasons, the address of the fast-path functions of the
RegEx driver is not contained in the *regexdev_ops* structure.
Instead, they are directly stored at the beginning of the *rte_regexdev*
structure to avoid an extra indirect memory access during their
invocation.

RTE RegEx device drivers do not use interrupts for enqueue or dequeue
operation. Instead, RegEx drivers export Poll-Mode enqueue and dequeue
functions to applications.

The *enqueue* operation submits a burst of RegEx pattern matching
request to the RegEx device and the *dequeue* operation gets a burst of
pattern matching response for the ones submitted through *enqueue*
operation.

Typical application utilisation of the RegEx device API will follow the
following programming flow.

- rte_regexdev_configure()
- rte_regexdev_queue_pair_setup()
- rte_regexdev_rule_db_update() Needs to invoke if precompiled rule
  database not
  provided in rte_regexdev_config::rule_db for rte_regexdev_configure()
  and/or application needs to update rule database.
- rte_regexdev_rule_db_compile_activate() Needs to invoke if
  rte_regexdev_rule_db_update function was used.
- Create or reuse exiting mempool for *rte_regex_ops* objects.
- rte_regexdev_start()
- rte_regexdev_enqueue_burst()
- rte_regexdev_dequeue_burst()

Signed-off-by: Jerin Jacob <jerinj@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Ori Kam <orika@mellanox.com>
2020-07-07 00:24:38 +02:00
Bruce Richardson
68b1f1cda5 build: check AVX512 rather than binutils version
Rather than checking the binutils version number, which can lead to
unnecessary disabling of AVX512 if fixes have been backported to distro
versions, we can instead check the output of "as" from binutils to see if
it is correct.

The check in the script uses the minimal assembly reproduction code posted
to the public bug tracker for gcc/binutils for those issues [1]. If the
binutils bug is present, the instruction parameters - specifically the
displacement parameter - will be different in the disassembled output
compared to the input. Therefore the check involves assembling a single
instruction and disassembling it again, checking that the two match.

[1] https://gcc.gnu.org/bugzilla/show_bug.cgi?id=90028

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Harry van Haaren <harry.van.haaren@intel.com>
2020-07-05 21:32:40 +02:00
Bruce Richardson
b29ac33ffc mem: fix 32-bit init config with meson
When building with meson, the default size of virtual address space
reserved for mapping pages was globally set at 512GB, which is too big for
use in 32-bit processes. To match the behaviour with "make", we configure
this to be 512GB for 64-bit and 2GB for 32-bit builds.

Bugzilla ID: 498
Fixes: 66cc45e293 ("mem: replace memseg with memseg lists")
Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-07-05 16:46:35 +02:00
Ruifeng Wang
78bfe1666b net/i40e: support aarch32
Expand vector PMD support to aarch32.
Enable i40e PMD by default for armv7 make build.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-06-30 14:52:30 +02:00
Ruifeng Wang
50dd63b9bf net/ixgbe: support aarch32
Expand vector PMD support to aarch32.
Enable ixgbe PMD by default for armv7 make build.

Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-06-30 14:52:30 +02:00
Harman Kalra
9311beeea4 net/octeontx2: support CN98xx
New cn98xx SOC comes up with two NIX blocks wrt
cn96xx, cn93xx, to achieve higher performance.
Also the no of cores increased to 36 from 24.

Adding support for cn98xx where need a logic to
detect if the LF is attached to NIX0 or NIX1 and
then accordingly use the respective NIX block.

Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
2020-06-30 14:52:30 +02:00
Thomas Monjalon
3b6431396a devtools: add Windows cross-build test with MinGW
The Meson cross file is renamed from meson_mingw.txt to cross-mingw,
and is added to test-meson-builds.sh.

The only example supported on Windows so far is "helloworld",
that's why the default list of examples is overridden.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2020-06-30 01:18:35 +02:00
Thomas Monjalon
b2b3858a08 devtools: add ppc64 in meson build test
Add cross-compilation support of a PPC target in the build test matrix.
The CPU is defined as Power8, running as little endian.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>
2020-06-30 01:18:35 +02:00
Wisam Jaddo
3344cf2e30 app/flow-perf: add flow performance skeleton
Add flow performance application skeleton.

Signed-off-by: Wisam Jaddo <wisamm@mellanox.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Xiaoyu Min <jackmin@mellanox.com>
2020-06-29 15:47:36 +02:00
Tal Shnaiderman
c91717eb75 eal/windows: support exit and panic
Support the debug functions in eal_common_debug.c for Windows.

Implementation of rte_dump_stack to get a backtrace similarly to Unix
and of rte_eal_cleanup in eal.c.

Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Tested-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
2020-06-24 11:02:51 +02:00
Dmitry Kozlyuk
a9bdee6ad9 config/x86: remove path for MinGW-w64 cross toolchain
Absolute paths in Meson cross-file impose unnecessary limitation
on build environment, remove them.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Pallavi Kadam <pallavi.kadam@intel.com>
2020-06-23 16:40:24 +02:00
Dmitry Kozlyuk
9d8feca542 config: never link with pthread on Windows
Even if pthread is provided by the toolchain, it is not needed for DPDK
on Windows, because internal shim is used. As a side-effect, this
enables cross-build with MinGW configured with non-POSIX thread library,
e.g. mcfgthread, which is the default on some distributions.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Acked-by: Pallavi Kadam <pallavi.kadam@intel.com>
2020-06-23 16:38:58 +02:00
Dmitry Kozlyuk
2a5d547a4a eal/windows: implement basic memory management
Basic memory management supports core libraries and PMDs operating in
IOVA as PA mode. It uses a kernel-mode driver, virt2phys, to obtain
IOVAs of hugepages allocated from user-mode. Multi-process mode is not
implemented and is forcefully disabled at startup. Assign myself as a
maintainer for Windows file and memory management implementation.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
2020-06-15 19:30:54 +02:00
Dmitry Kozlyuk
c08bd191b1 eal/windows: initialize hugepage info
Add hugepages discovery ("large pages" in Windows terminology)
and update documentation for required privilege setup. Only 2MB
hugepages are supported and their number is estimated roughly
due to the lack or unstable status of suitable OS APIs.
Assign myself as maintainer for the implementation file.

Signed-off-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
2020-06-15 19:30:32 +02:00
Kevin Traynor
cfacbcb5a2 build: disable gcc 10 zero-length-bounds warning
gcc 10 issues warnings about the use of rearm_data marker
from struct rte_mbuf.

e.g.
../drivers/net/enic/enic_rxtx_vec_avx2.c: In function ‘rx_one’:
../drivers/net/enic/enic_rxtx_vec_avx2.c:21:2:
warning:
array subscript 0 is outside the bounds of an interior zero-length array
‘RTE_MARKER64’ {aka ‘long unsigned int[0]’} [-Wzero-length-bounds]
   21 |  *(uint64_t *)&mb->rearm_data = enic->mbuf_initializer;
      |  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ../lib/librte_mbuf/rte_mbuf.h:45,
                 from ../drivers/net/enic/enic_rxtx_vec_avx2.c:6:
../lib/librte_mbuf/rte_mbuf_core.h:484:15:
note: while referencing ‘rearm_data’
  484 |  RTE_MARKER64 rearm_data;
      |

Disable this warning for gcc 10 in order to allow v20.05 to build
without changes to struct rte_mbuf.

Bugzilla ID: 396
Cc: stable@dpdk.org

Signed-off-by: Kevin Traynor <ktraynor@redhat.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
2020-05-18 13:54:36 +02:00
Ciara Power
eeb486f3ba eal: add telemetry as dependency
This patch moves telemetry further down the build, and adds it as a
dependency for EAL. Telemetry V2 is now configured to build by default,
and the legacy support is built when the telemetry config flag is set.

Telemetry now has EAL flags, shown below:
"--telemetry" = Enables telemetry (this is default if no flags given)
"--no-telemetry" = Disables telemetry

When telemetry is enabled, it will attempt to open the new socket
version, and also the legacy support socket (this will depend on Jansson
external dependency and telemetry config flag, as before).

Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
2020-05-11 00:37:16 +02:00
Bruce Richardson
44dfb297af build: add arch-specific header path to global includes
The global include path, which is used by anything built before EAL,
points to the EAL header files so they utility macros etc. can be used
anywhere in DPDK. This path included the OS-specific EAL header files,
but not the architecture-specific ones. This patch moves the selection
of target architecture to the top-level meson.build file so that the
global include can reference that.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Keith Wiles <keith.wiles@intel.com>
2020-05-10 23:45:02 +02:00
Nithin Dabilpuram
13fcf8aff7 node: add logging and null node
Add log infra for node specific logging.
Also, add null rte_node that just ignores all the objects
directed to it.

Signed-off-by: Nithin Dabilpuram <ndabilpuram@marvell.com>
Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Signed-off-by: Kiran Kumar K <kirankumark@marvell.com>
2020-05-05 23:37:43 +02:00