Commit Graph

93 Commits

Author SHA1 Message Date
David Marchand
1acb7f5474 dev: hide driver object
Make rte_driver opaque for non internal users.
This will make extending this object possible without breaking the ABI.

Introduce a new driver header and move rte_driver definition.
Update drivers and library to use the internal header.

Some applications may have been dereferencing rte_driver objects, mark
this object's accessors as stable.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Jay Jayatheerthan <jay.jayatheerthan@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Akhil Goyal <gakhil@marvell.com>
Acked-by: Abhinandan Gujjar <abhinandan.gujjar@intel.com>
2022-09-23 16:14:34 +02:00
David Marchand
4851ef2b40 bus/vdev: make driver-only headers private
The vdev bus interface is for drivers only.
Mark as internal and move the header in the driver headers list.

While at it, cleanup the code:
- fix indentation,
- remove unneeded reference to bus specific singleton object,
- remove unneeded list head structure type,
- reorder the definitions and macro manipulating the bus singleton object,
- remove inclusion of rte_bus.h and fix the code that relied on implicit
  inclusion,

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
2022-09-23 16:14:34 +02:00
David Marchand
72206323a5 version: 22.11-rc0
Start a new release cycle with empty release notes.

The ABI version becomes 23.0.
The map files are updated to the new ABI major number (23).
The ABI exceptions are dropped and CI ABI checks are disabled because
compatibility is not preserved.
Special handling of removed drivers is also dropped in check-abi.sh and
a note has been added in libabigail.abignore as a reminder.

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
2022-07-21 12:13:48 +02:00
Xiaoyun Li
b275e298c1 net/af_xdp: allow using copy mode in XSK
DPDK assumes that users only want AF_XDP socket (XSK) into zero copy
mode when the kernel supports it. However, sometimes kernel driver
doesn't support it well and copy mode is more stable and preferred.

This patch allows using devarg "-a xx:xx.x,force_copy=1" to force the
AF_XDP socket into copy mode.

Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
2022-06-23 12:49:51 +02:00
Ciara Loftus
6dd3286f3a net/af_xdp: fix shared UMEM fill queue reserve
Commit 81fe6720f8 ("net/af_xdp: reserve fill queue before socket create")
moves the fill queue reserve logic to before the creation of the socket in
order to suppress kernel logs like:

XSK buffer pool does not provide enough addresses to fill 2047 buffers on
Rx ring 0

However, for queues that share umem, the fill queue reserve must occur
after the socket creation, because the fill queue is not valid until
that point.

This commit uses the umem refcnt value to determine whether the queue is
sharing a umem, and performs the fill queue reservation either before or
after the socket creation, depending on the refcnt value.

The kernel logs will still be seen for the shared umem queues.

Fixes: 81fe6720f8 ("net/af_xdp: reserve fill queue before socket create")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2022-03-14 10:48:18 +01:00
Junxiao Shi
6f6134c35e net/af_xdp: fix custom program loading with multiple queues
When the PMD is configured to load a custom XDP program, it sets
XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD flag to prevent libbpf from
loading its default XDP program. However, when queue_count is set to
greater than 1, this flag is only set for the first XSK socket but not
for subsequent XSK sockets. This causes XSK socket creation failure.

This commit ensures that XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD flag is
set for all XSK socket creations when custom XDP program is being used.

Fixes: 01fa83c94d ("net/af_xdp: workaround custom program loading")
Cc: stable@dpdk.org

Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
2022-03-14 09:39:00 +01:00
Ciara Loftus
81fe6720f8 net/af_xdp: reserve fill queue before socket create
Some zero copy AF_XDP drivers eg. ice require that there are addresses
already in the fill queue before the socket is created. Otherwise you may
see log messages such as:

XSK buffer pool does not provide enough addresses to fill 2047 buffers on
Rx ring 0

This commit ensures that the addresses are available before creating the
socket, instead of after.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Tested-by: Ferruh Yigit <ferruh.yigit@intel.com>
2022-02-18 19:48:17 +01:00
Ciara Loftus
b26431a617 net/af_xdp: ensure socket is deleted on Rx queue setup error
The Rx queue setup can fail for many reasons eg. failure to setup the
custom program, failure to allocate or reserve fill queue buffers,
failure to configure busy polling etc. When a failure like one of these
occurs, if the xsk is already set up it should be deleted before
returning. This commit ensures this happens.

Fixes: d8a210774e ("net/af_xdp: support unaligned umem chunks")
Fixes: 288a85aef1 ("net/af_xdp: enable custom XDP program loading")
Fixes: 055a393626 ("net/af_xdp: prefer busy polling")
Fixes: 01fa83c94d ("net/af_xdp: workaround custom program loading")
Cc: stable@dpdk.org

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-18 19:47:52 +01:00
Ciara Loftus
8d3d9c7251 net/af_xdp: make compatible with libbpf >= 0.7.0
libbpf v0.7.0 deprecates the bpf_prog_load function. Use meson to detect
if libbpf >= v0.7.0 is linked and if so, use the recommended replacement
functions bpf_object__open_file and bpf_object__load.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-18 11:37:34 +01:00
David Marchand
744fd41269 net/af_xdp: add missing trailing newline in logs
Caught while trying --in-memory mode, some log messages in this driver
are not terminated with a newline:
rte_pmd_af_xdp_probe(): net_af_xdp: Failed to register multi-process IPC
callback: Operation not supportedvdev_probe(): failed to initialize
net_af_xdp device

Other locations in this driver had the same issue, fix all at once.

Fixes: f1debd77ef ("net/af_xdp: introduce AF_XDP PMD")
Fixes: d8a210774e ("net/af_xdp: support unaligned umem chunks")
Fixes: 9876cf8316 ("net/af_xdp: re-enable secondary process support")
Cc: stable@dpdk.org

Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-17 18:47:57 +01:00
Junxiao Shi
17ec9678d4 net/af_xdp: allow probing when multiprocess is disabled
If EAL multiprocess feature has been disabled via rte_mp_disable()
function, AF_XDP driver may not be able to register its IPC callback.
Previously this leads to probe failure.
This commit adds a check for this condition so that AF_XDP can still be
used even if multiprocess is disabled.

Fixes: 9876cf8316 ("net/af_xdp: re-enable secondary process support")

Signed-off-by: Junxiao Shi <git@mail1.yoursunny.com>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-17 18:26:00 +01:00
Ferruh Yigit
a41f593f1b ethdev: introduce generic dummy packet burst function
Multiple PMDs have dummy/noop Rx/Tx packet burst functions.

These dummy functions are very simple, introduce a common function in
the ethdev and update drivers to use it instead of each driver having
its own functions.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
2022-02-11 21:17:34 +01:00
Ciara Loftus
9876cf8316 net/af_xdp: re-enable secondary process support
Secondary process support had been disabled for the AF_XDP PMD because
there was no logic in place to share the AF_XDP socket file descriptors
between the processes. This commit introduces this logic using the IPC
APIs.

Rx and Tx are disabled in the secondary process due to memory mapping of
the AF_XDP rings being assigned by the kernel in the primary process only.
However other operations including retrieval of stats are permitted.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
2022-02-11 14:19:13 +01:00
Haiyue Wang
e1543baea3 net/af_xdp: make UMEM configure more readable
The below compile time defined style make the code not so readable, the
first function end block is after "#endif" segment.

	#if defined(XDP_UMEM_UNALIGNED_CHUNK_FLAG)

	xdp_umem_configure()
	{

	#else
	xdp_umem_configure()
	{

	#endif
		'shared code block'
	}

Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-10 11:23:17 +01:00
Ciara Loftus
fa4dfda5fe net/af_xdp: use libxdp if available
AF_XDP support is deprecated in libbpf since v0.7.0 [1]. The libxdp library
now provides the functionality which once was in libbpf and which the
AF_XDP PMD relies on. This commit updates the AF_XDP meson build to use the
libxdp library if a version >= v1.2.2 is available. If it is not available,
only versions of libbpf prior to v0.7.0 are allowed, as they still contain
the required AF_XDP functionality.

libbpf still remains a dependency even if libxdp is present, as we use
libbpf APIs for program loading.

The minimum required kernel version for libxdp for use with AF_XDP is v5.3.
For the library to be fully-featured, a kernel v5.10 or newer is
recommended. The full compatibility information can be found in the libxdp
README.

v1.2.2 of libxdp includes an important fix required for linking with DPDK
which is why this version or greater is required. Meson uses pkg-config to
verify the version of libxdp on the system, so it is necessary that the
library is discoverable using pkg-config in order for the PMD to use it. To
verify this, you can run: pkg-config --modversion libxdp

[1] https://github.com/libbpf/libbpf/commit/277846bc6c15

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2022-02-01 11:08:00 +01:00
Ciara Loftus
af8d89a0c6 net/af_xdp: fix build with -Wunused-function
The get_shared_umem function is only called when the kernel
flag XDP_UMEM_UNALIGNED_CHUNK_FLAG is defined. Move the
function implementation and associated helper so that it only
gets compiled when that flag is set.

Fixes: 74b46340e2 ("net/af_xdp: support shared UMEM")
Cc: stable@dpdk.org

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
2022-01-18 11:27:48 +01:00
Ciara Loftus
01fa83c94d net/af_xdp: workaround custom program loading
Since v0.4.0, if the underlying kernel supports it, libbpf uses 'bpf
link' to manage the programs on the interfaces of the XDP sockets (xsks).
This is not compatible with the PMD's custom XDP program loading feature
which uses the netlink-based method for loading custom programs.

The conflict arises when libbpf searches for a custom program on the
interface using bpf link, but doesn't find one because the netlink
method was used. libbpf then proceeds to try to load the default program
on the interface, but fails due to the presence of the custom program.

To work around this, the PMD now uses the
XSK_LIBBPF_FLAGS__INHIBIT_PROG_LOAD flag which prevents libbpf from
attempting to search for or load a program. One repercussion is that
DPDK must now insert the xsk into the xsks_map as this was previously
handled by libbpf during the routines for program loading/probing.

Ideally, the PMD would use bpf link to load the custom program, however
at present there is no convenient and reliable way of detecting whether
the underlying kernel supports bpf link. Perhaps this may become
available in a future libbpf release, at which point we can switch the
PMD over to the new bpf link based method.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-11-15 18:01:09 +01:00
Ciara Loftus
9f394004c3 net/af_xdp: revert use BPF link for XDP programs
The commit ae70cc6e89 ("net/af_xdp: use BPF link for XDP programs")
caused compilation errors on kernels older than v5.8 due to absence of
the bpf_link_info struct and some definitions in the linux/bpf.h header.
Since relying on the reported kernel version is not a robust solution
and also since there doesn't appear to be a suitable definition in the
bpf header that the preprocessor could rely on to determine support for
bpf link, we will take a different approach to solving the issue that
the original patch attempted to solve. The next commit will address
this.

Fixes: ae70cc6e89 ("net/af_xdp: use BPF link for XDP programs")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-11-15 18:01:09 +01:00
Ciara Loftus
ae70cc6e89 net/af_xdp: use BPF link for XDP programs
Since v0.4.0, if the underlying kernel supports it, libbpf uses 'bpf
link' to manage the programs on the interfaces of the xsks. This has two
repercussions for the PMD.

1. In the case where the PMD asks libbpf to load the default XDP
   program, the PMD no longer needs to remove it on teardown. This is
   because bpf link handles the unloading under the hood.
2. In the case where the PMD loads a custom program, libbpf expects this
   program to be linked via bpf link prior to creating the socket.

This patch introduces probes for the libbpf version and kernel support
for bpf link and orchestrates the loading and unloading of
programs according to the capabilities of the kernel and libbpf. The
libbpf version is checked with meson and pkg-config. The probe for
kernel support mirrors how it is implemented in libbpf. A bpf_link is
created and looked up on loopback device. If successful, bpf_link will
be used for the AF_XDP netdev.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Signed-off-by: Maciej Fijalkowski <maciej.fijalkowski@intel.com>
2021-11-02 17:36:46 +01:00
Ferruh Yigit
295968d174 ethdev: add namespace
Add 'RTE_ETH' namespace to all enums & macros in a backward compatible
way. The macros for backward compatibility can be removed in next LTS.
Also updated some struct names to have 'rte_eth' prefix.

All internal components switched to using new names.

Syntax fixed on lines that this patch touches.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Wisam Jaddo <wisamm@nvidia.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Somnath Kotur <somnath.kotur@broadcom.com>
2021-10-22 18:15:38 +02:00
Ciara Loftus
985e7673c0 net/af_xdp: fix max Rx packet length
Commit 1bb4a528c4 ("ethdev: fix max Rx packet length") clarified the
expected usage of the max_rx_pktlen and max_mtu values and implemented
some extra checks on these values to ensure they are sane. After this,
the AF_XDP PMD fails to initialise. The value for max_rx_pktlen which
represents the max size of the Ethernet frame was set to ETH_FRAME_LEN
(1514) and the max_mtu which represents the size of the payload was set
to the max size of the Ethernet frame. This did not make sense, as
naturally the maximum frame size should be greater than the payload
size.

Fix this by setting the max_rx_pktlen equal to the max size of the
Ethernet frame as expected, and the max MTU equal to the max_rx_pktlen
less the overhead which is set to the size of an Ethernet header plus
CRC.

Fixes: 1bb4a528c4 ("ethdev: fix max Rx packet length")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-10-22 17:12:50 +02:00
Ciara Loftus
0668d82908 net/af_xdp: disable secondary process support
Since the AF_XDP PMD does not work for secondary processes as reported
in Bugzilla 805, check for the process type at the beginning of probe
and return ENOTSUP if the process type is secondary.

It is planned that secondary processes will be supported by the PMD in
full in a future release by using rte_mp_msg to pass the state to the
secondary process that it requires in order to work.

Bugzilla ID: 805
Fixes: f1debd77ef ("net/af_xdp: introduce AF_XDP PMD")
Cc: stable@dpdk.org

Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
2021-10-08 13:06:55 +02:00
Xueming Li
49ed322469 ethdev: make queue release callback optional
Some drivers don't need Rx and Tx queue release callback, make them
optional. Clean up empty queue release callbacks for some drivers.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
2021-10-06 19:16:03 +02:00
Baruch Siach
0f2ffb49ce net/af_xdp: fix zero-copy Tx queue drain
Call xsk_ring_prod__submit() before kick_tx() so that the kernel
consumer sees the updated state of Tx ring. Otherwise, Tx packets are
stuck in the ring until the next call to af_xdp_tx_zc().

Fixes: d8a210774e ("net/af_xdp: support unaligned umem chunks")
Cc: stable@dpdk.org

Signed-off-by: Baruch Siach <baruch@tkos.co.il>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
2021-09-07 17:55:23 +02:00
Thomas Monjalon
fdab8f2e17 version: 21.11-rc0
Start a new release cycle with empty release notes.

The ABI version becomes 22.0.
The map files are updated to the new ABI major number (22).
The ABI exceptions are dropped and CI ABI checks are disabled because
compatibility is not preserved.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
2021-08-17 08:37:52 +02:00
Anatoly Burakov
43fb6eea49 net/af_xdp: support power monitoring
Implement support for .get_monitor_addr in AF_XDP driver.

Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
2021-07-09 21:13:13 +02:00
Ciara Loftus
89233c081c net/af_xdp: fix build with Linux < 5.4
Prior to this change, two implementations of rx_syscall_handler
existed although only one was needed (for the zero copy path which
is only available from kernel 5.4 and onwards). Remove the second
definition from compat.h and move the first definition back to where
it is called in the Rx function. Doing this removes a build warning
on kernels before 5.4 which complained about the second function
being defined but not used.

Fixes: 2aa51cdd55 ("net/af_xdp: fix trigger for syscall on Tx")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-05-11 16:11:26 +02:00
David Marchand
eeded2044a log: register with standardized names
Let's try to enforce the convention where most drivers use a pmd. logtype
with their class reflected in it, and libraries use a lib. logtype.

Introduce two new macros:
- RTE_LOG_REGISTER_DEFAULT can be used when a single logtype is
  used in a component. It is associated to the default name provided
  by the build system,
- RTE_LOG_REGISTER_SUFFIX can be used when multiple logtypes are used,
  and then the passed name is appended to the default name,

RTE_LOG_REGISTER is left untouched for existing external users
and for components that do not comply with the convention.

There is a new Meson variable log_prefix to adapt the default name
for baseband (pmd.bb.), bus (no pmd.) and mempool (no pmd.) classes.

Note: achieved with below commands + reverted change on net/bonding +
edits on crypto/virtio, compress/mlx5, regex/mlx5

$ git grep -l RTE_LOG_REGISTER drivers/ |
  while read file; do
    pattern=${file##drivers/};
    class=${pattern%%/*};
    pattern=${pattern#$class/};
    drv=${pattern%%/*};
    case "$class" in
      baseband) pattern=pmd.bb.$drv;;
      bus) pattern=bus.$drv;;
      mempool) pattern=mempool.$drv;;
      *) pattern=pmd.$class.$drv;;
    esac
    sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
    sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
  done

$ git grep -l RTE_LOG_REGISTER lib/ |
  while read file; do
    pattern=${file##lib/};
    pattern=lib.${pattern%%/*};
    sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
    sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
  done

Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2021-05-11 15:17:55 +02:00
Ciara Loftus
b42cffab05 net/af_xdp: use recvfrom only if busy polling enabled
The recvfrom() syscall is only supported by AF_XDP sockets since
kernel 5.11. Only use it if busy polling is configured. We can
assume a kernel >= 5.11 is in use if busy polling is configured
so we can safely call recvfrom() in that case.

Fixes: 63e8989fe5 ("net/af_xdp: use recvfrom instead of poll syscall")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-04-20 12:09:33 +02:00
Ciara Loftus
2aa51cdd55 net/af_xdp: fix trigger for syscall on Tx
The send() syscall on the Tx path is not concerned with busy polling
and as such its invocation should not depend on whether or not it is
configured. Fix this by distinguishing the conditions necessary for
syscalls on the Rx and Tx paths individually.

Fixes: 055a393626 ("net/af_xdp: prefer busy polling")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-04-20 12:09:33 +02:00
Bruce Richardson
4ad4b20a79 drivers: change indentation in build files
Switch from using tabs to 4 spaces for meson.build indentation.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2021-04-21 14:04:09 +02:00
Ciara Loftus
e66716d52e net/af_xdp: mark recvfrom return as ignored
Coverity complains that the return value of recvfrom() in the AF_XDP
datapath is not checked. We don't care about the return value because in
the case of an error we still return 0 from the receive function to
indicate no packets were received. So to make Coverity happy we cast the
return to 'void'.

Coverity issue: 369671
Fixes: 63e8989fe5 ("net/af_xdp: use recvfrom instead of poll syscall")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-03-30 15:50:04 +02:00
Ciara Loftus
055a393626 net/af_xdp: prefer busy polling
This commit introduces support for preferred busy polling
to the AF_XDP PMD. This feature aims to improve single-core
performance for AF_XDP sockets under heavy load.

A new vdev arg is introduced called 'busy_budget' whose default
value is 64. busy_budget is the value supplied to the kernel
with the SO_BUSY_POLL_BUDGET socket option and represents the
busy-polling NAPI budget. To set the budget to a different value
eg. 256:

--vdev=net_af_xdp0,iface=eth0,busy_budget=256

Preferred busy polling is enabled by default provided a kernel with
version >= v5.11 is in use. To disable it, set the budget to zero.

The following settings are also strongly recommended to be used in
conjunction with this feature:

echo 2 | sudo tee /sys/class/net/eth0/napi_defer_hard_irqs
echo 200000 | sudo tee /sys/class/net/eth0/gro_flush_timeout

.. where eth0 is the interface being used by the PMD.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-03-10 18:49:32 +01:00
Ciara Loftus
63e8989fe5 net/af_xdp: use recvfrom instead of poll syscall
poll() is more expensive and requires more tuning
when used with the upcoming busy polling functionality.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-03-10 18:49:32 +01:00
Ciara Loftus
d96394ea26 net/af_xdp: allow bigger batch sizes
Prior to this commit, the maximum batch sizes for zero-copy and
copy-mode rx and copy-mode tx were set to 32. Apart from zero-copy tx,
the user could never rx/tx any more than 32 packets at a time and
without inspecting the code the user wouldn't be aware of this.

This commit removes these upper limits placed on the user and instead
sets an internal batch size equal to the default ring size (2048).
Batches larger than this are still processed, however they are split
into smaller batches similar to how it's done in other drivers. This is
necessary because some arrays used during rx/tx need to be sized at
compile-time.

Allowing a larger batch size allows for fewer batches and thus larger
bulk operations, fewer ring accesses and fewer syscalls which should
yield improved performance.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-03-10 18:49:32 +01:00
Ciara Loftus
8312680223 net/af_xdp: fix error handling during Rx queue setup
Prior to this commit, if rte_pktmbuf_alloc_bullk failed during rx queue
setup the error was not returned to the user and they may incorrectly
assume that the rx queue had been successfully set up. This commit ensures
that the error is returned to the user.

Bugzilla ID: 643
Fixes: d8a210774e ("net/af_xdp: support unaligned umem chunks")
Cc: stable@dpdk.org

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2021-03-01 23:44:43 +01:00
Bruce Richardson
df96fd0d73 ethdev: make driver-only headers private
The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Steven Webster <steven.webster@windriver.com>
2021-01-29 20:59:09 +01:00
Bruce Richardson
7be7dc6dea build: force pkg-config for dependency detection
Meson can use cmake as a fallback for detecting packages, and this can
lead to picking up 64-libs for 32-bit builds. To work around this, force
the use of pkg-config only for detecting libcrypto, zlib, jansson and
other package dependencies.

Cc: stable@dpdk.org

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Tested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Tested-by: Liron Himi <lironh@marvell.com>
Tested-by: Lee Daly <lee.daly@intel.com>
Tested-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Tested-by: Martin Spinler <spinler@cesnet.cz>
2021-01-26 00:43:59 +01:00
Pallavi Kadam
b9d60b5434 drivers/net: build i40e and mlx5 on Windows
Allows i40e and mlx5 PMDs to compile on Windows and disable other drivers.

Disable few i40e warnings with Clang such as comparison of integers of
different signs and macro redefinitions.

Signed-off-by: Pallavi Kadam <pallavi.kadam@intel.com>
Reviewed-by: Ranjit Menon <ranjit.menon@intel.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-01-14 23:51:24 +01:00
RongQing Li
543e64d244 net/af_xdp: optimize Rx mbuf allocation
While receiving packets, the max bunch number of mbufs are allocated
and if hardware does not receive the max bunch number packets, it
will free redundancy mbufs, this is low performance.

So optimize Rx performance, by allocating number of mbuf based on
result of xsk_ring_cons__peek, to avoid to redundancy allocation,
and free mbuf when receive packets.

And Rx cached_cons must be roll backed if fails to allocate mbuf.

Signed-off-by: RongQing Li <lirongqing@baidu.com>
Signed-off-by: Dongsheng Rong <rongdongsheng@baidu.com>
Acked-by: Ciara Loftus <ciara.loftus@intel.com>
2021-01-08 16:03:04 +01:00
Ferruh Yigit
c275321105 net/af_xdp: remove useless assignment
Assignment of function parameter 'umem' removed.

Fixes: f0ce7af0e1 ("net/af_xdp: remove resources when port is closed")
Cc: stable@dpdk.org

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
2021-01-11 10:45:34 +01:00
Ferruh Yigit
470b5c23ad net/af_xdp: fix pointer storage size
'uint64_t' is used to hold the pointer, for 32-bits build this
assumption is wrong and giving following build error:

rte_eth_af_xdp.c: In function ‘xdp_umem_configure’:
rte_eth_af_xdp.c:970:15:
    error: cast to pointer from integer of different size
           [-Werror=int-to-pointer-cast]
  970 |   base_addr = (void *)get_base_addr(mb_pool, &align);
      |               ^

Replacing the 'uint64_t' return type of the 'get_base_addr()' to the
'uintptr_t'.
Although not sure if the overall logic supports the 32-bits, using
'uintptr_t' should be safe both for 64/32 bits.

Fixes: d8a210774e ("net/af_xdp: support unaligned umem chunks")
Cc: stable@dpdk.org

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Tested-by: Ciara Loftus <ciara.loftus@intel.com>
2020-11-13 19:43:26 +01:00
Martin Weiser
97671617ec net/af_xdp: fix integer overflow in umem size calculation
The multiplication of two u32 integers may cause an overflow with large
mempool sizes.

Fixes: 74b46340e2 ("net/af_xdp: support shared UMEM")

Signed-off-by: Martin Weiser <martin.weiser@allegro-packets.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-11-03 23:35:06 +01:00
Bruce Richardson
63b3907833 build: remove library name from version map file name
Since each version map file is contained in the subdirectory of the library
it refers to, there is no need to include the library name in the filename.
This makes things simpler in case of library renaming.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
2020-10-19 22:13:59 +02:00
Ferruh Yigit
f30e69b41f ethdev: add device flag to bypass auto-filled queue xstats
Queue stats are stored in 'struct rte_eth_stats' as array and array size
is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag.

As a result of technical board discussion, decided to remove the queue
statistics from 'struct rte_eth_stats' in the long term.

Instead PMDs should represent the queue statistics via xstats, this
gives more flexibility on the number of the queues supported.

Currently queue stats in the xstats are filled by ethdev layer, using
some basic stats, when queue stats removed from basic stats the
responsibility to fill the relevant xstats will be pushed to the PMDs.

During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS'
device flag is created. Initially all PMDs using xstats set this flag.
The PMDs implemented queue stats in the xstats should clear the flag.

When all PMDs switch to the xstats for the queue stats, queue stats
related fields from 'struct rte_eth_stats' will be removed, as well as
'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag.
Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be
removed.

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Xiao Wang <xiao.w.wang@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
2020-10-16 23:27:15 +02:00
Ivan Ilchenko
62024eb827 ethdev: change stop operation callback to return int
Change eth_dev_stop_t return value from void to int.
Make eth_dev_stop_t implementations across all drivers to return
negative errno values if case of error conditions.

Signed-off-by: Ivan Ilchenko <ivan.ilchenko@oktetlabs.ru>
Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-10-16 22:26:41 +02:00
Ciara Loftus
53a73b7b9d net/af_xdp: forbid umem sharing for xsks with same context
AF_XDP PMDs who wish to share a UMEM must have a unique context
(ctx) ie. netdev,qid tuple. For instance, the following will not
work since both PMDs' contexts are identical.

  --vdev net_af_xdp0,iface=ens786f1,start_queue=0,shared_umem=1
  --vdev net_af_xdp1,iface=ens786f1,start_queue=0,shared_umem=1

Supporting this scenario would require locks, which would impact
the performance of the more typical cases - xsks with different
netdev,qid tuples.

Fixes: 74b46340e2 ("net/af_xdp: support shared UMEM")

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
2020-10-16 19:48:17 +02:00
Ciara Loftus
6d3c595d3a net/af_xdp: use strlcpy instead of strncpy
strncpy may leave the destination buffer not NULL terminated so use
strlcpy instead.

Coverity issue: 362975
Fixes: 339b88c6a9 ("net/af_xdp: support multi-queue")
Cc: stable@dpdk.org

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2020-10-16 19:18:47 +02:00
Ciara Loftus
288a85aef1 net/af_xdp: enable custom XDP program loading
The new 'xdp_prog=<string>' vdev arg allows the user to specify the path to
a custom XDP program to be set on the device, instead of the default libbpf
one. The program must have an XSK_MAP of name 'xsks_map' which will allow
for the redirection of some packets to userspace and thus the PMD, using
some criteria defined in the program. This can be useful for filtering
purposes, for example if we only want a subset of packets to reach
userspace or to drop or process a subset of packets in the kernel.

Note: a netdev may only load one program.

Signed-off-by: Ciara Loftus <ciara.loftus@intel.com>
Tested-by: Xuekun Hu <xuekun.hu@intel.com>
2020-09-30 19:19:15 +02:00
Thomas Monjalon
3041049375 drivers/net: check process type in close operation
The secondary processes are not allowed to release shared resources.
Only process-private resources should be freed in a secondary process.
Most of the time, there is no process-private resource,
so the close operation is just forbidden in a secondary process.

After adding proper check in the port close functions,
some redundant checks in the device remove functions are dropped.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Sachin Saxena <sachin.saxena@oss.nxp.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Liron Himi <lironh@marvell.com>
Reviewed-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
2020-09-30 19:19:14 +02:00