It is not needed to use "console" syntax highlighting
for literal blocks.
The file is easier to read by removing the code-block lines
and simply having double colons in previous line.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>
This patch adds some missing features to Mellanox drivers release notes.
It also updates the mlx5/mlx4 documentations.
Fixes: d85b204b5d ("doc: update release notes for Mellanox drivers")
Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The Memory Region (MR) for DMA memory can't be created from secondary
process due to lib/driver limitation. Whenever it is needed, secondary
process can make a request to primary process through the EAL IPC
channel (rte_mp_msg) which is established on initialization. Once a MR
is created by primary process, it is immediately visible to secondary
process because the MR list is global per a device. Thus, secondary
process can look up the list after the request is successfully returned.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
A new PMD parameter (mr_ext_memseg_en) is added to control extension of
memseg when creating a MR. It is enabled by default.
If enabled, mlx4_mr_create() tries to maximize the range of MR
registration so that the LKey lookup tables on datapath become smalle
and get the best performance. However, it may worsen memory utilization
because registered memory is pinned by kernel driver. Even if a page in
the extended chunk is freed, that doesn't become reusable until the
entire memory is freed and the MR is destroyed.
To make freed pages available immediately, this parameter has to be
turned off but it could drop performance.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
In order to support secondary process, a few features are required.
a) rdma-core library should allocate device resources using DPDK's
memory allocator.
b) UAR should be remapped for secondary processes. Currently, in order
not to use different data structure for secondary processes, PMD
tries to reserve identical virtual address space for both primary
and secondary processes.
c) IPC channel is necessary, which can be easily set with rte_mp APIs.
Through the channel, Verbs command FD is delivered to the secondary
process and the device stop/start event is also broadcast from
primary process.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Some drivers (mlx, mvpp2, sfc) support the flow isolated mode,
but the feature was not advertised.
A reference to the feature description is added for each driver.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The libraries provided by rdma-core may be statically linked
if enabling CONFIG_RTE_IBVERBS_LINK_STATIC in the make-based build.
If CONFIG_RTE_BUILD_SHARED_LIB is disabled, the applications
will embed the mlx PMDs with ibverbs and the mlx libraries.
If CONFIG_RTE_BUILD_SHARED_LIB is enabled,
the mlx PMDs will embed ibverbs and the mlx libraries.
Support with meson may be added later.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Rename options CONFIG_RTE_LIBRTE_MLX4_DLOPEN_DEPS and
CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS to a single option
CONFIG_RTE_IBVERBS_LINK_DLOPEN.
Rename meson option enable_driver_mlx_glue to ibverbs_link.
There was no good reason for setting a different link option
for mlx4 and mlx5. Having a single common option makes it
easier to understand and unify make and meson systems.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This is the new design of Memory Region (MR) for mlx PMD, in order to:
- Accommodate the new memory hotplug model.
- Support non-contiguous Mempool.
There are multiple layers for MR search.
L0 is to look up the last-hit entry which is pointed by mr_ctrl->mru (Most
Recently Used). If L0 misses, L1 is to look up the address in a fixed-sized
array by linear search. L0/L1 is in an inline function -
mlx4_mr_lookup_cache().
If L1 misses, the bottom-half function is called to look up the address
from the bigger local cache of the queue. This is L2 - mlx4_mr_addr2mr_bh()
and it is not an inline function. Data structure for L2 is the Binary Tree.
If L2 misses, the search falls into the slowest path which takes locks in
order to access global device cache (priv->mr.cache) which is also a B-tree
and caches the original MR list (priv->mr.mr_list) of the device. Unless
the global cache is overflowed, it is all-inclusive of the MR list. This is
L3 - mlx4_mr_lookup_dev(). The size of the L3 cache table is limited and
can't be expanded on the fly due to deadlock. Refer to the comments in the
code for the details - mr_lookup_dev(). If L3 is overflowed, the list will
have to be searched directly bypassing the cache although it is slower.
If L3 misses, a new MR for the address should be created -
mlx4_mr_create(). When it creates a new MR, it tries to register adjacent
memsegs as much as possible which are virtually contiguous around the
address. This must take two locks - memory_hotplug_lock and
priv->mr.rwlock. Due to memory_hotplug_lock, there can't be any
allocation/free of memory inside.
In the free callback of the memory hotplug event, freed space is searched
from the MR list and corresponding bits are cleared from the bitmap of MRs.
This can fragment a MR and the MR will have multiple search entries in the
caches. Once there's a change by the event, the global cache must be
rebuilt and all the per-queue caches will be flushed as well. If memory is
frequently freed in run-time, that may cause jitter on dataplane processing
in the worst case by incurring MR cache flush and rebuild. But, it would be
the least probable scenario.
To guarantee the most optimal performance, it is highly recommended to use
an EAL option - '--socket-mem'. Then, the reserved memory will be pinned
and won't be freed dynamically. And it is also recommended to configure
per-lcore cache of Mempool. Even though there're many MRs for a device or
MRs are highly fragmented, the cache of Mempool will be much helpful to
reduce misses on per-queue caches anyway.
'--legacy-mem' is also supported.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
This patch removes current support of Memory Region (MR) in order to
accommodate the dynamic memory hotplug patch. This patch can be compiled
but traffic can't flow and HW will raise faults. Subsequent patches will
add new MR support.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
There are two capabilities related to CRC stripping:
1. mlx4 HW capability to perform CRC stripping on a received packet.
This capability is built in mlx4 HW. It should be returned by the API
call mlx4_get_rx_queue_offloads().
2. mlx4 driver capability to enable/disable HW CRC stripping. This
capability is dependent on the driver version.
Before this commit the second capability was falsely returned by
the mentioned API. This commit fixes it by returning the first
capability.
mlx4 HW performs CRC stripping by default and this capability is
always reported as "true".
The ability to enable/disable CRC stripping is supported since this
commit and requires OFED version 4.3-1.5.0.0 or rdma-core version v18.
CRC stripping will be done by default regardless of its configuration
when working with OFED or rdma-core versions earlier than those
previously specified or before this commit.
Fixes: de1df14e6e ("net/mlx4: support CRC strip toggling")
Cc: stable@dpdk.org
Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Aligning Mellanox SPDX copyrights to a single format.
In addition replace to SPDX licence files which were missed.
Signed-off-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Glue object files are looked up in RTE_EAL_PMD_PATH by default when set and
should be installed in this directory.
During startup, EAL attempts to load them automatically like other plug-ins
found there. While normally harmless, dlopen() fails when rdma-core is not
installed, EAL interprets this as a fatal error and terminates the
application.
This patch requests glue objects to be installed in a different directory
to prevent their automatic loading by EAL since they are PMD helpers, not
actual DPDK plug-ins.
Fixes: f6242d0655 ("net/mlx: make rdma-core glue path configurable")
Cc: stable@dpdk.org
Reported-by: Timothy Redaelli <tredaelli@redhat.com>
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Tested-by: Timothy Redaelli <tredaelli@redhat.com>
Since rdma-core glue libraries are intrinsically tied to their respective
PMDs and used as internal plug-ins, their presence in the default search
path among other system libraries for the dynamic linker is not necessarily
desired.
This commit enables their installation and subsequent look-up at run time
in RTE_EAL_PMD_PATH if configured to a nonempty string. This path can also
be overridden by environment variables MLX[45]_GLUE_PATH.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
When mlx4 is not compiled directly as an independent shared object (e.g.
CONFIG_RTE_BUILD_SHARED_LIB not enabled for performance reasons), DPDK
applications inherit its dependencies on libibverbs and libmlx4 through
rte.app.mk.
This is an issue both when DPDK is delivered as a binary package (Linux
distributions) and for end users because rdma-core then propagates as a
mandatory dependency for everything.
Application writers relying on binary DPDK packages are not necessarily
aware of this fact and may end up delivering packages with broken
dependencies.
This patch therefore introduces an intermediate internal plug-in
hard-linked with rdma-core (to preserve symbol versioning) loaded by the
PMD through dlopen(), so that a missing rdma-core does not cause unresolved
symbols, allowing applications to start normally.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
This workaround was needed to properly handle device removal with old
Mellanox OFED releases that are not supported by this PMD anymore.
Starting from rdma-core v16 this removal issue shouldn't happen when
setting MLX4_DEVICE_FATAL_CLEANUP environment variable to 1.
Set the aforementioned variable to 1.
Reverts: 5f4677c6ad ("net/mlx4: workaround verbs error after plug-out")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
This updates mlx4 documentation and DPDK release notes
to reflect the PMD support for rdma-core from linux-rdma.
- PMD is now freed from Mellanox OFED and now only depends on the
public rdma-core package (v15 and above) instead.
(see https://github.com/linux-rdma/rdma-core/releases)
This PMD should run under Linux v4.14 and above.
- In case any of the above requirements can't be satisfied,
Mellanox OFED v4.2 and above also provide an updated rdma-core
as well back-ported kernel modules for most Linux distributions
and previous Linux versions.
(see http://www.mellanox.com/page/products_dyn?product_family=26).
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
This commit removes the stale MLX4_INLINE_RECV_SIZE environment
variable from the documentation.
Fixes: 056eaf2e6d ("net/mlx4: drop inline receive support")
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
The Verbs API used to implement Tx and Rx burst functions is deprecated.
Drop scatter/gather support to ease refactoring while maintaining basic
single-segment Rx/Tx functionality in the meantime.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
The Verbs API used to implement Tx and Rx checksum offloads is deprecated.
Support for these will be added back after refactoring the PMD.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
The Verbs RSS API used in this PMD is now obsolete. It is superseded by an
enhanced API with fewer constraints already used in the mlx5 PMD.
Drop RSS support in preparation for a major refactoring. The ability to
configure several Rx queues is retained, these can be targeted directly by
creating specific flow rules.
There is no need for "ignored" Rx queues anymore since their number is no
longer limited to powers of two.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Only the default port MAC address remains and is not configurable.
This is done in preparation for a major refactoring.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
This option both sets the maximum number of segments for Rx/Tx packets and
whether scattered mode is supported at all. This commit removes the latter
as well as configuration file exposure since the most appropriate value
should be decided at run-time.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Current implementation is partial (Tx only), not convenient to use and
not of primary concern.
Remove this feature before refactoring the PMD.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Copyright lasts long enough not to require notices to be updated yearly.
The current approach of updating them occasionally while working on
unrelated tasks should be deprecated in favor of dedicated commits updating
all files at once when necessary.
Standardize on a single year per copyright owner.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Current mlx4 OFED version has bug which returns error to
ibv destroy functions when the device was plugged out, in
spite of the resources were destroyed correctly.
Hence, failsafe PMD was aborted, only in debug mode, when
it tries to remove the device in plug-out process.
The workaround added option to replace all claim_zero
assertions with debugging messages, by the way, this option
affects non ibv destroy assertions.
DPDK 18.02 release should work with Mellanox OFED-4.2 which will
include the verbs fix to this bug, then, this patch can
be removed.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Most ConnectX-3 adapters expose two physical ports on a single PCI bus
address.
Add a new port parameter allowing the user to choose
either or both physical ports to be used by the application.
This parameter is used as follows:
Selecting only the second port:
-w 00:00.0,port=1
Selecting both ports:
-w 00:00.0,port=0,port=1
If no parameter is given, the default behavior is unchanged: all ports
are probed.
Signed-off-by: Gaetan Rivet <gaetan.rivet@6wind.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
The coremask option in DPDK is difficult to use and we should be
promoting the use of the corelist (-l) option. The patch
adjusts the docs to use -l EAL option instead of the -c option.
The patch only changes the docs and not the code as the -c option
will continue to exist unless it is removed in the future. The -c
option should be kept to maintain backward compatibility.
Signed-off-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: John McNamara <john.mcnamara@intel.com>
The physically linked-together combined library has been an increasing
source of problems, as was predicted when library and symbol versioning
was introduced. Replace the complex and fragile construction with a
simple linker script which achieves the same without all the problems,
remove the related kludges from eg mlx drivers.
Since creating the linker script is practically zero cost, remove the
config option and just create it always.
Based on a patch by Sergio Gonzales Monroy, linker script approach
initially suggested by Neil Horman.
Suggested-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Suggested-by: Neil Horman <nhorman@tuxdriver.com>
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
-Split "Features" and "Limitations" sections.
-Update limitations with missing information.
-Update prerequisites with supported MLNX_OFED release, firmware and
CPU architectures.
-Enhance usage example with openibd script.
Signed-off-by: Olga Shern <olgas@mellanox.com>
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
librte_pmd_mlx4.so needs to be linked with libibverbs otherwise, the PMD is
not able to open Mellanox devices and the following message is printed by
testpmd at startup
"librte_pmd_mlx4: cannot access device, is mlx4_ib loaded?".
Applications dependency on libibverbs are moved to be only valid in static
mode, in shared mode, applications do not depend on it anymore,
librte_pmd_mlx4.so keeps this dependency and thus is linked with libibverbs.
MLX4 cannot be supported in combined shared library because there is no clean
way of adding -libverbs to the combined library.
Signed-off-by: Nelio Laranjeiro <nelio.laranjeiro@6wind.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
- libmlx4 and libibverbs dependencies distributed with Mellanox OFED are now
also available on DPDK.org to make installation easier.
- Document Mellanox OFED and firmware versions to use.
- Add links to Mellanox and its community websites.
- Add kernel modules parameters section.
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Create nics guide by moving chapters about Intel and Mellanox NICs.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Siobhan Butler <siobhan.a.butler@intel.com>
