Added <rte_cycles.h> inclusion, was not included on some
building setups (armv8).
Fixes: 71ab2d6472 ("net/mlx5: fix VXLAN port registration race condition")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
There is an intention to support VXLAN tunnel match without
hardware offloaded decapsulation, just to redirect ingress
tunnelled frame untouched. This small fix allows to specify
Flows with VXLAN VNI pattern and with or without following
decapsulation action.
Fixes: 251e8d02cf ("net/mlx5: add VXLAN to flow translate routine")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
E-Switch VXLAN tunneling rules require virtual VXLAN network
devices be created. These devices are managed by MLX5 PMD and
created/deleted dynamically.
Kernel creates the VXLAN devices and registers VXLAN UDP ports
to be hardware offloaded within the NIC kernel drivers. The
registration process is being performed into context of working
kernel thread and the race conditions might happen.
The VXLAN device is created and success code is returned to calling
application, but the UDP port registration process is not completed
yet and the next applied rule might be rejected by the driver with
ENOSUP code. This patch adds some timeout for new created devices,
allowing port registration process to be completed. The waiting
is performed once after device been created and first rule is being
applied.
Fixes: 95a464cecc ("net/mlx5: add E-switch VXLAN tunnel devices management")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
When tc rule is created via Netlink message application
can provide the unique rule value which can be accepted
by the kernel. Than rule is managed with this assigned
handle. It was found that kernel can reject the proposed
handle and assign its own handle value, the rule control
is lost, because application uses its initially prorosed
rule handle and knows nothing about handle been repleced.
The kernel can assign handle automatically, the application
can get the assigned handle value by specifying NLM_F_ECHO
flag in Netlink message when rule is being created. The
kernel sends back the full descriptor of rule and handle
can be retrieved from and stored by application for further
rule management.
Fixes: 57123c00c1 ("net/mlx5: add Linux TC flower driver for E-Switch flow")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch modifies function mlx5_flow_validate_action_rss(), to
prevent the setting of rule with rss action, but without specifying
any queues.
For example:
flow create 0 ingress pattern end actions rss queues end / end
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
During port start, function mlx5_ctrl_flow_vlan() is called to create
default ingress flow rules.
For specific use-cases, a port can be used for Tx only.
In such case, number of Rx queues can be set to 0 to save resources,
hence the default ingress rules are irrelevant.
This patch modifies function mlx5_ctrl_flow_vlan() to avoid the
creation of the default ingress rules when number of Rx queues is 0.
It also includes update of validation functions for relevant actions,
mlx5_flow_validate_action_queue() and mlx5_flow_validate_action_rss(),
to prevent creation of flow rules with these actions when number of Rx
queues is 0.
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
In rdma-core library IBV_WQ_FLAG_RX_END_PADDING is renamed to
IBV_WQ_FLAGS_PCI_WRITE_END_PADDING. Way to query the capability is also
changed.
Fixes: 43e9d9794c ("net/mlx5: support upstream rdma-core")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Reviewed-by: Erez Ferber <erezf@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Rx packet padding is supposed to be set by an environment variable -
MLX5_PMD_ENABLE_PADDING, but it has been missing for some time by mistake.
Rather than using such a variable, a PMD parameter (rxq_pkt_pad_en) is
added instead.
Fixes: a1366b1a2b ("net/mlx5: add reference counter on DPDK Rx queues")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Reviewed-by: Erez Ferber <erezf@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
On replenishing Rx buffers for vectorized Rx, mbuf->buf_addr isn't needed
to be accessed as it is static and easily calculated from the mbuf address.
Accessing the mbuf content causes unnecessary load stall and it is worsened
on ARM.
Fixes: 545b884b1d ("net/mlx5: fix buffer address posting in SSE Rx")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch adds the type-of-service and time-to-live IP header
fields validation on E-Switch, both for match pattern and
VXLAN encapsulation action IP header itesm. The E-Switch flows
will use the common mlx5_flow_validate_item_ipv4/6 routines
with added extra parameter, specifying the supported fields
mask.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch adds the type-of-service and time-to-live IP header
fields support on E-Switch. There match pattern for both fields
with masking is added. Also these fields can be set for VXLAN
tunnel encapsulation header.
This issue is critical for some Open VSwitch configuration
on overlayed (tunneled) networks, where the tos field can be
inherited from outer header to inner header.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch is a preparation for adding the type-of-service and
time-to-live IP header fields support on E-Switch. There are
two types of keys added - one for match pattern, other for
tunnel encapsulation header.
This issue is critical for some Open VSwitch configuration
on overlayed (tunneled) networks, where the tos field can be
inherited from outer header to inner header.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
RH7.2 with kernel 3.10.0-327 does not support VXLAN
devices metadata and IFLA_VXLAN_COLLECT_METADATA
key is neither defined nor supported. We must specify
VNI parameter, which will be actually ignored by kernel,
applied rules will be processed by mlx5 kernel driver
and the actual VNI from rules will be used.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Current design uses the VXLAN virtual devices attached
to outer network interface for decapsulation. Kernel
allows to use non-attached devices, so now we can create
not attached device and use it both for encapsulation
and decapsulation. Devices management becomes simpler,
less VXLAN devices are created and used.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The VXLAN encapsulation neigh/local rules will use
the new introduced structure, which keeps the
rules lists, related to specified outer interface,
instead of attached VTEP structure. It allows us to
unbind VTEP structure from keeping the rules for
interface.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Currently the VXLAN encapsulation neigh/local rules
are stored in the list contained in the VTEP device
structure. Encapsulation VTEP device is attached to
outer interface and stored rules are related to this
underlying interface. We are going to use unattached
VXLAN devices for encapsulation (kernel does not use
attached interface to find egress one), so we should
introduce the structure to keep interface related
neigh/local rules instead of VTEP structure. This
patch introduces internal tcf_irule structure, and
its create/delete methods.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch removes unnecessary local varialbles and optimizes
local and neigh encapsulation rules search.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch add support for inner and outer ethernet types for the
E-Switch Flows with tunnels. Inner and outer ethernet type match
can be specified with ethernet items, vlan items, or implicitly
deduced from IP address items. The tcm_info field in Netlink message
tcm structure is filled always with outer protocol.
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch updates the validation routine for the E-Switch Flows.
The ethernet type field can be specified within inner and outer
tunnel ethernet items, by vlan item or implicitly deduced from
IP address items. The validation routine checks all these items
and their combinations for mutual compatibility issues and possible
conflicts.
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch updates the translation routine for the E-Switch Flows.
Inner tunnel pattern items are translated into Netlink message,
support for tunnel inner IP addresses (v4 or v6), IP protocol,
and TCP and UDP ports is added.
We are going to support Flows matching with outer tunnel items
and not containing the explicit tunnel decap action (this one
might be drop, redirect or table jump, for exapmle).
So we can not rely on presence of tunnel decap action in the
list to decide whether the Flow is for tunnel, instead we will
use the presence of tunnel item. Item translation is rebound
to presence of tunnel items, instead of relying on decap action.
There is no way to tell kernel driver the outer address type
(IPv4 or IPv6) but specify the address flower key. The outer
address key is put on Netlink with zero mask if there is no
RTE item is specified in the list.
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch updates the validation routine for the E-Switch Flows.
The inner/outer item flags are added and set correctly, the
validation routine will accept and check the inner items
which follow the tunnel item (like VNI).
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch removes unnecessary outer tunnel parameters check in the
validation routine for the E-Switch Flows. IPv4/IPv6 may have any
spec and mask, and transferred to tc without changes, all checks
are performed by kernel.
We are going to support Flows matching with outer tunnel items
and not containing the explicit tunnel decap action (this one
might be drop, redirect or table jump, for exapmle). So we can
not rely on presence of tunnel decap action in the list to decide
whether the Flow is for tunnel, instead we will use the presence
of tunnel item (like RTE_FLOW_ITEM_TYPE_VXLAN) in the item list.
The tunnel pattern checks within Flow validation routine are
rebound to presence of tunnel item. VXLAN decap action checks
for presence of VXLAN VNI item.
The tunnel UDP item is checked at the point of processing the tunnel
item (i.e. VXLAN). We can not perform UDP item check as tunnel once
UDP item encountered in the list, because it is not known yet whether
the tunnel item follows. The pointer to UDP item is saved and
checked as outer ones if tunnel item found.
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The libraries provided by rdma-core may be statically linked
if enabling CONFIG_RTE_IBVERBS_LINK_STATIC in the make-based build.
If CONFIG_RTE_BUILD_SHARED_LIB is disabled, the applications
will embed the mlx PMDs with ibverbs and the mlx libraries.
If CONFIG_RTE_BUILD_SHARED_LIB is enabled,
the mlx PMDs will embed ibverbs and the mlx libraries.
Support with meson may be added later.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Rename options CONFIG_RTE_LIBRTE_MLX4_DLOPEN_DEPS and
CONFIG_RTE_LIBRTE_MLX5_DLOPEN_DEPS to a single option
CONFIG_RTE_IBVERBS_LINK_DLOPEN.
Rename meson option enable_driver_mlx_glue to ibverbs_link.
There was no good reason for setting a different link option
for mlx4 and mlx5. Having a single common option makes it
easier to understand and unify make and meson systems.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This commit adds counters support when creating flows via direct
verbs. The implementation uses devx interface in order to create
query and delete the counters.
This support requires MLNX_OFED_LINUX-4.5-0.1.0.1 installation.
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch adds glue functions for operations:
- dv_open_device.
- devx object create, destroy, query and modify.
- devx general command
The new operations depend on HAVE_IBV_DEVX_OBJ.
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This commit fixes the logic for searching and allocating a shared
counter in mlx5_flow_verbs.
Now only the shared counters in the counters list are checked for
a match and not all the counters as before.
Fixes: 84c406e745 ("net/mlx5: add flow translate function")
Cc: stable@dpdk.org
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
RTE_MBUF_INDIRECT() is replaced with RTE_MBUF_CLONED() and removed.
This macro was deprecated in release 18.05 when EXT_ATTACHED_MBUF was
introduced.
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
This commit includes the add of:
- ConnectX-6 device ID
- ConnectX-6 SRIOV device ID
Signed-off-by: Wisam Jaddo <wisamm@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch implements the set of actions to support offload
of packet header modifications to MLX5 NIC.
Implementation is based on RFC [1].
[1] http://mails.dpdk.org/archives/dev/2018-November/119971.html
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
When MPRQ mempool is freed, the pointer stored in priv structure must be
reset to null. Otherwise, the mempool can be freed again if the port is
restarted.
Fixes: 7d6bf6b866 ("net/mlx5: add Multi-Packet Rx support")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Function mlx5_ctrl_flow_vlan() is used to set the rss rule in
MLX5 PMD, using priv->reta_idx_n as number of Rx queues.
This number is passed to mlx5_flow_validate_action_rss(), which
attempts to access the Rx queues at priv->rxqs.
In case priv->rxqs_n is 0, priv->rxqs is empty, and
mlx5_flow_validate_action_rss() will crash with segmentation fault.
priv->reta_idx_n can never be 0, even if priv->rxqs_n is set to 0.
But when priv->rxqs_n is set to 0, setting the rss rule is invalid.
This patch updates mlx5_ctrl_flow_vlan(), if priv->rxqs_n is 0 the
function will fail with EINVAL errno.
Fixes: 8086cf08b2 ("net/mlx5: handle RSS hash configuration in RSS flow")
Cc: stable@dpdk.org
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
tso and vlan parameters were removed from the signature
of txq_mbuf_to_swp function.
The documentation of the function was not updated accordingly.
Remove the tso and vlan documentation to match the function signature.
Fixes: 8f6d9e13a9 ("net/mlx5: remove redundant checks")
Cc: stable@dpdk.org
Signed-off-by: Asaf Penso <asafp@mellanox.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
The imissed counters (number of packets dropped because the queues were
full) were actually reported through xstats as "rx_out_of_buffer"
but was not reported through stats.
Following a recent discussion on the ML, as there is no way to tell the
user if a counter is implemented or not, this should be considered a
bug. For example, user looking at imissed will think the packets are
lost before reaching the device.
Signed-off-by: Tom Barbette <barbette@kth.se>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The VLAN push action on E-Switch supports only 802.1Q (0x8100)
and 802.1AD (0x88A8) Tag Protocol ID (TPID) insertions. The
parameter check for RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN action
is added.
Fixes: 57123c00c1 ("net/mlx5: add Linux TC flower driver for E-Switch flow")
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The TCA_FLOWER_KEY_VLAN_ETH_TYPE should be specified for the E-Switch
Flows with VLAN and L3 pattern items in the Netlink messages. The patch
adds missing flower key to the messages. This patch partially reverts to
the code smashed by http://patches.dpdk.org/patch/47781
Fixes: 251e8d02cf ("net/mlx5: add VXLAN to flow translate routine")
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Update the mlx5_flow_validate_item_mpls() function to allow
MPLS over IP, UDP, and GRE.
Modify the flow_dv_validate() function with the new logic introduced
in previous patch of this series: set new variable last_item
after each validation, update item_flags after each item iteration.
The new variable last_item is sent to mlx5_flow_validate_item_mpls()
and used to validate the MPLS item.
Same change implemented in flow_verbs_validate().
Update the mlx5_flow_validate_item_mpls() function to verify that
device configuration has mpls_en set to 1.
This code was added earlier this year, but unintentionaly removed
in recent versions.
Fixes: 84c406e745 ("net/mlx5: add flow translate function")
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The support in MPLS on this flow engine was overlooked. It's absence is
critical because there are required actions for MPLS which can be done
only with the DV engine.
To set correctly the MPLS filter, we need to reason about the flow item
before the MPLS (UDP, GRE or other).
To do that, a new variable last_item was added and updated after each
translation. the full item flags are updated after each item iteration.
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Unnecessary volatile attribute keeps compiler from further optimizing the
code and this results in a little performance drop (~2%). Because of memory
barriers, it is safe to remove.
Fixes: 6bf10ab69b ("net/mlx5: support 32-bit systems")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
There's some performance drop due to extra condition checks on the
datapath. Checking for external memory registration should be consolidated
to the existing bottom-half.
Fixes: 7e43a32ee0 ("net/mlx5: support externally allocated static memory")
Cc: stable@dpdk.org
Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The flow_dv_query() just returns -ENOTSUP value and does not
set provided error parameter structure, that crashes the
port_flow_query(). The patch fixes flow_db_query(), now it
sets an error parameter structure.
Fixes: 684dafe795 ("net/mlx5: add flow query abstraction interface")
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch fixes compilation errors with meson and the clang
compiler caused by some of the struct members not being
initialized.
```
../drivers/net/mlx5/mlx5_mr.c:345:37: error: missing field 'end'
initializer [-Werror,-Wmissing-field-initializers]
struct mlx5_mr_cache entry = { 0, };
^
../drivers/net/mlx5/mlx5_mr.c:389:36: error: missing field 'end'
initializer [-Werror,-Wmissing-field-initializers]
struct mlx5_mr_cache ret = { 0, };
^
../drivers/net/mlx5/mlx5_mr.c:691:35: error: missing field 'end'
initializer [-Werror,-Wmissing-field-initializers]
struct mlx5_mr_cache ret = { 0, };
^
```
The compilation errors reproduce with
clang version 3.4.2 (tags/RELEASE_34/dot2-final) on RHEL.
Fixes: e1114ff6a5 ("net/mlx5: support e-switch flow count action")
Fixes: db48f9db5d ("net/mlx5: support new flow counter API")
Fixes: 974f1e7ef1 ("net/mlx5: add new memory region support")
Fixes: 65c9d24170 ("net/mlx5: enable loopback by configured mode")
Fixes: 87011737b7 ("mlx5: add software counters")
Cc: stable@dpdk.org
Signed-off-by: Ali Alnubani <alialnu@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
This patch adds the in_hw flag check for tc flower filter rules.
If rule is applied without skip_sw flag set driver should check
whether the in_hw flag is set after rule applying. If no in_hw
flag set found it means the rule is not hardware offloaded and
error should be returned to application.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
The tc flower filter rules are used to control E-switch from
the application side. In order to gain garanteed rule hardware
offload the skip_sw flag should be specified while applying the
rule. But some tc rules is rejected by kernel if skip_sw flag
is set by design. Currently this regards VXLAN tunneling rules,
which are applied to special VXLAN virtual devices. Albeit these
rules are applied with skip_sw flag reset kernel tries to
perform hardware offload. If kernel succeeded the in_hw flag
is set in rule properties and application should check this
flag to get know whether the applied rule is actually hardware
offloaded.
This patch prepares for the rule flags query and check in_hw flag.
The driver checks only the rules with skip_sw flag reset, so we
need to test the actual flags of rule is being applied. The pointer
to flags field into translated rule is introduced and flags are
tested directly in the rule body. It is more reliable than save and
check flags copy.
Also patch swaps the flow_tcf_apply() and flow_tcf_remove(),
we are going to call flow_tcf_remove() from flow_tcf_appy() if
no in_hw flag set found in applied rule.
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
While receiving the Netlink reply messages we should stop at DONE
or ACK message. The existing implementation stops at DONE message
or if no multiple message flag set ( NLM_F_MULTI). It prevents
the single query requests from working, these requests send the
single reply message without multi-message flag followed by
ACK message. This patch fixes receiving part of Netlink
communication routine.
Fixes: 6e74990b34 ("net/mlx5: update E-Switch VXLAN netlink routines")
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
