To support multi-thread flow insertion, this patch converts push VLAN
action cache list to thread safe cache list.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this patch convert port id
action cache list to thread safe cache list.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this path converts matcher list
to use thread safe cache list API.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit applies the cache linked list to Rx queue to make it thread
safe.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
When create shared RSS hrxq, the hrxq will be created directly, no hrxq
will be reused.
In this case, add the shared RSS hrxq to the queue list is redundant.
And it also hurts the generic queue lookup.
This commit avoids add the shared RSS hrxq to the queue list.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
New API of linked list for cache:
- Optimized for small amount cache list.
- Optimized for read-most list.
- Thread safe.
- Since number of entries are limited, entries allocated by API.
- For dynamic entry size, pass 0 as entry size, then the creation
callback allocate the entry.
- Since number of entries are limited, no need to use indexed pool to
allocate memory. API will remove entry and free with mlx5_free.
- Search API is not supposed to be used in multi-thread.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this patch updates flow header
reformat action list to use thread safe hash list with write-most mode.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this patch updates metadata copy
flow list to use thread safe hash list.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
After non-cache mode feature was implemented, the flows can only be
created when port started. No need to check if the mreg flows are
created in port stopped status, and apply the mreg flows after port
start will also never happen.
This commit removed the relevant not used mreg copy code.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this patch updates flow modify
action list to use thread safe hash list with write-most mode.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion, this patch updates flow tag list
to use thread safe hash list with write-most mode.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
All table, tag, header modify, header reformat are supported only on DV
mode. For the OFED version doesn't support these, create the related
redundant DV resources waste the memory.
Add the code section in the HAVE_IBV_FLOW_DV_SUPPORT macro to avoid the
redundant resources allocation.
Fixes: 2eb4d0107a ("net/mlx5: refactor PCI probing on Linux")
Cc: stable@dpdk.org
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow insertion/removal, this patch uses thread
safe hash list API for flow table cache hash list.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
As flow table is shared between the ports in the same shared IB device,
flow table may be created by one port and released by other port.
Currently, the tunnel offloading active check in flow table release is
based on the port which release the flow table. Since the flow table
create port and release port may have different tunnel offloading
configuration, it will cause invalid tunnel offloading release or
tunnel offloading resource leaks.
Add the flow table tunnel offloading attribute to indicate the flow
table has tunnel offloading resource or not to avoid wrong tunnel
offloading operation.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
In order to support hash list concurrent access, adding next:
1. List level read/write lock.
2. Entry reference counter.
3. Entry create/match/remove callback.
4. Remove insert/lookup/remove function which are not thread safe.
5. Add register/unregister function to support entry reuse.
For better performance, lookup function uses read lock to
allow concurrent lookup from different thread, all other hash list
modification functions uses write lock which blocks concurrent
modification and lookups from other thread.
The exact objects change will be applied in the next patches.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit creates the global drop action for flows instead of
maintain it in flow insertion time. The uniqueu global drop action
makes it thread safe.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit creates the global default miss action instead of maintain
it in flow insertion time. This makes the action to be thread safe.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit changes the jump action in table to be created with table
creation in advanced. In this case, the jump action is safe to be used
in multiple thread. The jump action will be destroyed when table is not
used anymore and released.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit protects the VLAN VM workaround area using a spinlock
in multiple-thread flow insertion to make it thread safe.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This commit adds the spinlock for the meter action to make it be thread
safe. Atomic reference counter in all is not enough as the meter action
should be created synchronized with reference counter increment. With
only atomic reference counter, even the counter is increased, the action
may still not be created.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To support multi-thread flow operations, this patch introduces list lock
for the rte_flow list manages all the rte_flow handlers.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
The ID generation API used an integer pool to save released ID, To
support multiple flow, it has to be enhanced to be thread safe.
Indexed pool could be used to generate unique ID by setting size of pool
entry to zero. Since bitmap is used, an extra benefits is saving memory
to about one bit per entry. Further more indexed pool could be thread
safe by enabling lock.
This patch leverages indexed pool to generate ID, removes
unused ID generating API.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
To make indexed pool to be used as ID generator, this patch allows entry
size to be zero.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Hairpin flow matching required a unique flow ID for matching.
This patch reuses flow ID as hairpin flow ID, this will save some code
to generate a separate hairpin ID, also saves flow memory by removing
hairpin ID.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
As part of multi-thread flow support, this patch moves flow intermediate
data to thread specific, makes them a flow workspace. The workspace is
allocated per thread, destroyed along with thread life-cycle.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
As mlx5 PMD is changed to be thread safe, all the flow-related
sub-objects inside the PMD should be thread safe. This commit
changes the index memory pools' lock configuration to be enabled.
That makes the index pool be thread safe.
Signed-off-by: Suanming Mou <suanmingm@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Rx RSS hash offload should be controlled by the user
and should not be enforced by RSS multi-queue Rx mode.
Fixes: 8b945a7f7d ("drivers/net: update Rx RSS hash offload capabilities")
Cc: stable@dpdk.org
Author: Andrew Rybchenko <arybchenko@solarflare.com>
Signed-off-by: Alexander Kozyrev <akozyrev@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
When creating a flow, the rule itself might not take effort
immediately once the function call returns with success. It would
take some time to let the steering synchronize with the hardware.
If the application wants the packet to be sent to hit the flow after
it is created, this flow sync API can be used to clear the steering
HW cache to enforce next packet hits the latest rules.
For TX, usually the NIC TX domain and/or the FDB domain should be
synchronized depends in which domain the flow is created.
The application could also try to synchronize the NIC RX and/or the
FDB domain for the ingress packets.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In rdma-core, the "mlx5dv_dr_domain_sync" function was already
provided. It is used to flush the rule submission queue. The wrapper
function in the glue layer is added for using this.
It only supports DR flows right now the same as domain creating and
destroying functions.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The rte_atomic API is deprecated and needs to be replaced with
C11 atomic builtins. Use the relaxed ordering for RTE flow tables.
Enforce Acquire/Release model for managing DevX pools.
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The rte_atomic API is deprecated and needs to be replaced with
C11 atomic builtins. Use the relaxed ordering for RxQ/TxQ refcounts.
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The rte_atomic API is deprecated and needs to be replaced with
C11 atomic builtins. Use __atomic_add_fetch instead of
rte_atomic32_add_return to generate a Netlink sequence number.
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The rte_atomic API is deprecated and needs to be replaced with
C11 atomic builtins. Use the relaxed ordering for mlx5 mallocs.
Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The Tx queue stop\start operations update the HW state of the Tx queue
object. The stop API should update the state from ready to reset in
order to stop any queue traffic and the start API should update the
state from reset to ready in order to open the traffic path.
The start API wrongly tried to change the state from ready to ready what
caused a failure in FW on the current state validation.
Replace ready to ready command by reset to ready command in the Tx start
API.
Fixes: 161d103b23 ("net/mlx5: add queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Asaf Penso <asafp@nvidia.com>
Update the flow verbs error message to "item type X not supported",
when it is not supported,
instead of a generic error message "item not supported".
Signed-off-by: Li Zhang <lizh@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
This adds the ConnectX-7 and Bluefield-3 device ids to the list of
supported Mellanox devices that run the MLX5 PMDs.
The devices is still in development stage.
Signed-off-by: Raslan Darawsheh <rasland@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
The fields ``has_vlan`` and ``has_more_vlan`` were added in rte_flow by
patch [1].
Using these fields, the application can match all the VLAN options by
single flow: any, VLAN only and non-VLAN only.
Add the support for the fields.
By the way, add the support for QinQ packets matching.
VLAN\QinQ limitations are listed in the driver document.
[1] https://patches.dpdk.org/patch/80965/
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Dekel Peled <dekelp@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
Hairpin between two ports will be supported by mlx5 PMD.
The supported scenarios and limitations are listed in "mlx5.rst".
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In the current implementation, the hairpin flow will be split into
two flows implicitly if there is some action that only belongs to the
Tx part. A Tx device flow will be inserted by the mlx5 PMD itself.
In hairpin between two ports, the explicit Tx flow mode will be the
only one to be supported. It is not the appropriate behavior to
insert a Tx flow into another device implicitly. The application
could create any flow as it likes and has full control of the user
flows. Hairpin flows will have no difference from standard flows and
the application can decide how to chain Rx and Tx flows together.
Even in the single port hairpin, this explicit Tx flow mode could
also be supported.
When checking if the hairpin needs to be split, it will just return
if the hairpin queue is with "tx_explicit" attribute. Then in the
following steps for validation and translation, the code path will
be the same as that for standard flows.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In the current implementation of the single port hairpin, there is
an implicit splitting process for actions. When inserting a hairpin
flow, all the actions will be included with the ingress attribute.
The flow engine will check and decide which actions should be moved
into the TX flow part, e.g., encapsulation, VLAN push.
In some NICs, some actions can only be done in one direction. Since
the hairpin flow will be split into two parts, such validation will
be skipped.
With the hairpin explicit TX flow mode, no splitting is needed any
more. The hairpin flow may have no big difference from a standard
flow (except the queue). The application should take full charge of
the actions and the flow engine should validate the hairpin flow in
the same way as other flows.
In the meanwhile, a new internal API is added to get the hairpin
configuration. This will bypass the useless atomic operation to save
the CPU cycles.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In single port hairpin mode, after the queues are configured during
start up. The binding process will be enabled automatically in the
port start phase and the default control flow for egress will be
created.
When switching to two ports hairpin mode, the auto binding process
should be skipped if there is no TX queue with the peer RX queue on
the same device, and it should be skipped also if the queues are
configured with manual bind attribute.
If the explicit TX flow rule mode is configured or hairpin is
between two ports, the default control flows for TX queues should
not be created.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In real-life business, one device could be attached and detached
dynamically. The hairpin configuration of this port to/from all the
other ports should be enabled and disabled accordingly.
The RTE ethdev lib and PMD should provide this ability to get the
peer ports list in case that the application doesn't save it. It is
recommended that the size of the array to save the port IDs is as
large as the "RTE_MAX_ETHPORTS" to have the maximal capacity.
The order of the peer port IDs may be different from that during
hairpin queues set in the initialization stage. The peer port ID
could be the same as the current device port ID when the hairpin
peer ports contain itself - the single port hairpin.
The application should check the ports' status and decide if the
peer port should be bound / unbound when starting / stopping the
current device.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In order to support hairpin between two ports, mlx5 PMD needs to
implement the functions and provide them as the function pointers.
The bind and unbind functions are executed per port pairs. All the
hairpin queues between the two ports should have the same attributes
during queues setup. Different configurations among queue pairs from
the same ports are not supported. It is allowed that two ports only
have one direction hairpin.
In order to set up the connection between two queues, peer Rx queue
HW information must be fetched via the internal RTE API and the queue
information could be used to modify the SQ object. Then the RQ object
will be modified with the Tx queue HW information. The reverse
operation is not supported right now.
When disconnecting the queues pair, SQ and RQ object should be reset
without any peer HW information. The unbinding operation will try to
disconnect all Tx queues from the port from the Rx queues of the peer
port.
Tx explicit mode attribute will be saved and used when creating a
hairpin flow.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In the current implementation of single port mode hairpin, the peer
queue should belong to the same port of the current queue. When the
two ports hairpin mode is introduced, such checking should be removed
to make the hairpin queue setup execute successfully since it is not
an invalid condition, if the Tx port and Rx port are not the same.
In the meanwhile, different devices could have different queue
configurations. The queues number of peer port is unknown to the
current device. The checking should be removed also.
If the Tx and Rx port IDs of a hairpin peer are different, only the
manual binding and explicit Tx flows are supported. Or else, the four
combinations of modes could be supported. The mode attributes
consistency checking will be done when connecting the queue with its
peer queue.
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
In the refactor of mlx5 common layer, the PCI driver name to the RTE
device was changed from "net_mlx5" to "mlx5_pci". The string of name
"mlx5_pci" is used directly in the structure rte_pci_driver.
In the past, a macro "MLX5_DRIVER_NAME" is used instead of any direct
string, and now it is missing. The functions that use
"MLX5_DRIVER_NAME" will get some mismatch, e.g mlx5_eth_find_next.
It needs to use this macro again in all code to make everything get
aligned.
Fixes: 8a41f4decc ("common/mlx5: introduce layer for multiple class drivers")
Cc: stable@dpdk.org
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Reviewed-by: Parav Pandit <parav@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
The spinlock was trying to protect scenarios where rx_queue stop/start
could be initiated dynamically. Assigning bnxt_dummy_recv_pkts and
bnxt_dummy_xmit_pkts immediately to avoid concurrent access of mbuf in Rx
and cleanup path should help achieve the same result.
Fixes: 14255b3515 ("net/bnxt: fix queue start/stop operations")
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Somnath Kotur <somnath.kotur@broadcom.com>
Signed-off-by: Rahul Gupta <rahul.gupta@broadcom.com>
PMD supports thread-safe flow operations. Set the
RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE dev_flag to indicate this info
to the application. rte_flow API functions can avoid using its
own mutex for safe multi-thread flow handling.
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reset mbuf->data_off before handing the Rx packet to the application.
We were not doing this in the TPA path. It can cause applications
using this field for post processing to work incorrectly.
Fixes: 0958d8b643 ("net/bnxt: support LRO")
Cc: stable@dpdk.org
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Lance Richardson <lance.richardson@broadcom.com>
LRO aka TPA and jumbo frame support uses aggregation ring for placing
Rx buffers. These features can generate multiple Rx completions for a
single Rx packet. Increase size of Rx Completion Queue to handle TPA
and aggregation ring events.
Fixes: daef48efe5 ("net/bnxt: support set MTU")
Cc: stable@dpdk.org
Signed-off-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Reviewed-by: Qingmin Liu <qingmin.liu@broadcom.com>
Reviewed-by: Randy Schacher <stuart.schacher@broadcom.com>
VXLAN decap offload can happen in stages. The offload request may
not come as a single flow request rather may come as two flow offload
requests F1 & F2. This patch is adding support for this two stage
offload design. The match criteria for F1 is O_DMAC, O_SMAC,
O_DST_IP, O_UDP_DPORT and actions are COUNT, MARK, JUMP. The match
criteria for F2 is O_SRC_IP, O_DST_IP, VNI and inner header fields.
F1 and F2 flow offload requests can come in any order. If F2 flow
offload request comes first then F2 can’t be offloaded as there is
no O_DMAC information in F2. In this case, F2 will be deferred until
F1 flow offload request arrives. When F1 flow offload request is
received it will have O_DMAC information. Using F1’s O_DMAC, driver
creates an L2 context entry in the hardware as part of offloading F1.
F2 will now use F1’s O_DMAC to get the L2 context id associated with
this O_DMAC and other flow fields that are cached already at the time
of deferring F2 for offloading. F2s that arrive after F1 is offloaded
will be directly programmed and not cached.
Signed-off-by: Venkat Duvvuru <venkatkumar.duvvuru@broadcom.com>
Reviewed-by: Kishore Padmanabha <kishore.padmanabha@broadcom.com>
Reviewed-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
