Currently, there is no flow aging check and age-out event callback
mechanism for mlx5 driver, this patch implements it. It's included:
- Splitting the current counter container to aged or no-aged container
since reducing memory consumption. Aged container will allocate extra
memory to save the aging parameter from user configuration.
- Aging check and age-out event callback mechanism based on current
counter. When a flow be checked aged-out, RTE_ETH_EVENT_FLOW_AGED
event will be triggered to applications.
- Implement the new API: rte_flow_get_aged_flows, applications can use
this API to get aged flows.
Signed-off-by: Dong Zhou <dongz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
The maximal supported header modifications number of a single modify
context on the root table cannot be queried from firmware directly.
It is a fixed value of 16 in the latest releases. In the validation
stage, PMD driver should ensure that no more than 16 header modify
actions exist in a single context.
In some old firmware releases, the supported value is 8. PMD driver
should try its best to create the flow. Firmware will return error
and refuse to create the flow if the actions number exceeds the
maximal value.
Fixes: 72a944dba1 ("net/mlx5: fix header modify action validation")
Cc: stable@dpdk.org
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
The header modify actions number supported now has some limitation,
and it is decided by both driver and hardware. If the configuration
is different or the table to insert the flow is different, the result
might be different if the flow contains header modify actions.
Currently, the actual action number could only be calculated in the
later stage called translate, from user specified value to the driver
format. And the action numbers checking is missed in the flow
validation. So PMD will return incorrect result to indicate the
flow actions are valid by rte_flow_validate but then it will fail
when calling rte_flow_create.
Adding some simple checking in the validation will help to get rid
of this incorrect checking. Most of the actions will only consume 1
SW action field except the MAC address and IPv6 address. And from
SW POV, the maximal action fields for these will be consumed even if
only part of such field will be modified because that there is no
mask in the flow actions and the mask will always be all ONEs.
The metering or extra metadata supports will cost one more action.
Fixes: 9597330c68 ("net/mlx5: update modify header action translator")
Cc: stable@dpdk.org
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit focus on flow meter data structures
optimization: mlx5_flow_meter.
Optimize memory consumption of flow meter data structure.
Reorganize flow meter data structure,delete unnecessary
data fields.
Signed-off-by: Wentao Cui <wentaoc@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Currently, the rte flow structure is not fully aligned and has some
bits wasted. The members can be optimized and reorganized to save
memory.
1. The drv_type uses only limited bits, change the type to 2 bits what
it needs.
2. Align the hairpin_flow_id, drv_type, fdir, copy_applied to 32 bits.
As hairpin never uses the full 32 bits.
3. __rte_packed helps tight up the structure memory layout.
The optimization totally helps save 14 bytes for the structure.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit allocates rte flow from indexed memory pool.
Allocate rte flow memory from indexed memory pool helps save more than
MALLOC_ELEM_OVERHEAD bytes memory from rte_malloc().
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
When destroy the flow with RSS, flow can invoke the queues information
from hrxq index table object, since the queue number and list are both
saved to the index table object. No need to save the duplicated data in
rte flow.
Save the RSS description information to the intermediate private data
when create the flow with RSS action helps to save the memory for rte
flow.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit is for mlx5 fdir flow memory optimization.
Currently for the fdir member in rte_flow structure. It saves the fdir
memory pointer directly. As fdir is fading away, use one bit help to
indicate the function in the flow and add the content to an extra list
save the memory for the other widely usage cases.
Signed-off-by: Wentao Cui <wentaoc@mellanox.com>
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Allocate mark copy resource from indexed pool helps rte flow saves the 4
bytes index instead of 8 bytes pointer. For mark copy resource itself,
it helps save MALLOC_ELEM_OVERHEAD bytes from rte_malloc().
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This patch allocate the meter object memory from indexed memory pool
which will help to save the MALLOC_ELEM_OVERHEAD memory taken by
rte_malloc().
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
While flow attaches the meter handle, the meter id can be the unique tag
for the flow to get the meter handle. It's no need for flow to save the
pointer of the meter handle.
Save the meter id instead of pointer helps reduce the size for rte flow
structure.
As the supported maximum meter rule is 4K, uint16_t type is selected for
the meter id.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Currently, the mlx5_flow_handle struct is not fully aligned and has some
bits wasted. The members can be optimized and reorganized to save memory.
1. As metadata and meter is sharing the same flow match id, now the flow
id is limited to 24 bits due to the 8 MSBs are used as for the meter
color. Align the flow id to other bit members to 32 bits to save the
mlx5 flow handle memory.
2. The vlan_vf in struct mlx5_flow_handle_dv was already moved to struct
mlx5_flow_handle. Remove the legacy vlan_vf in struct
mlx5_flow_handle_dv.
3. Reorganize the vlan_vf in mlx5_flow_handle with member SILIST_ENTRY
next to make it align with 8 bytes.
4. Reorganize the header modify in mlx5_flow_handle_dv to ILIST_ENTRY
next to make it align to with bytes.
5. Introduce __rte_pack attribute to make the struct tightly organized.
It will totally save 20 bytes memory for mlx5_flow_handle struct.
For the resource objects which are converted to indexed, align the names
with the prefix of rix_.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
As only limited bits is used in act_flags for flow destroy, it's a bit
expensive to save the whole 64 bits. Move the act_flags out of flow
handle and save the needed bits for flow destroy to save some bytes for
the flow handle data struct.
The fate action type and mark bits are reserved as they will be used in
flow destroy.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Currently, one flow only has one fate action, the fate actions members
in the flow struct can be reorganized as union to save the memory for
flow struct.
This commit reorganizes the fate actions as union, the act_flags helps
to identify the fate action type when flow destroys.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit converts flow dev handle to indexed.
Change the mlx5 flow handle from pointer to uint32_t saves memory for
flow. With million flow, it saves several MBytes memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit converts hrxq to indexed.
Using the uint32_t index instead of pointer saves 4 bytes memory for the
flow handle. For millions flows, it will save several MBytes of memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit convert jump resource to indexed.
The table data struct is allocated from indexed memory. As it is add in
the hash list, the pointer is still used for hash list search. The index
is added to the table struct, and the pointer in flow handle is decrease
to uint32_t type. For flow without jump flows, it saves 4 bytes memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit converts port id action to indexed.
Using the uint32_t index instead of pointer saves 4 bytes memory for the
flow handle. For millions flows, it will save several MBytes of memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit convert tag resource to indexed.
As tag resources are add in the hash list, to avoid introduce
performance issue and keep the hash list, only the tag resource memory
is allocated from indexed memory. The resources is still added to the
hash list. Add four bytes index in the tag resource struct and change
the tag resources in the flow handle from pointer to uint32_t seems be
no benefit for tag resource, but it saves memory for flows without tag
action. And also for sub flows share one tag action resource.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit converts the push VLAN resource to indexed.
Using the uint32_t index instead of pointer saves 4 bytes memory for the
flow handle. For millions flows, it will save several MBytes of memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit converts the flow encap/decap resource to indexed.
Using the uint32_t index instead of pointer saves 4 bytes memory for the
flow handle. For millions flows, it will save several MBytes of memory.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Currently, the meter suffix table is created and saved in the mlx5
shared struct. It causes the suffix table will never be released
even without any meter rules.
Move the suffix table to meter domain struct to help the suffix table
be released when all the meter rules are destroyed.
Fixes: 46a5e6bc6a ("net/mlx5: prepare meter flow tables")
Cc: stable@dpdk.org
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Currently, DV and verbs counters are both changed to indexed. It means
while creating the flow with counter, flow can save the indexed value to
address the counter.
Save the 4 bytes indexed value in the rte_flow instead of 8 bytes
pointer helps to save memory with millions of flows.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Only the members of flow handle structure will be used when trying
to destroy a flow. Other members of mlx5 device flow resource will
only be used for flow creating, and they could be reused for different
flows.
So only the device flow handle structure needs to be saved for further
usage. This could be separated from the whole mlx5 device flow and
stored with a list for each rte flow.
Other members will be pre-allocated with an array, and an index will
be used to help to apply each device flow to the hardware.
The flow handle sizes of Verbs and DV mode will be different, and
some calculation could be done before allocating a verbs handle.
Then the total memory consumption will less for Verbs when there is
no inbox driver being used.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Common structures used for mlx5 flow creating and destroying are
reorganized in order to separating the parts only for destroying
from all the items.
The "mlx5_flow" will contain the common items of DV and Verbs flow,
specific items for DV / Verbs only. These items will only be used
when creating a flow.
At the end of "mlx5_flow", a nested structure "mlx5_flow_handle"
located. It contains all the items used both for creating and
destroying a flow. Also, it consists of common items, and DV / Verbs
specific items.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Meter suffix subflow only has the port id and tag match item, if VLAN
push and set VLAN id actions exist in the suffix subflow, the user
defined VLAN items is required for the actions to set a correct VLAN
id.
Currently, the VLAN item stays in the meter prefix subflow. Without
the VLAN item, VLAN id or pcp will not be inherited.
Actions require the VLAN item as below:
RTE_FLOW_ACTION_TYPE_OF_PUSH_VLAN
RTE_FLOW_ACTION_TYPE_OF_SET_VLAN_VID
Add a private VLAN item to copy the user defined VLAN item to the meter
suffix subflow, so the suffix subflow will have the chance to get the
correct original VLAN id and pcp value from the VLAN item.
Fixes: 9ea9b049a9 ("net/mlx5: split meter flow")
Cc: stable@dpdk.org
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
The encapsulation and decapsulation actions are divided into 2 types:
L2 and L3.
In order to configure L3 xcapsulation actions the user should use both
RAW_DECAP and RAW_ENCAP and setting the appropriated data sizes in
their action configuration structures.
The PMD flow validation wrongly didn't detect the RAW_DECAP
and RAW_ENCAP combination to distinguish between L3_DECAP and L3_ENCAP.
Thus, some xcapsulation related validation failed.
For example, when configuring modify header action before L3_DECAP.
Simplify the xcapsulation defines and fix the L3 xcapsulation detection
using the action configuration data sizes.
By the way, add the hairpin validation in this area.
Fixes: d85c7b5ea5 ("net/mlx5: split hairpin flows")
Fixes: 8ba9eee4ce ("net/mlx5: add raw data encap/decap to Direct Verbs")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
There are RDMA-CORE versions which are not supported multi-table for
some Mellanox mlx5 devices.
Hence, the optimization added in commit [1] which forwards all the FDB
traffic to table 1 cannot be configured.
Make the above optimization optional:
Do not fail when either table 1 cannot be created or the jump rule
(all =>jump to table 1) is not configured successfully.
In this case, all the flows will be configured to table 0.
[1] commit b67b4ecbde ("net/mlx5: skip table zero to improve
insertion rate")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
A new Mellanox vdpa PMD will be added to support vdpa operations by
Mellanox adapters.
This vdpa PMD design includes mlx5_glue and mlx5_devx operations and
large parts of them are shared with the net/mlx5 PMD.
Create a new common library in drivers/common for mlx5 PMDs.
Move mlx5_glue, mlx5_devx_cmds and their dependencies to the new mlx5
common library in drivers/common.
The files mlx5_devx_cmds.c, mlx5_devx_cmds.h, mlx5_glue.c,
mlx5_glue.h and mlx5_prm.h are moved as is from drivers/net/mlx5 to
drivers/common/mlx5.
Share the log mechanism macros.
Separate also the log mechanism to allow different log level control to
the common library.
Build files and version files are adjusted accordingly.
Include lines are adjusted accordingly.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
The id allocated is for the register unique id match. Some registers may
not use the full 32 bits. Add the maximum id to avoid allocate id over
the register restriction.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Compilation massage example:
"dpdk/drivers/net/mlx5/mlx5_flow_dv.c:1087:10: error: comparison of
unsigned enum expression < 0 is always false
[-Werror,-Wtautological-compare]
if (reg < 0)
~~~ ^ ~
"
enum modify_reg holds only non-negative integers and in some places in
the code it was used to be compared with negative value, hence
compilation was failed.
Change all thus places to use integer instead of enum modify_reg.
Fixes: 3e8edd0ef8 ("net/mlx5: update metadata register ID query")
Fixes: 55deee1715 ("net/mlx5: extend flow mark support")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
PUSH_VLAN and POP_VLAN actions flags were added to ENCAP_ACTIONS
and DECAP_ACTIONS bit masks, respectively.
This is incorrect, because VLAN actions are considered as 'modify
header' actions, not as 'packet reformat' actions.
This patch removes the PUSH_VLAN and POP_VLAN actions flags from
ENCAP_ACTIONS and DECAP_ACTIONS bit masks.
Fixes: 9aee7a8418 ("net/mlx5: support push flow action on VLAN header")
Fixes: b41e47da25 ("net/mlx5: support pop flow action on VLAN header")
Cc: stable@dpdk.org
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
In the root table, there is some limitation of total number of header
modify actions, 16 or 8 for each. But in other tables, there is no
such strict limitation. In an IPv6 case, the IP fields modifying
will occupy more actions than that in IPv4, so the total support
number should be increased in order to support as many actions as
possible for an IPv6 + TCP packet.
And in the meanwhile, the memory consumption should also be taken
into consideration because sometimes only several actions are needed.
The root table checking could also be done in low layer driver and
the error code will be returned if the actions number is over the
maximal supported value.
Fixes: 0e9d000276 ("net/mlx5: check maximum modify actions number")
Cc: stable@dpdk.org
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This patch adds to MLX5 PMD support of matching on GTP item,
fields msg_type and teid, according to RFC [1].
GTP item validation and translation functions are added and called.
GTP tunnel type is added to supported tunnels.
[1] http://mails.dpdk.org/archives/dev/2019-December/152799.html
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
Rx hash fields were copied from input parameter into TIR attributes
directly, with no translation. As result the copied value was wrong.
This patch adds translation of value from input bitmap to the
appropriate format.
Fixes: dc9ceff73c ("net/mlx5: create advanced RxQ via DevX")
Cc: stable@dpdk.org
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Bit-masks MLX5_FLOW_LAYER_IPV4_LRO and MLX5_FLOW_LAYER_IPV6_LRO were
added during work on LRO.
They are not used anywhere in current code.
This patch removes these redundant definitions.
Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
The layer bits should be unique otherwise layer info will be
interpreted wrongly.
Fixes: 70d84dc797 ("net/mlx5: add internal tag item and action")
Fixes: 55deee1715 ("net/mlx5: extend flow mark support")
Signed-off-by: Xiaoyu Min <jackmin@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Tag action for flow mark/flag could be reused by different flows.
When creating a new flow with mark, the existing tag resources will
be traversed in order to confirm if the action is already created.
If only one linked list is used, the searching rate will drop
significantly with the number of tag actions increasing.
By using a hash lists table, it will speed up the searching process
and in the meanwhile, the memory consumption won't be large if only
a small number tag action resources are created(compared to other
hash table implementations). The list heads array size could be
optimized with some extendable hash table in the future.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Matchers are created on the specific table. If a single linked list
is used to store these, then the finding process might be the
bottleneck when there are a lot of different flow matchers on a
huge amount of tables. The matchers could be move into the table
data resource structure in order to reduce the comparison times
when finding.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Jump object is associated with table object, so there is no need to
use a single linked list to store it. All the jump objects could be
put together with related flow tables.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
In the current flow tables organization, arrays are used. This is
fast for searching, creating related object that will be used in
flow creation. But it introduces some limitation to the table index.
Then we can reorganize the flow tables information with hash list.
When using hash list, there is no need to maintain three arrays for
NIC TX, RX and FDB tables object information.
This attribute could be used together with the table ID to generate
a 64-bits key that is unique for the hash list insertion, lookup and
deletion.
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
When the port is closed or program exits ungraceful, the meter rulers
should be flushed after the flow destroyed.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
In the meter flow split, metadata flow will be as the sub flow of meter
suffix flow. In meter suffix flow, there is already a unique id tag
exist as for the meter prefix and suffix flow match.
Make metadata feature and meter both share the unique id tag for match.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
The flow with meter action will be divided to three sub flows, the
prefix flow, the meter flow and the suffix flow.
For these three sub flows, as the prefix flow and meter flow have the
meter color match. In order the make the packet from prefix flow to
match with the suffix flow, an extra metadata register is allocated.
The prefix flow will add a unique id to the register and the suffix
flow matches on that unique id.
As RSS will also divided the flow to several sub flows, flow with meter
will be divided as the sub flow of the RSS sub flows if have as below:
Original flow ->
RSS sub flow 1 ->
Meter sub flow 1 (Contain three sub flows.)
RSS sub flow 2 ->
Meter sub flow 2 (Contain three sub flows.)
......
RSS sub flow n ->
Meter sub flow n (Contain three sub flows.)
The metadata feature flow will be split as the sub flow of the meter
suffix flow.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Add the meter attach and detach for the flow create.
When create the flow with meter, first try to find any created meter
action matches the flow meter id. If the meter action is already
created, just attach to it and increase the ref_cnt. If not, create
one.
For the dettach, decrease the ref_cnt, destroy the meter action while
the ref_cnt decreased to zero.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Add count action to meter for metering packet statistics. All the
packets be colored and dropped will be recorded.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Expose the flow counter management mechanism for other components to
use.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
This commit add meter enable and disable supoort.
New internal functions in rte_mtr_ops callback:
1. meter_enable()
2. meter_disable()
The meter_enable() enables the meter action and the meter_disable()
disables the meter action.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
This commit add the basic meter operations for meter create and destroy.
New internal functions in rte_mtr_ops callback:
1. create()
2. destroy()
The create() callback will create the corresponding flow rules on the
meter table.
The destroy() callback destroys the flow rules on the meter table.
Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
