69 Commits

Author SHA1 Message Date
Gregory Etelson
750ff30a8f net/mlx5: fix tunnel flow priority
PMD flow priority is different from application flow priority.  Flow
rules with higher match granularity assigned higher PMD priority. Also
PMD splits internally RSS flows according to flow RSS layer.

Final PMD flow rule priority derived from the last match item network
level, after PMD adjusts flow rule, where L4 match gets the highest
priority and L2 the lowest.

The patch adjusts tunnels flow rule priority calculation for PMDs
running verb API.

Introduce MLX5_TUNNEL_PRIO_GET macro.

Fixes: 4a78c88e3bae ("net/mlx5: fix Verbs flow tunnel")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
9d60f54569 common/mlx5: remove inclusion of Verbs header files
Several source files include Verbs header files as in (1). These source
files will not compile under non-Linux operating systems. This commit
removes this inclusion in two cases:

Case 1: There is no usage of ibv_* or mlx5dv_* symbols in the source
file so the inclusion in (1) can be safely removed.

Case 2: Verbs symbols are used. Please note the inclusion in (1) already
appears in file linux/mlx5_glue.h (which represents the interface
to the rdma-core library). Therefore, replace (1) in the source file
with (2).  Under non-Linux operating systems - file mlx5_glue.h will not
include (1).

(1)
 #include <infiniband/verbs.h>
 #include <infiniband/mlx5dv.h>

(2)
 #include <mlx5_glue.h>

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
3eca5f8a61 net/mlx5: move flow priority discovery to Verbs file
Function calls mlx5_flow_adjust_priority() and
mlx5_flow_discover_priorities() are Verbs based. Move them from file
mlx5_flow.c to file mlx5_flow_verbs.c

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Suanming Mou
83c2047c5f net/mlx5: convert control path memory to unified malloc
This commit allocates the control path memory from unified malloc
function.

The objects be changed:

1. hlist;
2. rss key;
3. vlan vmwa;
4. indexed pool;
5. fdir objects;
6. meter profile;
7. flow counter pool;
8. hrxq and indirect table;
9. flow object cache resources;
10. temporary resources in flow create;

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Dekel Peled
341c894104 net/mlx5: rename Verbs flow to generic name
As part of the effort to support DPDK on Windows and other OS,
rename from IB related name to generic name.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-07 23:38:26 +02:00
Suanming Mou
ac79183dc6 net/mlx5: optimize free counter lookup
Currently, when allocate a new counter, it needs loop the whole
container pool list to get a free counter.

In the case with millions of counters allocated, and all the pools
are empty, allocate the new counter will still need to loop the
whole container pool list first, then allocate a new pool to get a
free counter. It wastes the cycles during the pool list traversal.

Add a global free counter list in the container helps to get the free
counters more efficiently.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-30 14:52:30 +02:00
Ophir Munk
9138989036 net/mlx5: rename ib in names
Renames in this commit:
mlx5_ibv_list -> mlx5_dev_ctx_list
mlx5_alloc_shared_ibctx -> mlx5_alloc_shared_dev_ctx
mlx5_free_shared_ibctx -> mlx5_free_shared_dev_ctx
mlx5_ibv_shared_port -> mlx5_dev_shared_port
ibv_port -> dev_port

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-16 19:21:07 +02:00
Suanming Mou
d71d5b949c net/mlx5: fix Verbs counter pool allocation
When create the Verbs flows with counter, randomly SEGSEV will also
comes. The reason is that the counter pool memory is not allocated
sufficiently and initialized correctly in Verbs case.

As the mlx5_flow_counter array member is moved out of the counter pool
struct, the counter pool memory layout currently contain implicitly
with mlx5_flow_counter, mlx5_age_param(if the pool is an age pool),
mlx5_flow_counter_ext(if the pool is a none batch pool). When allocate
the pool memory, the pool size should be calculated based on the pool
type accordingly.

Currently, for Verbs counter pool, both mlx5_flow_counter and
mlx5_flow_counter_ext need to be taken into account in the pool size.
And the pool type should also be initialized as CNT_POOL_TYPE_EXT.

This patch add the missing size and type for the Verbs counter pool.

Fixes: 8d93c830e450 ("net/mlx5: modify ext-counter memory allocation")

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-05-18 20:35:57 +02:00
Matan Azrad
5af61440dd net/mlx5: fix flow counter container resize
The design of counter container resize used double buffer algorithm in
order to synchronize between the query thread to the control thread.
When the control thread detected resize need, it created new bigger
buffer for the counter pools in a new container and change the container
index atomically.
In case the query thread had not detect the previous resize before a new
one need was detected by the control thread, the control thread returned
EAGAIN to the flow creation API used a COUNT action.

The rte_flow API doesn't allow unblocked commands and doesn't expect to
get EAGAIN error type.

So, when a lot of flows were created between 2 different periodic
queries, 2 different resizes might try to be created and caused EAGAIN
error.
This behavior may blame flow creations.

Change the synchronization way to use lock instead of double buffer
algorithm.

The critical section of this lock is very small, so flow insertion
rate should not be decreased.

Fixes: ebbac312e448 ("net/mlx5: resize a full counter container")
Cc: stable@dpdk.org

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-05-18 20:35:57 +02:00
Raslan Darawsheh
8a2e026add net/mlx5: fix matching for UDP tunnels with Verbs
When creating flow rule with zero specs it will cause
matching all UDP packets like following:
 eth / ipv4 / udp / vxlan / end
Such rule will match all udp packets.

This change the behavior to match the dv flow engine
which will automatically set the match on relative
outer UDP port if the user didn't specify any.

Fixes: 84c406e74524 ("net/mlx5: add flow translate function")
Cc: stable@dpdk.org

Signed-off-by: Raslan Darawsheh <rasland@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-05-11 22:27:39 +02:00
Ophir Munk
0efc99bed3 net/mlx5: fix flow rules with Verbs
The flow_verbs_translate() function accumulates hash fields while
iterating through the flow items (SRC_IPV4, DST_IPV4, SRC_IPV6,
DST_IPV6, SRC_PORT_TCP, DST_PORT_TCP, SRC_PORT_UDP, DST_PORT_UDP).
Before this commit the dev_flow handle structure was reused in each new
flow_verbs_translate() call, however the dev_flow->hash_fields variable
was not reset before each call. As a result hash_fields from previous
calls remained present in the current flow which lead to invalid
combinations (e.g.  simultaneous IPv4 and IPv6 specs). This scenario
happens for example in the next flows sequence, when running in verbs
mode (dv_flow_en=0).

flow create 0 ingress group 0 pattern eth / ipv4 / end <rss actions>
flow create 0 ingress group 0 pattern eth / ipv6 / end <rss actions>

The fix is to reset dev_flow->hash_fields in flow_verbs_prepare().

Fixes: e7bfa3596a0a ("net/mlx5: separate the flow handle resource")

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-05-05 15:54:27 +02:00
Dong Zhou
fa2d01c87d net/mlx5: support flow aging
Currently, there is no flow aging check and age-out event callback
mechanism for mlx5 driver, this patch implements it. It's included:
- Splitting the current counter container to aged or no-aged container
  since reducing memory consumption. Aged container will allocate extra
  memory to save the aging parameter from user configuration.
- Aging check and age-out event callback mechanism based on current
  counter. When a flow be checked aged-out, RTE_ETH_EVENT_FLOW_AGED
  event will be triggered to applications.
- Implement the new API: rte_flow_get_aged_flows, applications can use
  this API to get aged flows.

Signed-off-by: Dong Zhou <dongz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-05-05 15:54:27 +02:00
Dong Zhou
8d93c830e4 net/mlx5: modify ext-counter memory allocation
Currently, the counter pool needs 512 ext-counter memory for no batch
counters, it's allocated separately by once, behind the 512
basic-counter memory. This is not easy to get ext-counter pointer by
corresponding basic-counter pointer. This is also no easy for expanding
some other potential additional type of counter memory.

So, need allocate every one of ext-counter and basic-counter together,
as a single piece of memory. It's will be same for further additional
type of counter memory. In this case, one piece of memory contains all
type of memory for one counter, it's easy to get each type memory by
using offsetting.

Signed-off-by: Dong Zhou <dongz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-05-05 15:54:27 +02:00
Bing Zhao
72a944dba1 net/mlx5: fix header modify action validation
The header modify actions number supported now has some limitation,
and it is decided by both driver and hardware. If the configuration
is different or the table to insert the flow is different, the result
might be different if the flow contains header modify actions.
Currently, the actual action number could only be calculated in the
later stage called translate, from user specified value to the driver
format. And the action numbers checking is missed in the flow
validation. So PMD will return incorrect result to indicate the
flow actions are valid by rte_flow_validate but then it will fail
when calling rte_flow_create.

Adding some simple checking in the validation will help to get rid
of this incorrect checking. Most of the actions will only consume 1
SW action field except the MAC address and IPv6 address. And from
SW POV, the maximal action fields for these will be consumed even if
only part of such field will be modified because that there is no
mask in the flow actions and the mask will always be all ONEs.

The metering or extra metadata supports will cost one more action.

Fixes: 9597330c6844 ("net/mlx5: update modify header action translator")
Cc: stable@dpdk.org

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 22:28:06 +02:00
Suanming Mou
e745f90007 net/mlx5: optimize flow RSS struct
When destroy the flow with RSS, flow can invoke the queues information
from hrxq index table object, since the queue number and list are both
saved to the index table object. No need to save the duplicated data in
rte flow.

Save the RSS description information to the intermediate private data
when create the flow with RSS action helps to save the memory for rte
flow.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Suanming Mou
77749adab9 net/mlx5: reorganize flow handle struct
Currently, the mlx5_flow_handle struct is not fully aligned and has some
bits wasted. The members can be optimized and reorganized to save memory.

1. As metadata and meter is sharing the same flow match id, now the flow
   id is limited to 24 bits due to the 8 MSBs are used as for the meter
   color.  Align the flow id to other bit members to 32 bits to save the
   mlx5 flow handle memory.

2. The vlan_vf in struct mlx5_flow_handle_dv was already moved to struct
   mlx5_flow_handle. Remove the legacy vlan_vf in struct
   mlx5_flow_handle_dv.

3. Reorganize the vlan_vf in mlx5_flow_handle with member SILIST_ENTRY
   next to make it align with 8 bytes.

4. Reorganize the header modify in mlx5_flow_handle_dv to ILIST_ENTRY
   next to make it align to with bytes.

5. Introduce __rte_pack attribute to make the struct tightly organized.

It will totally save 20 bytes memory for mlx5_flow_handle struct.

For the resource objects which are converted to indexed, align the names
with the prefix of rix_.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Suanming Mou
488d13abdc net/mlx5: optimize action flags in flow handle
As only limited bits is used in act_flags for flow destroy, it's a bit
expensive to save the whole 64 bits. Move the act_flags out of flow
handle and save the needed bits for flow destroy to save some bytes for
the flow handle data struct.

The fate action type and mark bits are reserved as they will be used in
flow destroy.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Suanming Mou
6fc183924b net/mlx5: reorganize fate actions as union
Currently, one flow only has one fate action, the fate actions members
in the flow struct can be reorganized as union to save the memory for
flow struct.

This commit reorganizes the fate actions as union, the act_flags helps
to identify the fate action type when flow destroys.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Suanming Mou
b88341ca35 net/mlx5: convert flow dev handle to indexed
This commit converts flow dev handle to indexed.

Change the mlx5 flow handle from pointer to uint32_t saves memory for
flow. With million flow, it saves several MBytes memory.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Suanming Mou
772dc0eb83 net/mlx5: convert hrxq to indexed
This commit converts hrxq to indexed.

Using the uint32_t index instead of pointer saves 4 bytes memory for the
flow handle. For millions flows, it will save several MBytes of memory.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:09 +02:00
Bing Zhao
3ac3d8234b net/mlx5: fix index when creating flow
When creating a flow, usually the creating routine is called in
serial. No parallel execution is supported right now. The same
function will be called only once for a single flow creation.

But there is a special case that the creating routine will be called
nested. If the xmeta feature is enabled and there is FLAG / MARK in
the actions list, some metadata reg copy flow needs to be created
before the original flow is applied to the hardware.
In the flow non-cached mode, resources only for flow creation will
not be saved anymore. The memory space is pre-allocated and reused
for each flow. A global index for each device is used to indicate
the memory address of the resources. If the function is called in a
nested mode, then the index will be reset and make everything get
corrupted.

To solve this, a nested index is introduced to save the position for
the original flow creation. Currently, only one level nested call
of the flow creating routine is supported.

Fixes: e7bfa3596a0a ("net/mlx5: separate the flow handle resource")

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-04-21 13:57:07 +02:00
Suanming Mou
826b8a8732 net/mlx5: split flow counter struct
Currently, the counter struct saves both the members used by batch
counters and none batch counters. The members which are only used
by none batch counters cost 16 bytes extra memory for batch counters.
As normally there will be limited none batch counters, mix the none
batch counter and batch counter members becomes quite expensive for
batch counter. If 1 million batch counters are created, it means 16 MB
memory which will not be used by the batch counters are allocated.

Split the mlx5_flow_counter struct for batch and none batch counters
helps save the memory.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-04-21 13:57:07 +02:00
Suanming Mou
956d5c74d7 net/mlx5: optimize flow counter handle type
Currently, DV and verbs counters are both changed to indexed. It means
while creating the flow with counter, flow can save the indexed value to
address the counter.

Save the 4 bytes indexed value in the rte_flow instead of 8 bytes
pointer helps to save memory with millions of flows.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-04-21 13:57:07 +02:00
Suanming Mou
c3d3b14099 net/mlx5: change verbs counter allocator to indexed
This is part of the counter optimize which will save the indexed counter
id instead of the counter pointer in the rte_flow.

Place the verbs counter into the container pool helps the counter to be
indexed correctly independent with the raw counter.

The counter pointer in rte_flow will be changed to indexed value after
the DV counter is also changed to indexed.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-04-21 13:57:07 +02:00
Bing Zhao
e7bfa3596a net/mlx5: separate the flow handle resource
Only the members of flow handle structure will be used when trying
to destroy a flow. Other members of mlx5 device flow resource will
only be used for flow creating, and they could be reused for different
flows.
So only the device flow handle structure needs to be saved for further
usage. This could be separated from the whole mlx5 device flow and
stored with a list for each rte flow.
Other members will be pre-allocated with an array, and an index will
be used to help to apply each device flow to the hardware.
The flow handle sizes of Verbs and DV mode will be different, and
some calculation could be done before allocating a verbs handle.
Then the total memory consumption will less for Verbs when there is
no inbox driver being used.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-04-21 13:57:05 +02:00
Bing Zhao
c42f44bd1b net/mlx5: reorganize flow structures
Common structures used for mlx5 flow creating and destroying are
reorganized in order to separating the parts only for destroying
from all the items.
The "mlx5_flow" will contain the common items of DV and Verbs flow,
specific items for DV / Verbs only. These items will only be used
when creating a flow.
At the end of "mlx5_flow", a nested structure "mlx5_flow_handle"
located. It contains all the items used both for creating and
destroying a flow. Also, it consists of common items, and DV / Verbs
specific items.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-04-21 13:57:05 +02:00
Dekel Peled
6bd5efb23d net/mlx5: fix mask used for IPv6 item validation
The nic_mask struct in function mlx5_flow_validate_item_ipv6()
includes hop_limits field.
This is redundant since matching on this item is not supported using
Verbs flow engine (dv_flow_en=0).

(Separate patch will be sent to handle this issue for DV flow engine.)

Setting a rule including match on this field will fail:
testpmd> flow create 0 ingress pattern eth / ipv6 hop is 1 / end actions
drop / end
port_flow_complain(): Caught PMD error type 1 (cause unspecified):
hardware refuses to create flow: Operation not supported

This patch removes the redundant field from nic_mask, to ensure that
such flow rules will be rejected by PMD validation:
testpmd> flow create 0 ingress pattern eth / ipv6 hop is 1 / end actions
drop / end
port_flow_complain(): Caught PMD error type 13 (specific pattern item):
cause: 0x7fffffffa2c8, mask enables non supported bits: Operation not
supported

The related lines in function flow_verbs_translate_item_ipv6() are
removed as well, since they are redundant.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Fixes: 84c406e74524 ("net/mlx5: add flow translate function")
Cc: stable@dpdk.org

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-03-18 10:21:41 +01:00
Alexander Kozyrev
8e46d4e18f common/mlx5: improve assert control
Use the MLX5_ASSERT macros instead of the standard assert clause.
Depends on the RTE_LIBRTE_MLX5_DEBUG configuration option to define it.
If RTE_LIBRTE_MLX5_DEBUG is enabled MLX5_ASSERT is equal to RTE_VERIFY
to bypass the global CONFIG_RTE_ENABLE_ASSERT option.
If RTE_LIBRTE_MLX5_DEBUG is disabled, the global CONFIG_RTE_ENABLE_ASSERT
can still make this assert active by calling RTE_VERIFY inside RTE_ASSERT.

Signed-off-by: Alexander Kozyrev <akozyrev@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-02-05 09:51:21 +01:00
Matan Azrad
7b4f1e6bd3 common/mlx5: introduce common library
A new Mellanox vdpa PMD will be added to support vdpa operations by
Mellanox adapters.

This vdpa PMD design includes mlx5_glue and mlx5_devx operations and
large parts of them are shared with the net/mlx5 PMD.

Create a new common library in drivers/common for mlx5 PMDs.
Move mlx5_glue, mlx5_devx_cmds and their dependencies to the new mlx5
common library in drivers/common.

The files mlx5_devx_cmds.c, mlx5_devx_cmds.h, mlx5_glue.c,
mlx5_glue.h and mlx5_prm.h are moved as is from drivers/net/mlx5 to
drivers/common/mlx5.

Share the log mechanism macros.
Separate also the log mechanism to allow different log level control to
the common library.

Build files and version files are adjusted accordingly.
Include lines are adjusted accordingly.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-02-05 09:51:20 +01:00
Dekel Peled
70faf9ae0a net/mlx5: unify validation of drop action
According to PRM: "Drop action is mutually-exclusive with any other
action, except for Count action".
In current code this limitation is checked separately in validation
function of each action.

This patch removes the discrete checks, and adds a single check common
for all actions.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-02-05 09:51:20 +01:00
Dekel Peled
50d23b9ace net/mlx5: fix verbs flow counter query
Function flow_verbs_counter_query() was recently modified.
The new 'if' condition uses a pointer to flow counter-set.
This pointer is valid only if flow contains a count action.

This patch adds check to verify the pointer is valid.

Fixes: d85c7b5ea59f ("net/mlx5: split hairpin flows")

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2019-11-20 17:36:05 +01:00
Viacheslav Ovsiienko
e205c95f74 net/mlx5: refactor flow structure
Some rte_flow fields which are local to subflows have been moved to
mlx5_flow structure. RSS attributes are grouped by mlx5_flow_rss structure.
tag_resource is moved to mlx5_flow_dv structure.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2019-11-11 14:23:01 +01:00
Xiaoyu Min
fba3213015 net/mlx5: improve flow item IP validation
Currently PMD doesn't check whether the user specified ethernet type is
conflicting with the followed IPv4/IPv6 items, which leads to HW refuse
to create rule, for example:

  ... pattern eth type is 0x86dd / ipv4 / end ...

ethernet type is IPv6 but IPv4 is following, this should be validated
as failure and report corresponding error in detail.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Cc: stable@dpdk.org

Signed-off-by: Xiaoyu Min <jackmin@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
2019-11-08 23:15:05 +01:00
Ori Kam
d85c7b5ea5 net/mlx5: split hairpin flows
Since the encap action is not supported in RX, we need to split the
hairpin flow into RX and TX.

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2019-11-08 23:15:04 +01:00
Dekel Peled
b67b4ecbde net/mlx5: skip table zero to improve insertion rate
E-switch tables one and above provide higher insertion rate
than table zero, as well as enhanced functionality.

This patch adds a mechanism to utilize these advantages, by creating
a default rule on port start, which directs all packets from e-switch
table zero to table one.
Other flow rules, requested for group n, will be created in
e-switch table n+1.
Jump action to e-switch group n will be created to group n+1.

Utility function mlx5_flow_group_to_table() is added to translate the
rte_flow group value to HW table value, and is called by PMD flow
engine on flow rule validation and creation.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2019-09-20 10:19:41 +02:00
Viacheslav Ovsiienko
dfedf3e3f9 net/mlx5: add workaround for VLAN in virtual machine
On some virtual setups (particularly on ESXi) when we have SR-IOV and
E-Switch enabled there is the problem to receive VLAN traffic on VF
interfaces. The NIC driver in ESXi hypervisor does not setup E-Switch
vport setting correctly and VLAN traffic targeted to VF is dropped.

The patch provides the temporary workaround - if the rule
containing the VLAN pattern is being installed for VF the VLAN
network interface over VF is created, like the command does:

  ip link add link vf.if name mlx5.wa.1.100 type vlan id 100

The PMD in DPDK maintains the database of created VLAN interfaces
for each existing VF and requested VLAN tags. When all of the RTE
Flows using the given VLAN tag are removed the created VLAN interface
with this VLAN tag is deleted.

The name of created VLAN interface follows the format:

  evmlx.d1.d2, where d1 is VF interface ifindex, d2 - VLAN ifindex

Implementation limitations:

- mask in rules is ignored, rule must specify VLAN tags exactly,
  no wildcards (which are implemented by the masks) are allowed

- virtual environment is detected via rte_hypervisor() call,
  and the type of hypervisor is checked. Currently we engage
  the workaround for ESXi and unrecognized hypervisors (which
  always happen on platforms other than x86 - it means workaround
  applied for the Flow over PCI VF). There are no confirmed data
  the other hypervisors (HyperV, Qemu) need this workaround,
  we are trying to reduce the list of configurations on those
  workaround should be applied.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-08-06 17:42:12 +02:00
Matan Azrad
5158260917 net/mlx5: allow implicit LRO flow
When a user configures LRO in the port offloads, he probably wants each
TCP packet will have a chance to open an LRO session.

The PMD wasn't configure LRO in the flow TIR if the flow is not
explicitly configured TCP item despite the flow included TCP traffic.

For example, the next flows were not LRO offloaded:
pattern eth / end, pattern eth / ip / end, pattern eth / ipv6 / end.

Enable LRO configuration for all the TIRs if LRO is configured in the
port.

No performance impact for non-LRO traffic in these TIRs.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2019-07-29 16:54:27 +02:00
Dekel Peled
940f0a1d07 net/mlx5: support LRO with single RxQ object
Implement LRO support using a single RQ object per DPDK RxQ.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2019-07-23 14:31:36 +02:00
Matan Azrad
5382d28c21 net/mlx5: accelerate DV flow counter transactions
The DevX interface exposes a new feature to the PMD that can allocate a
batch of counters by one FW command. It can improve the flow
transaction rate (with count action).

Add a new counter pools mechanism to manage HW counters in the PMD.
So, for each flow with counter creation the PMD will try to find a free
counter in the PMD pools container and only if there is no a free
counter, it will allocate a new DevX batch counters.

Currently we cannot support batch counter for a group 0 flow, so
create a 2 container types, one which allocates counters one by
one and one which allocates X counters by the batch feature.

The allocated counters objects are never released back to the HW
assuming the flows maximum number will be close to the actual value of
the flows number.
Later, it can be updated, and dynamic release mechanism can be added.

The counters are contained in pools, each pool with 512 counters.
The pools are contained in counter containers according to the
allocation resolution type - single or batch.
The cache memory of the counters statistics is saved as raw data per
pool.
All the raw data memory is allocated for all the container in one
memory allocation and is managed by counter_stats_mem_mng structure
which registers all the raw memory to the HW.
Each pool points to one raw data structure.

The query operation is in pool resolution which updates all the pool
counter raw data by one operation.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-07-23 14:31:35 +02:00
Olivier Matz
24ac604ef7 net: add rte prefix to IP defines
Add 'RTE_' prefix to defines:
- rename IPv4( as RTE_IPv4(.
- rename IPV4_MAX_PKT_LEN as RTE_IPV4_MAX_PKT_LEN.
- rename IPV4_HDR_IHL_MASK as RTE_IPV4_HDR_IHL_MASK.
- rename IPV4_IHL_MULTIPLIER as RTE_IPV4_IHL_MULTIPLIER.
- rename IPV4_HDR_DF_SHIFT as RTE_IPV4_HDR_DF_SHIFT.
- rename IPV4_HDR_MF_SHIFT as RTE_IPV4_HDR_MF_SHIFT.
- rename IPV4_HDR_FO_SHIFT as RTE_IPV4_HDR_FO_SHIFT.
- rename IPV4_HDR_DF_FLAG as RTE_IPV4_HDR_DF_FLAG.
- rename IPV4_HDR_MF_FLAG as RTE_IPV4_HDR_MF_FLAG.
- rename IPV4_HDR_OFFSET_MASK as RTE_IPV4_HDR_OFFSET_MASK.
- rename IPV4_HDR_OFFSET_UNITS as RTE_IPV4_HDR_OFFSET_UNITS.
- rename IPV4_ANY as RTE_IPV4_ANY.
- rename IPV4_LOOPBACK as RTE_IPV4_LOOPBACK.
- rename IPV4_BROADCAST as RTE_IPV4_BROADCAST.
- rename IPV4_ALLHOSTS_GROUP as RTE_IPV4_ALLHOSTS_GROUP.
- rename IPV4_ALLRTRS_GROUP as RTE_IPV4_ALLRTRS_GROUP.
- rename IPV4_MAX_LOCAL_GROUP as RTE_IPV4_MAX_LOCAL_GROUP.
- rename IPV4_MIN_MCAST as RTE_IPV4_MIN_MCAST.
- rename IPV4_MAX_MCAST as RTE_IPV4_MAX_MCAST.
- rename IS_IPV4_MCAST as RTE_IS_IPV4_MCAST.
- rename IPV6_HDR_FL_SHIFT as RTE_IPV6_HDR_FL_SHIFT.
- rename IPV6_HDR_TC_SHIFT as RTE_IPV6_HDR_TC_SHIFT.
- rename IPV6_HDR_FL_MASK as RTE_IPV6_HDR_FL_MASK.
- rename IPV6_HDR_TC_MASK as RTE_IPV6_HDR_TC_MASK.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2019-05-24 13:34:46 +02:00
Olivier Matz
35b2d13fd6 net: add rte prefix to ether defines
Add 'RTE_' prefix to defines:
- rename ETHER_ADDR_LEN as RTE_ETHER_ADDR_LEN.
- rename ETHER_TYPE_LEN as RTE_ETHER_TYPE_LEN.
- rename ETHER_CRC_LEN as RTE_ETHER_CRC_LEN.
- rename ETHER_HDR_LEN as RTE_ETHER_HDR_LEN.
- rename ETHER_MIN_LEN as RTE_ETHER_MIN_LEN.
- rename ETHER_MAX_LEN as RTE_ETHER_MAX_LEN.
- rename ETHER_MTU as RTE_ETHER_MTU.
- rename ETHER_MAX_VLAN_FRAME_LEN as RTE_ETHER_MAX_VLAN_FRAME_LEN.
- rename ETHER_MAX_VLAN_ID as RTE_ETHER_MAX_VLAN_ID.
- rename ETHER_MAX_JUMBO_FRAME_LEN as RTE_ETHER_MAX_JUMBO_FRAME_LEN.
- rename ETHER_MIN_MTU as RTE_ETHER_MIN_MTU.
- rename ETHER_LOCAL_ADMIN_ADDR as RTE_ETHER_LOCAL_ADMIN_ADDR.
- rename ETHER_GROUP_ADDR as RTE_ETHER_GROUP_ADDR.
- rename ETHER_TYPE_IPv4 as RTE_ETHER_TYPE_IPv4.
- rename ETHER_TYPE_IPv6 as RTE_ETHER_TYPE_IPv6.
- rename ETHER_TYPE_ARP as RTE_ETHER_TYPE_ARP.
- rename ETHER_TYPE_VLAN as RTE_ETHER_TYPE_VLAN.
- rename ETHER_TYPE_RARP as RTE_ETHER_TYPE_RARP.
- rename ETHER_TYPE_QINQ as RTE_ETHER_TYPE_QINQ.
- rename ETHER_TYPE_ETAG as RTE_ETHER_TYPE_ETAG.
- rename ETHER_TYPE_1588 as RTE_ETHER_TYPE_1588.
- rename ETHER_TYPE_SLOW as RTE_ETHER_TYPE_SLOW.
- rename ETHER_TYPE_TEB as RTE_ETHER_TYPE_TEB.
- rename ETHER_TYPE_LLDP as RTE_ETHER_TYPE_LLDP.
- rename ETHER_TYPE_MPLS as RTE_ETHER_TYPE_MPLS.
- rename ETHER_TYPE_MPLSM as RTE_ETHER_TYPE_MPLSM.
- rename ETHER_VXLAN_HLEN as RTE_ETHER_VXLAN_HLEN.
- rename ETHER_ADDR_FMT_SIZE as RTE_ETHER_ADDR_FMT_SIZE.
- rename VXLAN_GPE_TYPE_IPV4 as RTE_VXLAN_GPE_TYPE_IPV4.
- rename VXLAN_GPE_TYPE_IPV6 as RTE_VXLAN_GPE_TYPE_IPV6.
- rename VXLAN_GPE_TYPE_ETH as RTE_VXLAN_GPE_TYPE_ETH.
- rename VXLAN_GPE_TYPE_NSH as RTE_VXLAN_GPE_TYPE_NSH.
- rename VXLAN_GPE_TYPE_MPLS as RTE_VXLAN_GPE_TYPE_MPLS.
- rename VXLAN_GPE_TYPE_GBP as RTE_VXLAN_GPE_TYPE_GBP.
- rename VXLAN_GPE_TYPE_VBNG as RTE_VXLAN_GPE_TYPE_VBNG.
- rename ETHER_VXLAN_GPE_HLEN as RTE_ETHER_VXLAN_GPE_HLEN.

Do not update the command line library to avoid adding a dependency to
librte_net.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2019-05-24 13:34:45 +02:00
Thomas Monjalon
b7ed955a20 ethdev: deprecate legacy filter API
As stated in the deprecation notice from December 2016,
"the legacy filter API, including rte_eth_dev_filter_supported(),
rte_eth_dev_filter_ctrl() as well as filter types MACVLAN, ETHERTYPE,
FLEXIBLE, SYN, NTUPLE, TUNNEL, FDIR, HASH and L2_TUNNEL, is superseded
by the generic flow API (rte_flow)".

After a long wait of more than two years, the legacy filter API
is marked as deprecated, while still tested with testpmd and
the tep_termination example.

The next step will be to announce a deadline for complete removal.
As preparation of the removal of rte_eth_ctrl.h,
RTE_ETH_FLOW_*, RTE_TUNNEL_TYPE_* and RTE_ETH_HASH_FUNCTION_* definitions
are moved to rte_ethdev.h and rte_flow.h.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
Acked-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
2019-04-19 14:51:54 +02:00
Ori Kam
1183f12f14 net/mlx5: fix RSS validation function
The RSS validation function was missing the verifcation that
if RSS is requested on inner packet, the flow must have tunnel data.

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Cc: stable@dpdk.org

Signed-off-by: Ori Kam <orika@mellanox.com>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
2019-04-19 14:51:54 +02:00
Yongseok Koh
227684feb8 net/mlx5: fix recursive inclusion of header file
mlx5.h includes mlx5_rxtx.h and mlx5_rxtx.h includes mlx5.h recursively.

Signed-off-by: Yongseok Koh <yskoh@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-04-12 11:02:02 +02:00
Dekel Peled
de90612f40 net/mlx5: fix errno typos in comments
Correct typing mistake in several locations:
ernno ==> errno

Fixes: 23c1d42c7138 ("net/mlx5: split flow validation to dedicated function")
Cc: stable@dpdk.org

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-04-05 17:45:22 +02:00
Viacheslav Ovsiienko
1e14090e31 net/mlx5: provide IB port for the object being created
The code is updated to provide IB port index for the Verbs
objects being created - QPs and Verbs Flows.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-03-29 17:25:32 +01:00
Viacheslav Ovsiienko
f048f3d479 net/mlx5: switch to the shared IB device context
The code is updated to use the shared IB device context and
device handles. The IB device context is shared between
reprentors created over the single multiport IB device. All
Verbs and DevX objects will be created within this shared context.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-03-29 17:25:32 +01:00
Thomas Monjalon
dbeba4cf18 net/mlx: prefix private structure
The private structure stored in rte_eth_dev->data->dev_private
was named "struct priv".
In order to ease code browsing, the structure is renamed
"struct mlx[45]_priv".

Cc: stable@dpdk.org

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Yongseok Koh <yskoh@mellanox.com>
2019-03-01 18:17:35 +01:00
Viacheslav Ovsiienko
55c61fa714 net/mlx5: validate TOS and TTL on E-Switch
This patch adds the type-of-service and time-to-live IP header
fields validation on E-Switch, both for match pattern and
VXLAN encapsulation action IP header itesm. The E-Switch flows
will use the common mlx5_flow_validate_item_ipv4/6 routines
with added extra parameter, specifying the supported fields
mask.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-01-14 17:44:30 +01:00
Moti Haimovsky
5f09e80cf8 net/mlx5: fix shared counter allocation logic
This commit fixes the logic for searching and allocating a shared
counter in mlx5_flow_verbs.
Now only the shared counters in the counters list are checked for
a match and not all the counters as before.

Fixes: 84c406e74524 ("net/mlx5: add flow translate function")
Cc: stable@dpdk.org

Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Acked-by: Shahaf Shuler <shahafs@mellanox.com>
2019-01-14 17:44:29 +01:00