Commit Graph

10126 Commits

Author SHA1 Message Date
Michael Baum
e71e90938b net/mlx5: fix crash in NVGRE item translation
The flow_dv_translate_item_nvgre function add NVGRE item to matcher and
to the value.
It defines a pointer named nvrge_m that receives the item's mask into
it, and then copies some of it to the matcher.

Before copying, it checks for mask validation, and in case the mask is
NULL the function gives it a pointer to rte_flow_item_nvgre_mask.
However, the function calls from the vni mask's field before the check,
and if there is no mask, it actually does dereference to the NULL
pointer and indeed the program crashes with segfault.

Move the call from the vni field to post-validation.

Fixes: cd18e1b72f ("net/mlx5: fix build on Arm")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-30 00:41:23 +02:00
Michael Baum
4868ae8322 net/mlx5: fix initialization of steering registers
The mlx5_flow_action_copy_mreg structure contains a field called src
type enum modify_reg, similarly the mlx5_rte_flow_item_tag field
contains a field called id type enum modify_reg.
The enum modify_reg variable represents different registers in the
system and it also has a field called REG_NONE whose value is 0 which
means that the register does not exist.

The flow_mreg_add_copy_action function sets a variable of struct
mlx5_flow_action_copy_mreg type, and initializes the src field to be 0.
Similarly the flow_create_split_metadata function sets a variable of
struct mlx5_rte_flow_item_tag type and initializes the id field to be 0.
In both functions, they initialize a enum modify_reg type variable with
an int type value while modify_reg has an appropriate field for that
value (REG_NONE).

Replace assigning 0 with REG_NONE in both functions.

Fixes: dd3c774f6f ("net/mlx5: add metadata register copy table")
Fixes: 71e254bc02 ("net/mlx5: split Rx flows to provide metadata copy")
Cc: stable@dpdk.org

Signed-off-by: Michael Baum <michaelba@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-30 00:41:23 +02:00
Suanming Mou
e1293b10de net/mlx5: fix counter query
Currently, the counter query requires the counter ID should start
with 4 aligned. In none-batch mode, the counter pool might have the
chance to get the counter ID not 4 aligned. In this case, the counter
should be skipped, or the query will be failed.

Skip the counter with ID not 4 aligned as the first counter in the
none-batch count pool to avoid invalid counter query. Once having
new min_dcs ID in the poll less than the skipped counters, the
skipped counters will be returned to the pool free list to use.

Fixes: 5382d28c21 ("net/mlx5: accelerate DV flow counter transactions")
Cc: stable@dpdk.org

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-30 00:41:23 +02:00
Suanming Mou
fd143711a6 net/mlx5: separate aging counter pool range
Currently, when allocate the counter or counter based age from group 0,
counter and age may share the same counter dcs ID range. Both age and
pure counter need to sync up with each other's container to check if
the ID range exists and update the min_dcs.

It comes two disadvantages:
1. If the ID range is shared, this counter range will be queried twice
   both from age and pure counter container in 1s.
2. The same range counter check between the two container makes the
   counter allocate sync min_dcs time to time with extra min_dcs
   updating.

This patch avoid the same ID range to be shared when allocate the new
pool. If the same ID range exists in other container, just add the
counter to the other container until get new range which saves the
min_dcs sync up time to time.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-30 00:41:23 +02:00
Raslan Darawsheh
d13f976086 net/mlx5: fix flow items size calculation
flow_dv_get_item_len returns the actual header size of
an rte_flow item.

Changing any of the structs for rte_flow items by adding
or removing some extra fields will break this function.

This fixes the behavior by returning the actual header size
of each item.

Fixes: 34d41b7aa3 ("net/mlx5: add VXLAN encap action to Direct Verbs")
Cc: stable@dpdk.org

Signed-off-by: Raslan Darawsheh <rasland@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-30 00:41:23 +02:00
Ophir Munk
385c19397e net/mlx5: fix premature disabling of interrupt
RXQ interrupts under Linux are based on the epoll mechanism. An expected
order of operations is as follows:
1. Call rte_eth_dev_rx_intr_enable(), to arm the CQ for receiving events
   on data input.
2. Block on rte_epoll_wait() with an array of file descriptors
   representing the CQ events. Upon data arrival the kernel will signal
   an input event on the corresponding CQ fd.
3. Call rte_eth_dev_rx_intr_disable() after the event was received and
   continue in polling mode. The mlx5 implementation of
   rte_eth_dev_rx_intr_disable() is to get the CQ event and ack it.

In practice applications may wake up from rte_epoll_wait() due to
timeout with no event to ack but still call
rte_eth_dev_rx_intr_disable() unconditionally.  In such cases the call
should return EAGAIN (since the file descriptors are non-blocked), as
opposed to EINVAL which indicates a real failure.  In case of EAGAIN the
PMD should not warn on "Unable to disable interrupt on Rx queue".

This commit fixes a earlier commit where the returned value 0 from
function devx_get_event() - was considered an error.

Fixes: 08d1838f64 ("net/mlx5: implement CQ for Rx using DevX API")

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Raslan Darawsheh <rasland@mellanox.com>
2020-07-30 00:41:22 +02:00
Junyu Jiang
f0585e8559 net/ice: fix bytes statistics
This patch fixed the issue that rx/tx bytes overflowed
on 40 bit limitation by enlarging the limitation.

Fixes: a37bde5631 ("net/ice: support statistics")
Cc: stable@dpdk.org

Signed-off-by: Junyu Jiang <junyux.jiang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
2020-07-30 00:41:22 +02:00
Parav Pandit
f6d099d7da common/mlx5: remove class check from class drivers
Now that mlx5_pci PMD checks for enabled classes and performs
probe(), remove() of associated classes, individual class driver
does not need to check if other driver is enabled.

Signed-off-by: Parav Pandit <parav@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-28 19:01:30 +02:00
Parav Pandit
392bf9084d common/mlx5: register class drivers through common layer
Migrate mlx5 net, vdpa and regex PMD to start using mlx5 common class
driver.

Signed-off-by: Parav Pandit <parav@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-28 19:01:30 +02:00
Parav Pandit
8208800163 common/mlx5: avoid class constructor priority
mlx5_common is shared library between mlx5 net, VDPA and regex PMD.
It is better to use common initialization helper instead of using
RTE_PRIORITY_CLASS priority.

Suggested-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Parav Pandit <parav@mellanox.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
2020-07-28 18:52:11 +02:00
Matan Azrad
604052ae53 net/vhost: support queue update
The commit below changed the readiness condition of vhost device to fix
multi-queues issues showed with QEMU versions.

Now, the vhost device is ready when the first queue-pair is ready.
When more queues are being ready, the queue state callback will be
triggered to notify the vhost manager.

In case of Rx interrupt configuration, the vhost driver set the
kickfd queue file descriptor in order to be notified on Rx traffic.

So, when queue becomes ready, the kickfd may be changed and should be
updated in the Rx interrupt structure.

Update kickfd when the queue state callback is invoked.
Also update event notification when it is enabled by the user.

Fixes: d0fcc38f5f ("vhost: improve device readiness notifications")

Suggested-by: Marvin Liu <yong.liu@intel.com>
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-07-21 19:03:24 +02:00
Andrew Rybchenko
6fd44a47a7 net/sfc: do not enforce hash offload in RSS multi-queue
Rx RSS hash offload should be controlled by the user and should
not be enforced by RSS multi-queue Rx mode.

Fixes: 8b945a7f7d ("drivers/net: update Rx RSS hash offload capabilities")
Cc: stable@dpdk.org

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
2020-07-21 18:13:45 +02:00
Andrew Rybchenko
98e2783af3 net/sfc: avoid unnecessary actions on dummy default MAC set
Just an optimization to avoid extra reconfiguration when it
is not actually required.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
2020-07-21 18:13:45 +02:00
Andrew Rybchenko
793f6dc325 net/sfc: remove inclusion of unused headers
Defines and functions from rte_mbuf_ptype.h are not used.

Only libefx types and EF10 register definitions are used.
Native datapaths should be independent from main libefx interface.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
2020-07-21 18:13:45 +02:00
Andrew Rybchenko
268df85bc6 net/sfc/base: improve headers independence
efx_types.h uses defines from efx_annote.h, but does not include the
header. As the result if efx_types.h is included by a driver first,
build fails.

Signed-off-by: Andrew Rybchenko <arybchenko@solarflare.com>
2020-07-21 18:13:45 +02:00
Pavan Nikhilesh
d52a0478bb net/octeontx2: free CQ ring memzone on queue release
Free CQ ring memzone on Rx queue release. This prevents CQ using
incorrect memory size when ring size is reconfigured.

Signed-off-by: Pavan Nikhilesh <pbhagavatula@marvell.com>
Acked-by: Jerin Jacob <jerinj@marvell.com>
2020-07-21 16:04:06 +02:00
Gregory Etelson
750ff30a8f net/mlx5: fix tunnel flow priority
PMD flow priority is different from application flow priority.  Flow
rules with higher match granularity assigned higher PMD priority. Also
PMD splits internally RSS flows according to flow RSS layer.

Final PMD flow rule priority derived from the last match item network
level, after PMD adjusts flow rule, where L4 match gets the highest
priority and L2 the lowest.

The patch adjusts tunnels flow rule priority calculation for PMDs
running verb API.

Introduce MLX5_TUNNEL_PRIO_GET macro.

Fixes: 4a78c88e3b ("net/mlx5: fix Verbs flow tunnel")
Cc: stable@dpdk.org

Signed-off-by: Gregory Etelson <getelson@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Acked-by: Ori Kam <orika@mellanox.com>
2020-07-21 15:46:30 +02:00
Dekel Peled
210008309b net/mlx5: fix VLAN push action on hairpin queue
Push VLAN action is allowed on Tx only, same as encap action.
Flow rules for hairpin queue are created on Rx, and split
by PMD to Rx and Tx rules, according to the above limitation.
In current implementation the encap action is split to Tx rule.
This patch adds the same handling for push-vlan action, as well as
its complementing actions set-vlan-vid and set-vlan-pcp.

Fixes: d85c7b5ea5 ("net/mlx5: split hairpin flows")
Cc: stable@dpdk.org

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Dekel Peled
dd6745da6d net/mlx5: fix VLAN pop with decap action validation
The combination of decap action followed by pop VLAN action is not
fully validated in existing code.

This patch updates the validation function of pop vlan action.
Pop VLAN with preceding Decap requires inner header with VLAN.
Pop VLAN without preceding Decap requires outer header with VLAN.

Fixes: b41e47da25 ("net/mlx5: support pop flow action on VLAN header")
Cc: stable@dpdk.org

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Shy Shyman
038e7fc085 net/mlx5: fix HW counters path in switchdev mode
When debugging performance of a DPDK application the user may
need to view the different statistics of DPDK (for example out_of_buffer)
This can be enabled by using testpmd command 'show port xstats
<port_id>' for example.

The current implementation assumes legacy mode in which the counters
are at <ibdev_path>/<port_id>/hw_counters/<file_name>.
In switchdev mode the counters file is located right after the device
name, hence resides at <ibdev_path>/hw_counters.

The fix tries to open the path in the second location after a failure
to open the file from the first location.

Fixes: 9c0a9eed37 ("net/mlx5: switch to the names in the shared IB context")
Cc: stable@dpdk.org

Signed-off-by: Shy Shyman <shys@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:46:30 +02:00
Viacheslav Ovsiienko
161d103b23 net/mlx5: add queue start and stop
The mlx5 PMD did not support queue_start and queue_stop eth_dev API
routines, queue could not be suspended and resumed during device
operation.

There is the use case when this feature is crucial for applications:

- there is the secondary process handling the queue
- secondary process crashed/aborted
- some mbufs were allocated or used by secondary application
- some mbufs were allocated by Rx queues to receive packets
- some mbufs were placed to send queue
- queue goes to undefined state

In this case there was no reliable way to recovery queue handling
by restarted secondary process but reset queue to initial state
freeing all involved resources, including buffers involved in queue
operations, reset the mbuf pools, and then reinitialize queue
to working state:

- reset mbuf pool, allocate all mbuf to initialize pool into
  safe state after the crush and allow safe mbuf free calls
- stop queue, free all potentially involved mbufs
- reset mbuf pool again
- start queue, reallocate mbufs needed

This patch introduces the queue start/stop feature with some
limitations:

- hairpin queues are not supported
- it is application responsibility to synchronize start/stop
  with datapath routines, rx/tx_burst must be suspended during
  the queue_start/queue_stop calls
- it is application responsibility to track queue usage and
  provide coordinated queue_start/queue_stop calls from
  secondary and primary processes.
- Rx queues with vectorized Rx routine and engaged CQE
  compression are not supported by this patch currently

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:46:30 +02:00
Dekel Peled
08d1838f64 net/mlx5: implement CQ for Rx using DevX API
This patch continues the work to use DevX API for different objects
creation and management.
On Rx control path, the RQ, RQT, and TIR objects can already be
created using DevX API.
This patch adds the support to create CQ for RxQ using DevX API.
The corresponding event channel is also created and utilized using
DevX API.

Signed-off-by: Dekel Peled <dekelp@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
9d60f54569 common/mlx5: remove inclusion of Verbs header files
Several source files include Verbs header files as in (1). These source
files will not compile under non-Linux operating systems. This commit
removes this inclusion in two cases:

Case 1: There is no usage of ibv_* or mlx5dv_* symbols in the source
file so the inclusion in (1) can be safely removed.

Case 2: Verbs symbols are used. Please note the inclusion in (1) already
appears in file linux/mlx5_glue.h (which represents the interface
to the rdma-core library). Therefore, replace (1) in the source file
with (2).  Under non-Linux operating systems - file mlx5_glue.h will not
include (1).

(1)
 #include <infiniband/verbs.h>
 #include <infiniband/mlx5dv.h>

(2)
 #include <mlx5_glue.h>

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
2e86c4e5c7 net/mlx5: refactor multi-process communication
1. The shared data communication between the primary and the secondary
processes is implemented using Linux API. Move the Linux API code under
linux directory (file linux/mlx5_os.c).

2. File net/mlx5/mlx5_mp.c handles requests to the primary and secondary
processes (e.g. start_rxtx, stop_rxtx). It is Linux based so it is moved
under linux (new file linux/mlx5_mp_os.c).

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
ef9ee13f6e net/mlx5: cleanup header file
The cleanup refers to header file mlx5.h.
1. Remove unused prototypes.
2. Move prototypes under their correct title.
3. Change functions to static and remove their prototye from the header
file.

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
98c4b12afa net/mlx5: eliminate dependency on Linux in shared header
This commit eliminates Linux dependencies in shared file mlx5.h.

1. All functions using 'struct ifreq' are moved to file
linux/mlx5_ethdev_os.c such that this struct can be removed from mlx5.h.
2. Function mlx5_set_flags() that uses Linux flags (e.g. IFF_UP) is
changed to static and its prototype is removed from mlx5.h.
3. Remove redundant member verbs_action from 'struct mlx5_priv'.

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
4d18abd130 net/mlx5: wrap Linux promiscuous and multicast functions
This commit adds Linux implementation of routines mlx5_os_set_promisc()
and mlx5_os_set_promisc(). The routines call netlink APIs.

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
ab27cdd93a net/mlx5: refactor Linux MAC operations
Move OS specific MAC operations add, remove, modify VF into file
linux/mlx5_os.c.
Remove unused function mlx5_get_mac().

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
2aba9fc725 net/mlx5: replace Linux specific calls
The following Linux calls are replaced by their matching rte APIs.

mmap ==> rte_mem_map()
munmap == >rte_mem_unmap()
sysconf(_SC_PAGESIZE) ==> rte_mem_page_size()

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Ophir Munk
3eca5f8a61 net/mlx5: move flow priority discovery to Verbs file
Function calls mlx5_flow_adjust_priority() and
mlx5_flow_discover_priorities() are Verbs based. Move them from file
mlx5_flow.c to file mlx5_flow_verbs.c

Signed-off-by: Ophir Munk <ophirmu@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Suanming Mou
50f95b23c9 net/mlx5: add option to configure FCS or decapsulation
There are some limitations on some NICs (at least on ConnectX-6 Dx
and BlueField 2) with supporting FCS (frame checksum) scattering for
the tunnel decapsulated packets.

For the case only one of the features can be supported in the same time,
and the new devarg "decap_en" is introduced to provide the choice to the
users.

If FCS scattering feature is not supposed to be engaged by application,
this new devarg should be specified as "decap_en=0", forcing the FCS
feature enable and rejecting tunnel decap actions in the rte_flow engine.
If FCS scatter is not needed and application supposes to use tunnel
decapsulation in rte_flow, the devarg can be omitted or set to non-zero
value (this is default settings).

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:46:30 +02:00
Suanming Mou
ac3fc732c4 net/mlx5: convert queue objects to unified malloc
This commit allocates the Rx/Tx queue objects from unified malloc
function.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Suanming Mou
2175c4dc62 net/mlx5: convert configuration objects to unified malloc
This commit allocates the miscellaneous configuration objects from the
unified malloc function.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:46:30 +02:00
Suanming Mou
83c2047c5f net/mlx5: convert control path memory to unified malloc
This commit allocates the control path memory from unified malloc
function.

The objects be changed:

1. hlist;
2. rss key;
3. vlan vmwa;
4. indexed pool;
5. fdir objects;
6. meter profile;
7. flow counter pool;
8. hrxq and indirect table;
9. flow object cache resources;
10. temporary resources in flow create;

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Suanming Mou
5522da6b20 net/mlx5: add option to allocate memory from system
Currently, for MLX5 PMD, once millions of flows created, the memory
consumption of the flows are also very huge. For the system with limited
memory, it means the system need to reserve most of the memory as huge
page memory to serve the flows in advance. And other normal applications
will have no chance to use this reserved memory any more. While most of
the time, the system will not have lots of flows, the  reserved huge
page memory becomes a bit waste of memory at most of the time.

By the new sys_mem_en devarg, once set it to be true, it allows the PMD
allocate the memory from system by default with the new add mlx5 memory
management functions. Only once the MLX5_MEM_RTE flag is set, the memory
will be allocate from rte, otherwise, it allocates memory from system.

So in this case, the system with limited memory no need to reserve most
of the memory for hugepage. Only some needed memory for datapath objects
will be enough to allocated with explicitly flag. Other memory will be
allocated from system. For system with enough memory, no need to care
about the devarg, the memory will always be from rte hugepage.

One restriction is that for DPDK application with multiple PCI devices,
if the sys_mem_en devargs are different between the devices, the
sys_mem_en only gets the value from the first device devargs, and print
out a message to warn that.

Signed-off-by: Suanming Mou <suanmingm@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Bing Zhao
d7c49561d3 net/mlx5: add eCPRI flex parser capacity check
If the NIC or the FW does not support the dynamic flex parser,
it will return error when trying to create the parser for eCRPI.
Then it is hard to know the detail error reason of the failure.
Before creating the parser node and the following usage of the
parser, the capacity bit saved in the HCA_CAP could be used to
confirm if the dynamic flex parser is supported.
If no, an error will be returned directly with ENOTSUP to prevent
the following steps to be executed.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:44:36 +02:00
Bing Zhao
1c5064044f net/mlx5: create and destroy eCPRI flex parser
eCPRI protocol has unified format layout for the variants, over
ETH layer (including .1Q) and UDP layer.

The common header of the message has 4 bytes fixed length, and the
message payload layers are different based on the type field. Now
only type #0, #2 and #5 will be supported, and 2 bytes are needed.

When creating the flex parser, the header will be extended to 8
bytes and 2 DW samples are needed. The 1st DW starts from offset 0
and will be used for the type field of the common header. The 2nd
DW starts from offset 4 and will be used for the physical channel
ID, real-time control ID or measurement ID fields.

The parser will be created once a flow with eCPRI item is observed
for the first time. After creating, it will remain in the system
and HW until the device is stopped. Right now, there is no need to
destroy the eCPRI flex parser after the last flow with eCPRI item
is destroyed. This is to get rid of the alternate states of creating
and destroying eCPRI flex parser with a single eCPRI flow.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:44:36 +02:00
Bing Zhao
711aedf187 common/mlx5: add flex parser DevX structures
The structures and other definitions will be used for the dynamic
flex parser creation via Devx command interface. These structures
will be used as some some intermediate variables and input
parameters for the parser creation API.
It is better to keep all members consistent with the PRM definition
even though some of them will not be used.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:44:36 +02:00
Bing Zhao
daa38a8924 net/mlx5: add flow translation of eCPRI header
In the translation stage, the eCPRI item should be translated into
the format that lower layer driver could use. All the fields that
need to match must be in network byte order after translation, as
well as the mask. Since the header in the item belongs to the network
layers stack, and the input parameter of the header is considered to
be in big-endian format already.

Base on the definition in the PRM, the DW samples will be used for
matching in the FTE/STE. Now, the type field and only the PC ID, RTC
ID, and DLY MSR ID of the payload will be supported. The masks should
be 00 ff 00 00 ff ff(00) 00 00 in the network order. Two DWs are
needed to support such matching. The mask fields could be zeros to
support some wildcard rules. But it makes no sense to support the
rule matching only on the payload but without matching type field.

The DW samples should be stored after the flex parser creation for
eCPRI. There is no need to query the sample IDs each time when
creating a flow rule with eCPRI item. It will not introduce
insertion rate degradation significantly.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:44:36 +02:00
Bing Zhao
c7eca23657 net/mlx5: add flow validation of eCPRI header
When creating a flow with eCPRI header item, the validation of it is
mandatory. The detailed limitations are listed below:
  1. Over Ether / VLAN, ethertype must be 0xAEFE.
  2. No tunnel support is described in the specification now.
  3. L3 layer is only supported when L4 is UDP, see #4.
  4. Over TCP is not supported from the specification, and over UDP
     is not supported right now.
  5. Concatenation indicator matching is not supported now.
  6. No need to check the revision.
  7. Only type field in the common header is mandatory, and one byte
     should be matched integrally.
  8. Fields in the message payload header are optional.
  9. Only messages with type #0, #2 and #5 are supported now.

Some limitations are only from software right now, because there is
no need to support all the message types and variants of protocol
stack listed in the specification.

Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
a2854c4de1 net/mlx5: convert Rx timestamps in real-time format
The ConnectX-6DX supports the timestamps in various formats,
the new realtime format is introduced - the upper 32-bit word
of timestamp contains the UTC seconds and the lower 32-bit word
contains the nanoseconds. This patch detects what format is
configured in the NIC and performs the conversion accordingly.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
3b025c0ca4 net/mlx5: provide send scheduling error statistics
The mlx5 PMD exposes the following new introduced
extended statistics counter to report the errors
of packet send scheduling on timestamps:

  - txpp_err_miss_int - rearm queue interrupt was not handled
    was not handled in time and service routine might miss
    the completions

  - txpp_err_rearm_queue - reports errors in rearm queue
  - txpp_err_clock_queue - reports errors in clock queue

  - txpp_err_ts_past - timestamps in the packet being sent
    were found in the past, timestamps were ignored

  - txpp_err_ts_future - timestamps in the packet being sent
    were found in the too distant future (beyond HW/clock queue
    capabilities to schedule, typically it is about 16M of
    tx_pp devarg periods)

  - txpp_jitter - estimated jitter in device clocks between
    8K completions of Clock Queue.

  - txpp_wander - estimated wander in device clocks between
    16M completions of Clock Queue.

  - txpp_sync_lost - error flag, the Clock Queue completions
    synchronization is lost, accurate packet scheduling can
    not be handled, timestamps are being ignored, the restart
    of all ports using scheduling must be performed.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
b94d93ca73 net/mlx5: support reading device clock
If send schedule feature is engaged there is the Clock Queue
created, that reports reliable the current device clock counter
value. The device clock counter can be read directly from the
Clock Queue CQE.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
2f827f5ea6 net/mlx5: support scheduling on send routine template
This patch adds send scheduling on timestamps into tx_burst
routine template. The feature is controlled by static configuration
flag, the actual routines supporting the new feature are generated
over this updated template.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
0febfcce36 net/mlx5: prepare Tx to support scheduling
The new static control flag is introduced to control
routine generating from template, enabling the scheduling
on timestamps.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
085ff447f0 net/mlx5: convert timestamp to completion index
The application provides timestamps in Tx mbuf as clocks,
the hardware performs scheduling on Clock Queue completion index
match. This patch introduces the timestamp-to-completion-index
inline routine.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
3172c471b8 net/mlx5: prepare Tx queue structures to support timestamp
The fields to support send scheduling on dynamic timestamp
field are introduced and initialized on device start.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
77522be0a5 net/mlx5: introduce clock queue service routine
Service routine is invoked periodically on Rearm Queue
completion interrupts, typically once per some milliseconds
(1-16) to track clock jitter and wander in robust fashion.
It performs the following:

- fetches the completed CQEs for Rearm Queue
- restarts Rearm Queue on errors
- pushes new requests to Rearm Queue to make it
  continuously running and pushing cross-channel requests
  to Clock Queue
- reads and caches the Clock Queue CQE to be used in datapath
- gathers statistics to estimate clock jitter and wander
- gathers Clock Queue errors statistics

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
aef1e20ebe net/mlx5: allocate packet pacing context
This patch allocates the Packet Pacing context from the kernel,
configures one according to requested pace send scheduling
granularity and assigns to Clock Queue.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00
Viacheslav Ovsiienko
3a87b964ed net/mlx5: create Tx queues with DevX
To provide the packet send schedule on mbuf timestamp the Tx
queue must be attached to the same UAR as Clock Queue is.
UAR is special hardware related resource mapped to the host
memory and provides doorbell registers, the assigning UAR
to the queue being created is provided via DevX API only.

Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 15:44:36 +02:00