d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity
87af0d1e1b
numam-dpdk/drivers/net/mlx5/linux/mlx5_os.c

2874 lines
83 KiB
C
Raw Normal View History

net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
/* SPDX-License-Identifier: BSD-3-Clause
* Copyright 2015 6WIND S.A.
* Copyright 2020 Mellanox Technologies, Ltd
*/
#include <stddef.h>
#include <unistd.h>
#include <string.h>
#include <stdint.h>
#include <stdlib.h>
#include <errno.h>
#include <net/if.h>
#include <linux/rtnetlink.h>
net/mlx5: refactor statistics mlx5 statistics are calculated by several methods: 1. In software when packets go through datapath. 2. Calling ioctl with ETHTOOL command (Linux specific). 3. Reading counters from SYSFS device path (Linux specific). The Linux related functions are moved to file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:33 +00:00
#include <linux/sockios.h>
#include <linux/ethtool.h>
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include <fcntl.h>
#include <rte_malloc.h>
ethdev: make driver-only headers private The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are for drivers only and should be a private to DPDK and not installed. Signed-off-by: Bruce Richardson <bruce.richardson@intel.com> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com> Acked-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Steven Webster <steven.webster@windriver.com>
2021-01-29 16:48:19 +00:00
#include <ethdev_driver.h>
#include <ethdev_pci.h>
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include <rte_pci.h>
#include <rte_bus_pci.h>
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
#include <rte_bus_auxiliary.h>
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include <rte_common.h>
#include <rte_kvargs.h>
#include <rte_rwlock.h>
#include <rte_spinlock.h>
#include <rte_string_fns.h>
#include <rte_alarm.h>
net/mlx5: replace Linux specific calls The following Linux calls are replaced by their matching rte APIs. mmap ==> rte_mem_map() munmap == >rte_mem_unmap() sysconf(_SC_PAGESIZE) ==> rte_mem_page_size() Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:10 +00:00
#include <rte_eal_paging.h>
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include <mlx5_glue.h>
#include <mlx5_devx_cmds.h>
#include <mlx5_common.h>
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#include <mlx5_common_mp.h>
net/mlx5: add memory region callbacks in per-device cache Prior to this commit MR operations were verbs based and hard coded under common/mlx5/linux directory. This commit enables upper layers (e.g. net/mlx5) to determine which MR operations to use. For example the net layer could set devx based MR operations in non-Linux environments. The reg_mr and dereg_mr callbacks are added to the global per-device MR cache 'struct mlx5_mr_share_cache'. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-16 09:44:45 +00:00
#include <mlx5_common_mr.h>
net/mlx5: add option to allocate memory from system Currently, for MLX5 PMD, once millions of flows created, the memory consumption of the flows are also very huge. For the system with limited memory, it means the system need to reserve most of the memory as huge page memory to serve the flows in advance. And other normal applications will have no chance to use this reserved memory any more. While most of the time, the system will not have lots of flows, the reserved huge page memory becomes a bit waste of memory at most of the time. By the new sys_mem_en devarg, once set it to be true, it allows the PMD allocate the memory from system by default with the new add mlx5 memory management functions. Only once the MLX5_MEM_RTE flag is set, the memory will be allocate from rte, otherwise, it allocates memory from system. So in this case, the system with limited memory no need to reserve most of the memory for hugepage. Only some needed memory for datapath objects will be enough to allocated with explicitly flag. Other memory will be allocated from system. For system with enough memory, no need to care about the devarg, the memory will always be from rte hugepage. One restriction is that for DPDK application with multiple PCI devices, if the sys_mem_en devargs are different between the devices, the sys_mem_en only gets the value from the first device devargs, and print out a message to warn that. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 03:41:57 +00:00
#include <mlx5_malloc.h>
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include "mlx5_defs.h"
#include "mlx5.h"
common/mlx5: move some getter functions from net driver Getter functions such as: 'mlx5_os_get_ctx_device_name', 'mlx5_os_get_ctx_device_path', 'mlx5_os_get_dev_device_name', 'mlx5_os_get_umem_id' are implemented under net directory. To enable additional devices (e.g. regex, vdpa) to access these getter functions they are moved under common directory. As part of this commit string sizes DEV_SYSFS_NAME_MAX and DEV_SYSFS_PATH_MAX are increased by 1 to make sure that the destination string size in strncpy() function is bigger than the source string size. This update will avoid GCC version 8 error -Werror=stringop-truncation. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-19 07:30:08 +00:00
#include "mlx5_common_os.h"
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include "mlx5_utils.h"
#include "mlx5_rxtx.h"
net/mlx5: separate Rx function declarations to another file The mlx5_rxtx.c file contains a lot of Tx burst functions, each of those is performance-optimized for the specific set of requested offloads. These ones are generated on the basis of the template function and it takes significant time to compile, just due to a large number of giant functions generated in the same file and this compilation is not being done in parallel with using multithreading. Therefore we can split the mlx5_rxtx.c file into several separate files to allow different functions to be compiled simultaneously. In this patch, we separate Rx function declarations to different header file in preparation for removing them from the source file and as an optional preparation step for further consolidation of Rx burst functions. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-04-12 06:32:20 +00:00
#include "mlx5_rx.h"
net/mlx5: separate Tx function declarations to another file This patch separates Tx function declarations to different header file in preparation for removing their implementation from the source file and as an optional preparation for Tx cleanup. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-04-12 06:32:22 +00:00
#include "mlx5_tx.h"
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
#include "mlx5_autoconf.h"
#include "mlx5_flow.h"
#include "rte_pmd_mlx5.h"
net/mlx5/linux: add memory region callbacks to Verbs Create a set of verbs callbacks in 'struct mlx5_verbs_ops' and add MR operations to it (file net/mlx5/linux/mlx5_verbs.c). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-16 09:44:46 +00:00
#include "mlx5_verbs.h"
net/mlx5: remove netlink dependency in shared code This commit adds Linux implementation of routine mlx5_os_mac_addr_flush as wrapper to Netlink API to avoid direct calls under non-Linux operating systems. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-08-25 09:31:09 +00:00
#include "mlx5_nl.h"
net/mlx5: separate Rx queue object creations As an arrangement to Windows OS support, the Verbs operations should be separated to another file. By this way, the build can easily cut the unsupported Verbs APIs from the compilation process. Define operation structure and DevX module in addition to the existing linux Verbs module. Separate Rx object creation into the Verbs/DevX modules and update the operation structure according to the OS support and the user configuration. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-09-03 10:13:36 +00:00
#include "mlx5_devx.h"
net/mlx5: add Linux-specific file with getter functions 'ctx' type (field in 'struct mlx5_ctx_shared') is changed from 'struct ibv_context *' to 'void *'. 'ctx' members which are verbs dependent (e.g. device_name) will be accessed through getter functions which are added to a new file under Linux directory: linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:56 +00:00
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifndef HAVE_IBV_MLX5_MOD_MPW
#define MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED (1 << 2)
#define MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW (1 << 3)
#endif
#ifndef HAVE_IBV_MLX5_MOD_CQE_128B_COMP
#define MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP (1 << 4)
#endif
net/mlx5: refactor multi-process communication 1. The shared data communication between the primary and the secondary processes is implemented using Linux API. Move the Linux API code under linux directory (file linux/mlx5_os.c). 2. File net/mlx5/mlx5_mp.c handles requests to the primary and secondary processes (e.g. start_rxtx, stop_rxtx). It is Linux based so it is moved under linux (new file linux/mlx5_mp_os.c). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:15 +00:00
static const char *MZ_MLX5_PMD_SHARED_DATA = "mlx5_pmd_shared_data";
/* Spinlock for mlx5_shared_data allocation. */
static rte_spinlock_t mlx5_shared_data_lock = RTE_SPINLOCK_INITIALIZER;
/* Process local data for secondary processes. */
static struct mlx5_local_data mlx5_local_data;
net/mlx5: replace flow list with indexed pool The flow list is used to save the create flows and to be used only when port closes all the flows need to be flushed. This commit takes advantage of the index pool foreach operation to flush all the allocated flows. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:39 +00:00
/* rte flow indexed pool configuration. */
static struct mlx5_indexed_pool_config icfg[] = {
{
.size = sizeof(struct rte_flow),
.trunk_size = 64,
.need_lock = 1,
.release_mem_en = 0,
.malloc = mlx5_malloc,
.free = mlx5_free,
.per_core_cache = 0,
.type = "ctl_flow_ipool",
},
{
.size = sizeof(struct rte_flow),
.trunk_size = 64,
.grow_trunk = 3,
.grow_shift = 2,
.need_lock = 1,
.release_mem_en = 0,
.malloc = mlx5_malloc,
.free = mlx5_free,
.per_core_cache = 1 << 14,
.type = "rte_flow_ipool",
},
{
.size = sizeof(struct rte_flow),
.trunk_size = 64,
.grow_trunk = 3,
.grow_shift = 2,
.need_lock = 1,
.release_mem_en = 0,
.malloc = mlx5_malloc,
.free = mlx5_free,
.per_core_cache = 0,
.type = "mcp_flow_ipool",
},
};
net/mlx5: implement CQ for Rx using DevX API This patch continues the work to use DevX API for different objects creation and management. On Rx control path, the RQ, RQT, and TIR objects can already be created using DevX API. This patch adds the support to create CQ for RxQ using DevX API. The corresponding event channel is also created and utilized using DevX API. Signed-off-by: Dekel Peled <dekelp@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-19 11:13:06 +00:00
/**
* Set the completion channel file descriptor interrupt as non-blocking.
*
* @param[in] rxq_obj
* Pointer to RQ channel object, which includes the channel fd
*
* @param[out] fd
fix spelling in comments and strings The tool comes from https://github.com/jsoref Signed-off-by: Josh Soref <jsoref@gmail.com> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-11-29 16:08:02 +00:00
* The file descriptor (representing the interrupt) used in this channel.
net/mlx5: implement CQ for Rx using DevX API This patch continues the work to use DevX API for different objects creation and management. On Rx control path, the RQ, RQT, and TIR objects can already be created using DevX API. This patch adds the support to create CQ for RxQ using DevX API. The corresponding event channel is also created and utilized using DevX API. Signed-off-by: Dekel Peled <dekelp@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-19 11:13:06 +00:00
*
* @return
* 0 on successfully setting the fd to non-blocking, non-zero otherwise.
*/
int
mlx5_os_set_nonblock_channel_fd(int fd)
{
int flags;
flags = fcntl(fd, F_GETFL);
return fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
/**
* Get mlx5 device attributes. The glue function query_device_ex() is called
* with out parameter of type 'struct ibv_device_attr_ex *'. Then fill in mlx5
* device attributes from the glue out parameter.
*
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
* @param sh
* Pointer to shared device context.
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
*
* @return
net/mlx5: fix errno update in shared context creation The mlx5_alloc_shared_dev_ctx() function has a local variable named "err" which contains the errno value in case of failure. When functions called by this function are failed, this variable is updated with their return value (that should be a positive errno value). However, some functions doesn't update errno value by themselves or return negative errno value. If one of them fails, the "err" variable contains negative value what cause to assertion failure. This patch updates all functions uses by mlx5_alloc_shared_dev_ctx() function to update rte_errno and take this value instead of "err" value. Fixes: 5dfa003db53f ("common/mlx5: fix post doorbell barrier") Fixes: 5d55a494f4e6 ("net/mlx5: split multi-thread flow handling per OS") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:55 +00:00
* 0 on success, a negative errno value otherwise and rte_errno is set.
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
*/
int
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
mlx5_os_capabilities_prepare(struct mlx5_dev_ctx_shared *sh)
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
{
int err;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
struct mlx5_common_device *cdev = sh->cdev;
struct mlx5_hca_attr *hca_attr = &cdev->config.hca_attr;
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
struct ibv_device_attr_ex attr_ex = { .comp_mask = 0 };
struct mlx5dv_context dv_attr = { .comp_mask = 0 };
common/mlx5: share HCA capabilities handle Add HCA attributes structure as a field of device config structure. It query in common probing, and updates the timestamp format fields. Each driver use HCA attributes from common device config structure, instead of query it for itself. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:55 +00:00
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
err = mlx5_glue->query_device_ex(cdev->ctx, NULL, &attr_ex);
net/mlx5: fix errno update in shared context creation The mlx5_alloc_shared_dev_ctx() function has a local variable named "err" which contains the errno value in case of failure. When functions called by this function are failed, this variable is updated with their return value (that should be a positive errno value). However, some functions doesn't update errno value by themselves or return negative errno value. If one of them fails, the "err" variable contains negative value what cause to assertion failure. This patch updates all functions uses by mlx5_alloc_shared_dev_ctx() function to update rte_errno and take this value instead of "err" value. Fixes: 5dfa003db53f ("common/mlx5: fix post doorbell barrier") Fixes: 5d55a494f4e6 ("net/mlx5: split multi-thread flow handling per OS") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:55 +00:00
if (err) {
rte_errno = errno;
return -rte_errno;
}
net/mlx5: remove Verbs query device duplication The sharing device context structure has a field named "device_attr" which is filled by mlx5_os_get_dev_attr() function. The spawn function calls mlx5_os_get_dev_attr() again and save it to local variable identical to "device_attr" field. There is no need for this duplication, because there is a reference to the sharing device context structure from spawn function. This patch removes the local "device_attr" from spawn function, and uses the context's field instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:00 +00:00
#ifdef HAVE_IBV_MLX5_MOD_SWP
dv_attr.comp_mask |= MLX5DV_CONTEXT_MASK_SWP;
#endif
#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT
dv_attr.comp_mask |= MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS;
#endif
#ifdef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT
dv_attr.comp_mask |= MLX5DV_CONTEXT_MASK_STRIDING_RQ;
#endif
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
err = mlx5_glue->dv_query_device(cdev->ctx, &dv_attr);
net/mlx5: fix errno update in shared context creation The mlx5_alloc_shared_dev_ctx() function has a local variable named "err" which contains the errno value in case of failure. When functions called by this function are failed, this variable is updated with their return value (that should be a positive errno value). However, some functions doesn't update errno value by themselves or return negative errno value. If one of them fails, the "err" variable contains negative value what cause to assertion failure. This patch updates all functions uses by mlx5_alloc_shared_dev_ctx() function to update rte_errno and take this value instead of "err" value. Fixes: 5dfa003db53f ("common/mlx5: fix post doorbell barrier") Fixes: 5d55a494f4e6 ("net/mlx5: split multi-thread flow handling per OS") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:55 +00:00
if (err) {
rte_errno = errno;
return -rte_errno;
}
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
memset(&sh->dev_cap, 0, sizeof(struct mlx5_dev_cap));
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (mlx5_dev_is_pci(cdev->dev))
sh->dev_cap.vf = mlx5_dev_is_vf_pci(RTE_DEV_TO_PCI(cdev->dev));
else
sh->dev_cap.sf = 1;
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
sh->dev_cap.max_qp_wr = attr_ex.orig_attr.max_qp_wr;
sh->dev_cap.max_sge = attr_ex.orig_attr.max_sge;
sh->dev_cap.max_cq = attr_ex.orig_attr.max_cq;
sh->dev_cap.max_qp = attr_ex.orig_attr.max_qp;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
#ifdef HAVE_MLX5DV_DR_ACTION_DEST_DEVX_TIR
sh->dev_cap.dest_tir = 1;
#endif
#if defined(HAVE_IBV_FLOW_DV_SUPPORT) && defined(HAVE_MLX5DV_DR)
DRV_LOG(DEBUG, "DV flow is supported.");
sh->dev_cap.dv_flow_en = 1;
#endif
#ifdef HAVE_MLX5DV_DR_ESWITCH
if (hca_attr->eswitch_manager && sh->dev_cap.dv_flow_en && sh->esw_mode)
sh->dev_cap.dv_esw_en = 1;
#endif
/*
* Multi-packet send is supported by ConnectX-4 Lx PF as well
* as all ConnectX-5 devices.
*/
if (dv_attr.flags & MLX5DV_CONTEXT_FLAGS_MPW_ALLOWED) {
if (dv_attr.flags & MLX5DV_CONTEXT_FLAGS_ENHANCED_MPW) {
DRV_LOG(DEBUG, "Enhanced MPW is supported.");
sh->dev_cap.mps = MLX5_MPW_ENHANCED;
} else {
DRV_LOG(DEBUG, "MPW is supported.");
sh->dev_cap.mps = MLX5_MPW;
}
} else {
DRV_LOG(DEBUG, "MPW isn't supported.");
sh->dev_cap.mps = MLX5_MPW_DISABLED;
}
#if (RTE_CACHE_LINE_SIZE == 128)
if (dv_attr.flags & MLX5DV_CONTEXT_FLAGS_CQE_128B_COMP)
sh->dev_cap.cqe_comp = 1;
DRV_LOG(DEBUG, "Rx CQE 128B compression is %ssupported.",
sh->dev_cap.cqe_comp ? "" : "not ");
#else
sh->dev_cap.cqe_comp = 1;
#endif
#ifdef HAVE_IBV_DEVICE_MPLS_SUPPORT
sh->dev_cap.mpls_en =
((dv_attr.tunnel_offloads_caps &
MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_GRE) &&
(dv_attr.tunnel_offloads_caps &
MLX5DV_RAW_PACKET_CAP_TUNNELED_OFFLOAD_CW_MPLS_OVER_UDP));
DRV_LOG(DEBUG, "MPLS over GRE/UDP tunnel offloading is %ssupported.",
sh->dev_cap.mpls_en ? "" : "not ");
#else
DRV_LOG(WARNING,
"MPLS over GRE/UDP tunnel offloading disabled due to old OFED/rdma-core version or firmware configuration");
#endif
#if defined(HAVE_IBV_WQ_FLAG_RX_END_PADDING)
sh->dev_cap.hw_padding = !!attr_ex.rx_pad_end_addr_align;
#elif defined(HAVE_IBV_WQ_FLAGS_PCI_WRITE_END_PADDING)
sh->dev_cap.hw_padding = !!(attr_ex.device_cap_flags_ex &
IBV_DEVICE_PCI_WRITE_END_PADDING);
#endif
sh->dev_cap.hw_csum =
!!(attr_ex.device_cap_flags_ex & IBV_DEVICE_RAW_IP_CSUM);
DRV_LOG(DEBUG, "Checksum offloading is %ssupported.",
sh->dev_cap.hw_csum ? "" : "not ");
sh->dev_cap.hw_vlan_strip = !!(attr_ex.raw_packet_caps &
IBV_RAW_PACKET_CAP_CVLAN_STRIPPING);
DRV_LOG(DEBUG, "VLAN stripping is %ssupported.",
(sh->dev_cap.hw_vlan_strip ? "" : "not "));
sh->dev_cap.hw_fcs_strip = !!(attr_ex.raw_packet_caps &
IBV_RAW_PACKET_CAP_SCATTER_FCS);
#if !defined(HAVE_IBV_DEVICE_COUNTERS_SET_V42) && \
!defined(HAVE_IBV_DEVICE_COUNTERS_SET_V45)
DRV_LOG(DEBUG, "Counters are not supported.");
#endif
/*
* DPDK doesn't support larger/variable indirection tables.
* Once DPDK supports it, take max size from device attr.
*/
sh->dev_cap.ind_table_max_size =
RTE_MIN(attr_ex.rss_caps.max_rwq_indirection_table_size,
(unsigned int)RTE_ETH_RSS_RETA_SIZE_512);
DRV_LOG(DEBUG, "Maximum Rx indirection table size is %u",
sh->dev_cap.ind_table_max_size);
sh->dev_cap.tso = (attr_ex.tso_caps.max_tso > 0 &&
(attr_ex.tso_caps.supported_qpts &
(1 << IBV_QPT_RAW_PACKET)));
if (sh->dev_cap.tso)
sh->dev_cap.tso_max_payload_sz = attr_ex.tso_caps.max_tso;
net/mlx5: rearrange device attribute structure Rearrange the mlx5_os_get_dev_attr() function in such a way that it first executes the queries and only then updates the fields. In addition, it changed its name in preparation for expanding its operations to configure the capabilities inside it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:05 +00:00
strlcpy(sh->dev_cap.fw_ver, attr_ex.orig_attr.fw_ver,
sizeof(sh->dev_cap.fw_ver));
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
#ifdef HAVE_IBV_MLX5_MOD_SWP
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (dv_attr.comp_mask & MLX5DV_CONTEXT_MASK_SWP)
sh->dev_cap.swp = dv_attr.sw_parsing_caps.sw_parsing_offloads &
(MLX5_SW_PARSING_CAP |
MLX5_SW_PARSING_CSUM_CAP |
MLX5_SW_PARSING_TSO_CAP);
DRV_LOG(DEBUG, "SWP support: %u", sh->dev_cap.swp);
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
#endif
net/mlx5: remove Verbs query device duplication The sharing device context structure has a field named "device_attr" which is filled by mlx5_os_get_dev_attr() function. The spawn function calls mlx5_os_get_dev_attr() again and save it to local variable identical to "device_attr" field. There is no need for this duplication, because there is a reference to the sharing device context structure from spawn function. This patch removes the local "device_attr" from spawn function, and uses the context's field instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:00 +00:00
#ifdef HAVE_IBV_DEVICE_STRIDING_RQ_SUPPORT
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (dv_attr.comp_mask & MLX5DV_CONTEXT_MASK_STRIDING_RQ) {
struct mlx5dv_striding_rq_caps *strd_rq_caps =
&dv_attr.striding_rq_caps;
sh->dev_cap.mprq.enabled = 1;
sh->dev_cap.mprq.log_min_stride_size =
strd_rq_caps->min_single_stride_log_num_of_bytes;
sh->dev_cap.mprq.log_max_stride_size =
strd_rq_caps->max_single_stride_log_num_of_bytes;
sh->dev_cap.mprq.log_min_stride_num =
strd_rq_caps->min_single_wqe_log_num_of_strides;
sh->dev_cap.mprq.log_max_stride_num =
strd_rq_caps->max_single_wqe_log_num_of_strides;
sh->dev_cap.mprq.log_min_stride_wqe_size =
cdev->config.devx ?
hca_attr->log_min_stride_wqe_sz :
MLX5_MPRQ_LOG_MIN_STRIDE_WQE_SIZE;
DRV_LOG(DEBUG, "\tmin_single_stride_log_num_of_bytes: %u",
sh->dev_cap.mprq.log_min_stride_size);
DRV_LOG(DEBUG, "\tmax_single_stride_log_num_of_bytes: %u",
sh->dev_cap.mprq.log_max_stride_size);
DRV_LOG(DEBUG, "\tmin_single_wqe_log_num_of_strides: %u",
sh->dev_cap.mprq.log_min_stride_num);
DRV_LOG(DEBUG, "\tmax_single_wqe_log_num_of_strides: %u",
sh->dev_cap.mprq.log_max_stride_num);
DRV_LOG(DEBUG, "\tmin_stride_wqe_log_size: %u",
sh->dev_cap.mprq.log_min_stride_wqe_size);
DRV_LOG(DEBUG, "\tsupported_qpts: %d",
strd_rq_caps->supported_qpts);
DRV_LOG(DEBUG, "Device supports Multi-Packet RQ.");
}
net/mlx5: remove Verbs query device duplication The sharing device context structure has a field named "device_attr" which is filled by mlx5_os_get_dev_attr() function. The spawn function calls mlx5_os_get_dev_attr() again and save it to local variable identical to "device_attr" field. There is no need for this duplication, because there is a reference to the sharing device context structure from spawn function. This patch removes the local "device_attr" from spawn function, and uses the context's field instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:00 +00:00
#endif
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
#ifdef HAVE_IBV_DEVICE_TUNNEL_SUPPORT
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (dv_attr.comp_mask & MLX5DV_CONTEXT_MASK_TUNNEL_OFFLOADS) {
sh->dev_cap.tunnel_en = dv_attr.tunnel_offloads_caps &
(MLX5_TUNNELED_OFFLOADS_VXLAN_CAP |
MLX5_TUNNELED_OFFLOADS_GRE_CAP |
MLX5_TUNNELED_OFFLOADS_GENEVE_CAP);
}
if (sh->dev_cap.tunnel_en) {
DRV_LOG(DEBUG, "Tunnel offloading is supported for %s%s%s",
sh->dev_cap.tunnel_en &
MLX5_TUNNELED_OFFLOADS_VXLAN_CAP ? "[VXLAN]" : "",
sh->dev_cap.tunnel_en &
MLX5_TUNNELED_OFFLOADS_GRE_CAP ? "[GRE]" : "",
sh->dev_cap.tunnel_en &
MLX5_TUNNELED_OFFLOADS_GENEVE_CAP ? "[GENEVE]" : "");
} else {
DRV_LOG(DEBUG, "Tunnel offloading is not supported.");
}
#else
DRV_LOG(WARNING,
"Tunnel offloading disabled due to old OFED/rdma-core version");
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
#endif
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (!sh->cdev->config.devx)
return 0;
/* Check capabilities for Packet Pacing. */
DRV_LOG(DEBUG, "Timestamp counter frequency %u kHz.",
hca_attr->dev_freq_khz);
DRV_LOG(DEBUG, "Packet pacing is %ssupported.",
hca_attr->qos.packet_pacing ? "" : "not ");
DRV_LOG(DEBUG, "Cross channel ops are %ssupported.",
hca_attr->cross_channel ? "" : "not ");
DRV_LOG(DEBUG, "WQE index ignore is %ssupported.",
hca_attr->wqe_index_ignore ? "" : "not ");
DRV_LOG(DEBUG, "Non-wire SQ feature is %ssupported.",
hca_attr->non_wire_sq ? "" : "not ");
DRV_LOG(DEBUG, "Static WQE SQ feature is %ssupported (%d)",
hca_attr->log_max_static_sq_wq ? "" : "not ",
hca_attr->log_max_static_sq_wq);
DRV_LOG(DEBUG, "WQE rate PP mode is %ssupported.",
hca_attr->qos.wqe_rate_pp ? "" : "not ");
sh->dev_cap.txpp_en = hca_attr->qos.packet_pacing;
if (!hca_attr->cross_channel) {
DRV_LOG(DEBUG,
"Cross channel operations are required for packet pacing.");
sh->dev_cap.txpp_en = 0;
}
if (!hca_attr->wqe_index_ignore) {
DRV_LOG(DEBUG,
"WQE index ignore feature is required for packet pacing.");
sh->dev_cap.txpp_en = 0;
}
if (!hca_attr->non_wire_sq) {
DRV_LOG(DEBUG,
"Non-wire SQ feature is required for packet pacing.");
sh->dev_cap.txpp_en = 0;
}
if (!hca_attr->log_max_static_sq_wq) {
DRV_LOG(DEBUG,
"Static WQE SQ feature is required for packet pacing.");
sh->dev_cap.txpp_en = 0;
}
if (!hca_attr->qos.wqe_rate_pp) {
DRV_LOG(DEBUG,
"WQE rate mode is required for packet pacing.");
sh->dev_cap.txpp_en = 0;
}
#ifndef HAVE_MLX5DV_DEVX_UAR_OFFSET
DRV_LOG(DEBUG,
"DevX does not provide UAR offset, can't create queues for packet pacing.");
sh->dev_cap.txpp_en = 0;
#endif
/* Check for LRO support. */
if (sh->dev_cap.dest_tir && sh->dev_cap.dv_flow_en &&
hca_attr->lro_cap) {
/* TBD check tunnel lro caps. */
sh->dev_cap.lro_supported = 1;
DRV_LOG(DEBUG, "Device supports LRO.");
DRV_LOG(DEBUG,
"LRO minimal size of TCP segment required for coalescing is %d bytes.",
hca_attr->lro_min_mss_size);
}
sh->dev_cap.scatter_fcs_w_decap_disable =
hca_attr->scatter_fcs_w_decap_disable;
sh->dev_cap.rq_delay_drop_en = hca_attr->rq_delay_drop;
mlx5_rt_timestamp_config(sh, hca_attr);
net/mlx5: fix errno update in shared context creation The mlx5_alloc_shared_dev_ctx() function has a local variable named "err" which contains the errno value in case of failure. When functions called by this function are failed, this variable is updated with their return value (that should be a positive errno value). However, some functions doesn't update errno value by themselves or return negative errno value. If one of them fails, the "err" variable contains negative value what cause to assertion failure. This patch updates all functions uses by mlx5_alloc_shared_dev_ctx() function to update rte_errno and take this value instead of "err" value. Fixes: 5dfa003db53f ("common/mlx5: fix post doorbell barrier") Fixes: 5d55a494f4e6 ("net/mlx5: split multi-thread flow handling per OS") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:55 +00:00
return 0;
net/mlx5: remove attributes dependency on Verbs Define 'struct mlx5_dev_attr' which is ibv and dv independent. It contains attribute that were originally contained in 'struct ibv_device_attr_ex' and 'struct mlx5dv_context dv_attr'. Add a new API mlx5_os_get_dev_attr() which fills in the new defined struct. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:05:58 +00:00
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
net/mlx5: support matching on VXLAN reserved field This adds matching on the reserved field of VXLAN header (the last 8-bits). The capability from rdma-core is detected by creating a dummy matcher using misc5 when the device is probed. For non-zero groups and FDB domain, the capability is detected from rdma-core, meanwhile for NIC domain group zero it's relying on the HCA_CAP from FW. Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Raslan Darawsheh <rasland@nvidia.com>
2021-07-13 12:09:19 +00:00
/**
* Detect misc5 support or not
*
* @param[in] priv
* Device private data pointer
*/
#ifdef HAVE_MLX5DV_DR
static void
__mlx5_discovery_misc5_cap(struct mlx5_priv *priv)
{
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
/* Dummy VxLAN matcher to detect rdma-core misc5 cap
* Case: IPv4--->UDP--->VxLAN--->vni
*/
void *tbl;
struct mlx5_flow_dv_match_params matcher_mask;
void *match_m;
void *matcher;
void *headers_m;
void *misc5_m;
uint32_t *tunnel_header_m;
struct mlx5dv_flow_matcher_attr dv_attr;
memset(&matcher_mask, 0, sizeof(matcher_mask));
matcher_mask.size = sizeof(matcher_mask.buf);
match_m = matcher_mask.buf;
headers_m = MLX5_ADDR_OF(fte_match_param, match_m, outer_headers);
misc5_m = MLX5_ADDR_OF(fte_match_param,
match_m, misc_parameters_5);
tunnel_header_m = (uint32_t *)
MLX5_ADDR_OF(fte_match_set_misc5,
misc5_m, tunnel_header_1);
MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_protocol, 0xff);
MLX5_SET(fte_match_set_lyr_2_4, headers_m, ip_version, 4);
MLX5_SET(fte_match_set_lyr_2_4, headers_m, udp_dport, 0xffff);
*tunnel_header_m = 0xffffff;
tbl = mlx5_glue->dr_create_flow_tbl(priv->sh->rx_domain, 1);
if (!tbl) {
DRV_LOG(INFO, "No SW steering support");
return;
}
dv_attr.type = IBV_FLOW_ATTR_NORMAL,
dv_attr.match_mask = (void *)&matcher_mask,
dv_attr.match_criteria_enable =
(1 << MLX5_MATCH_CRITERIA_ENABLE_OUTER_BIT) |
(1 << MLX5_MATCH_CRITERIA_ENABLE_MISC5_BIT);
dv_attr.priority = 3;
#ifdef HAVE_MLX5DV_DR_ESWITCH
void *misc2_m;
if (priv->config.dv_esw_en) {
/* FDB enabled reg_c_0 */
dv_attr.match_criteria_enable |=
(1 << MLX5_MATCH_CRITERIA_ENABLE_MISC2_BIT);
misc2_m = MLX5_ADDR_OF(fte_match_param,
match_m, misc_parameters_2);
MLX5_SET(fte_match_set_misc2, misc2_m,
metadata_reg_c_0, 0xffff);
}
#endif
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
matcher = mlx5_glue->dv_create_flow_matcher(priv->sh->cdev->ctx,
net/mlx5: support matching on VXLAN reserved field This adds matching on the reserved field of VXLAN header (the last 8-bits). The capability from rdma-core is detected by creating a dummy matcher using misc5 when the device is probed. For non-zero groups and FDB domain, the capability is detected from rdma-core, meanwhile for NIC domain group zero it's relying on the HCA_CAP from FW. Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Raslan Darawsheh <rasland@nvidia.com>
2021-07-13 12:09:19 +00:00
&dv_attr, tbl);
if (matcher) {
priv->sh->misc5_cap = 1;
mlx5_glue->dv_destroy_flow_matcher(matcher);
}
mlx5_glue->dr_destroy_flow_tbl(tbl);
#else
RTE_SET_USED(priv);
#endif
}
#endif
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/**
* Initialize DR related data within private structure.
* Routine checks the reference counter and does actual
* resources creation/initialization only if counter is zero.
*
* @param[in] priv
* Pointer to the private device data structure.
*
* @return
* Zero on success, positive error code otherwise.
*/
static int
mlx5_alloc_shared_dr(struct mlx5_priv *priv)
{
struct mlx5_dev_ctx_shared *sh = priv->sh;
common/mlx5: add per-lcore cache to hash list utility Using the mlx5 list utility object in the hlist buckets. This patch moves the list utility object to the common utility, creates all the clone operations for all the hlist instances in the driver. Also adjust all the utility callbacks to be generic for both list and hlist. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:50 +00:00
char s[MLX5_NAME_SIZE] __rte_unused;
net/mlx5: fix port shared data reference count When probe a representor, tag cache hash table and modification cache hash table allocated memory upon each port, overwrote previous existing cache in shared context data. This patch moves reference check of shared data prior to hash table allocation to avoid such issue. Fixes: 6801116688fe ("net/mlx5: fix multiple flow table hash list") Fixes: 1ef4cdef2682 ("net/mlx5: fix flow tag hash list conversion") Cc: stable@dpdk.org Acked-by: Matan Azrad <matan@nvidia.com> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
2020-10-21 11:15:23 +00:00
int err;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
net/mlx5: fix port shared data reference count When probe a representor, tag cache hash table and modification cache hash table allocated memory upon each port, overwrote previous existing cache in shared context data. This patch moves reference check of shared data prior to hash table allocation to avoid such issue. Fixes: 6801116688fe ("net/mlx5: fix multiple flow table hash list") Fixes: 1ef4cdef2682 ("net/mlx5: fix flow tag hash list conversion") Cc: stable@dpdk.org Acked-by: Matan Azrad <matan@nvidia.com> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
2020-10-21 11:15:23 +00:00
MLX5_ASSERT(sh && sh->refcnt);
if (sh->refcnt > 1)
return 0;
err = mlx5_alloc_table_hash_list(priv);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (err)
net/mlx5: fix redundant Direct Verbs resources allocate All table, tag, header modify, header reformat are supported only on DV mode. For the OFED version doesn't support these, create the related redundant DV resources waste the memory. Add the code section in the HAVE_IBV_FLOW_DV_SUPPORT macro to avoid the redundant resources allocation. Fixes: 2eb4d0107acc ("net/mlx5: refactor PCI probing on Linux") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:34 +00:00
goto error;
/* The resources below are only valid with DV support. */
#ifdef HAVE_IBV_FLOW_DV_SUPPORT
net/mlx5: add per-lcore cache to the list utility When mlx5 list object is accessed by multiple cores, the list lock counter is all the time written by all the cores what increases cache misses in the memory caches. In addition, when one thread accesses the list for add\remove\lookup operation, all the other threads coming to do an operation in the list are stuck in the lock. Add per lcore cache to allow thread manipulations to be lockless when the list objects are mostly reused. Synchronization with atomic operations should be done in order to allow threads to unregister an entry from other thread cache. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:42 +00:00
/* Init port id action list. */
net/mlx5: remove cache term from the list utility The internal mlx5 list tool is used mainly when the list objects need to be synchronized between multiple threads. The "cache" term is used in the internal mlx5 list API. Next enhancements on this tool will use the "cache" term for per thread cache management. To prevent confusing, remove the current "cache" term from the API's names. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:41 +00:00
snprintf(s, sizeof(s), "%s_port_id_action_list", sh->ibdev_name);
common/mlx5: add per-lcore sharing flag in object list Without lcores_share flag, mlx5 PMD was sharing the rdma-core objects between all lcores. Having lcores_share flag disabled, means each lcore will have its own objects, which will eventually lead to increased insertion/deletion rates. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:48 +00:00
sh->port_id_action_list = mlx5_list_create(s, sh, true,
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
flow_dv_port_id_create_cb,
flow_dv_port_id_match_cb,
flow_dv_port_id_remove_cb,
flow_dv_port_id_clone_cb,
flow_dv_port_id_clone_free_cb);
if (!sh->port_id_action_list)
goto error;
net/mlx5: add per-lcore cache to the list utility When mlx5 list object is accessed by multiple cores, the list lock counter is all the time written by all the cores what increases cache misses in the memory caches. In addition, when one thread accesses the list for add\remove\lookup operation, all the other threads coming to do an operation in the list are stuck in the lock. Add per lcore cache to allow thread manipulations to be lockless when the list objects are mostly reused. Synchronization with atomic operations should be done in order to allow threads to unregister an entry from other thread cache. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:42 +00:00
/* Init push vlan action list. */
net/mlx5: remove cache term from the list utility The internal mlx5 list tool is used mainly when the list objects need to be synchronized between multiple threads. The "cache" term is used in the internal mlx5 list API. Next enhancements on this tool will use the "cache" term for per thread cache management. To prevent confusing, remove the current "cache" term from the API's names. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:41 +00:00
snprintf(s, sizeof(s), "%s_push_vlan_action_list", sh->ibdev_name);
common/mlx5: add per-lcore sharing flag in object list Without lcores_share flag, mlx5 PMD was sharing the rdma-core objects between all lcores. Having lcores_share flag disabled, means each lcore will have its own objects, which will eventually lead to increased insertion/deletion rates. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:48 +00:00
sh->push_vlan_action_list = mlx5_list_create(s, sh, true,
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
flow_dv_push_vlan_create_cb,
flow_dv_push_vlan_match_cb,
flow_dv_push_vlan_remove_cb,
flow_dv_push_vlan_clone_cb,
flow_dv_push_vlan_clone_free_cb);
if (!sh->push_vlan_action_list)
goto error;
net/mlx5: add per-lcore cache to the list utility When mlx5 list object is accessed by multiple cores, the list lock counter is all the time written by all the cores what increases cache misses in the memory caches. In addition, when one thread accesses the list for add\remove\lookup operation, all the other threads coming to do an operation in the list are stuck in the lock. Add per lcore cache to allow thread manipulations to be lockless when the list objects are mostly reused. Synchronization with atomic operations should be done in order to allow threads to unregister an entry from other thread cache. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:42 +00:00
/* Init sample action list. */
net/mlx5: remove cache term from the list utility The internal mlx5 list tool is used mainly when the list objects need to be synchronized between multiple threads. The "cache" term is used in the internal mlx5 list API. Next enhancements on this tool will use the "cache" term for per thread cache management. To prevent confusing, remove the current "cache" term from the API's names. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:41 +00:00
snprintf(s, sizeof(s), "%s_sample_action_list", sh->ibdev_name);
common/mlx5: add per-lcore sharing flag in object list Without lcores_share flag, mlx5 PMD was sharing the rdma-core objects between all lcores. Having lcores_share flag disabled, means each lcore will have its own objects, which will eventually lead to increased insertion/deletion rates. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:48 +00:00
sh->sample_action_list = mlx5_list_create(s, sh, true,
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
flow_dv_sample_create_cb,
flow_dv_sample_match_cb,
flow_dv_sample_remove_cb,
flow_dv_sample_clone_cb,
flow_dv_sample_clone_free_cb);
if (!sh->sample_action_list)
goto error;
net/mlx5: add per-lcore cache to the list utility When mlx5 list object is accessed by multiple cores, the list lock counter is all the time written by all the cores what increases cache misses in the memory caches. In addition, when one thread accesses the list for add\remove\lookup operation, all the other threads coming to do an operation in the list are stuck in the lock. Add per lcore cache to allow thread manipulations to be lockless when the list objects are mostly reused. Synchronization with atomic operations should be done in order to allow threads to unregister an entry from other thread cache. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:42 +00:00
/* Init dest array action list. */
net/mlx5: remove cache term from the list utility The internal mlx5 list tool is used mainly when the list objects need to be synchronized between multiple threads. The "cache" term is used in the internal mlx5 list API. Next enhancements on this tool will use the "cache" term for per thread cache management. To prevent confusing, remove the current "cache" term from the API's names. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:41 +00:00
snprintf(s, sizeof(s), "%s_dest_array_list", sh->ibdev_name);
common/mlx5: add per-lcore sharing flag in object list Without lcores_share flag, mlx5 PMD was sharing the rdma-core objects between all lcores. Having lcores_share flag disabled, means each lcore will have its own objects, which will eventually lead to increased insertion/deletion rates. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:48 +00:00
sh->dest_array_list = mlx5_list_create(s, sh, true,
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
flow_dv_dest_array_create_cb,
flow_dv_dest_array_match_cb,
flow_dv_dest_array_remove_cb,
flow_dv_dest_array_clone_cb,
flow_dv_dest_array_clone_free_cb);
if (!sh->dest_array_list)
goto error;
net/mlx5: add flex parser DevX object management The DevX flex parsers can be shared between representors within the same IB context. We should put the flex parser objects into the shared list and engage the standard mlx5_list_xxx API to manage ones. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-02 08:53:43 +00:00
/* Init shared flex parsers list, no need lcore_share */
snprintf(s, sizeof(s), "%s_flex_parsers_list", sh->ibdev_name);
sh->flex_parsers_dv = mlx5_list_create(s, sh, false,
mlx5_flex_parser_create_cb,
mlx5_flex_parser_match_cb,
mlx5_flex_parser_remove_cb,
mlx5_flex_parser_clone_cb,
mlx5_flex_parser_clone_free_cb);
if (!sh->flex_parsers_dv)
goto error;
net/mlx5: fix redundant Direct Verbs resources allocate All table, tag, header modify, header reformat are supported only on DV mode. For the OFED version doesn't support these, create the related redundant DV resources waste the memory. Add the code section in the HAVE_IBV_FLOW_DV_SUPPORT macro to avoid the redundant resources allocation. Fixes: 2eb4d0107acc ("net/mlx5: refactor PCI probing on Linux") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:34 +00:00
#endif
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifdef HAVE_MLX5DV_DR
void *domain;
/* Reference counter is zero, we should initialize structures. */
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
domain = mlx5_glue->dr_create_domain(sh->cdev->ctx,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
MLX5DV_DR_DOMAIN_TYPE_NIC_RX);
if (!domain) {
DRV_LOG(ERR, "ingress mlx5dv_dr_create_domain failed");
err = errno;
goto error;
}
sh->rx_domain = domain;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
domain = mlx5_glue->dr_create_domain(sh->cdev->ctx,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
MLX5DV_DR_DOMAIN_TYPE_NIC_TX);
if (!domain) {
DRV_LOG(ERR, "egress mlx5dv_dr_create_domain failed");
err = errno;
goto error;
}
sh->tx_domain = domain;
#ifdef HAVE_MLX5DV_DR_ESWITCH
if (priv->config.dv_esw_en) {
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
domain = mlx5_glue->dr_create_domain(sh->cdev->ctx,
MLX5DV_DR_DOMAIN_TYPE_FDB);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!domain) {
DRV_LOG(ERR, "FDB mlx5dv_dr_create_domain failed");
err = errno;
goto error;
}
sh->fdb_domain = domain;
net/mlx5: fix drop action for Direct Rules/Verbs There are multiple branches in rdma-core library backing the rte flows: - Verbs - Direct Verbs (DV) - Direct Rules (DR) The Verbs API always requires the specifying the queue even if there is the drop action in the flow, though the kernel optimizes out the actual queue usage for the flows containing the drop action. The PMD handles the dedicated Rx queue to provide Verbs API compatibility. The DV/DR API does not require explicit specifying the queue at the flow creation, but PMD still specified the dedicated drop queue as action. It performed the packet forwarding to the dummy queue (that was not polled at all) causing the steering pipeline resources usage and degrading the overall packet processing rate. For example, with inserted flow to drop all the ingress packets the statistics reported only 15Mpps of 64B packets were received over 100Gbps line. Since the Direct Rule API for E-Switch was introduced the rdma-core supports the dedicated drop action, that is recognized both for DV and DR and can be used for the entire device in unified fashion, regardless of steering domain. The similar drop action was introduced for E-Switch, the usage of this one can be extended for other steering domains, not for E-Switch's one only. This patch: - renames esw_drop_action to dr_drop_action to emphasize the global nature of the variable (not only E-Switch domain) - specifies this global drop action instead of dedicated drop queue for the DR/DV flows Fixes: 34fa7c0268e7 ("net/mlx5: add drop action to Direct Verbs E-Switch") Fixes: 65b3cd0dc39b ("net/mlx5: create global drop action") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-05 09:59:01 +00:00
}
/*
* The drop action is just some dummy placeholder in rdma-core. It
* does not belong to domains and has no any attributes, and, can be
* shared by the entire device.
*/
sh->dr_drop_action = mlx5_glue->dr_create_flow_action_drop();
if (!sh->dr_drop_action) {
DRV_LOG(ERR, "FDB mlx5dv_dr_create_flow_action_drop");
err = errno;
goto error;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
#endif
net/mlx5: optimize hash list table allocate on demand Currently, all the hash list tables are allocated during start up. Since different applications may only use dedicated limited actions, optimized the hash list table allocate on demand will save initial memory. This commit optimizes hash list table allocate on demand. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:57 +00:00
if (!sh->tunnel_hub && priv->config.dv_miss_info)
net/mlx5: implement tunnel offload Tunnel Offload API provides hardware independent, unified model to offload tunneled traffic. Key model elements are: - apply matches to both outer and inner packet headers during entire offload procedure; - restore outer header of partially offloaded packet; - model is implemented as a set of helper functions. Implementation details: * tunnel_offload PMD parameter must be set to 1 to enable the feature. * application cannot use MARK and META flow actions with tunnel. * offload JUMP action is restricted to steering tunnel rule only. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-25 14:08:09 +00:00
err = mlx5_alloc_tunnel_hub(sh);
if (err) {
DRV_LOG(ERR, "mlx5_alloc_tunnel_hub failed err=%d", err);
goto error;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (priv->config.reclaim_mode == MLX5_RCM_AGGR) {
mlx5_glue->dr_reclaim_domain_memory(sh->rx_domain, 1);
mlx5_glue->dr_reclaim_domain_memory(sh->tx_domain, 1);
if (sh->fdb_domain)
mlx5_glue->dr_reclaim_domain_memory(sh->fdb_domain, 1);
}
sh->pop_vlan_action = mlx5_glue->dr_create_flow_action_pop_vlan();
net/mlx5: control flow rules with identical pattern In order to allow\disallow configuring rules with identical patterns, the new device argument 'allow_duplicate_pattern' is introduced. If allow, these rules be inserted successfully and only the first rule take affect. If disallow, the first rule will be inserted and other rules be rejected. The default is to allow. Set it to 0 if disallow, for example: -a <PCI_BDF>,allow_duplicate_pattern=0 Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-06 08:12:27 +00:00
if (!priv->config.allow_duplicate_pattern) {
#ifndef HAVE_MLX5_DR_ALLOW_DUPLICATE
DRV_LOG(WARNING, "Disallow duplicate pattern is not supported - maybe old rdma-core version?");
#endif
mlx5_glue->dr_allow_duplicate_rules(sh->rx_domain, 0);
mlx5_glue->dr_allow_duplicate_rules(sh->tx_domain, 0);
if (sh->fdb_domain)
mlx5_glue->dr_allow_duplicate_rules(sh->fdb_domain, 0);
}
net/mlx5: support matching on VXLAN reserved field This adds matching on the reserved field of VXLAN header (the last 8-bits). The capability from rdma-core is detected by creating a dummy matcher using misc5 when the device is probed. For non-zero groups and FDB domain, the capability is detected from rdma-core, meanwhile for NIC domain group zero it's relying on the HCA_CAP from FW. Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Raslan Darawsheh <rasland@nvidia.com>
2021-07-13 12:09:19 +00:00
__mlx5_discovery_misc5_cap(priv);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#endif /* HAVE_MLX5DV_DR */
net/mlx5: create global default miss action This commit creates the global default miss action instead of maintain it in flow insertion time. This makes the action to be thread safe. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:29 +00:00
sh->default_miss_action =
mlx5_glue->dr_create_flow_action_default_miss();
if (!sh->default_miss_action)
DRV_LOG(WARNING, "Default miss action is not supported.");
net/mlx5: support shared Rx queue This patch introduces shared RxQ. All shared Rx queues with same group and queue ID share the same rxq_ctrl. Rxq_ctrl and rxq_data are shared, all queues from different member port share same WQ and CQ, essentially one Rx WQ, mbufs are filled into this singleton WQ. Shared rxq_data is set into device Rx queues of all member ports as RxQ object, used for receiving packets. Polling queue of any member ports returns packets of any member, mbuf->port is used to identify source port. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-04 12:33:19 +00:00
LIST_INIT(&sh->shared_rxqs);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return 0;
error:
/* Rollback the created objects. */
if (sh->rx_domain) {
mlx5_glue->dr_destroy_domain(sh->rx_domain);
sh->rx_domain = NULL;
}
if (sh->tx_domain) {
mlx5_glue->dr_destroy_domain(sh->tx_domain);
sh->tx_domain = NULL;
}
if (sh->fdb_domain) {
mlx5_glue->dr_destroy_domain(sh->fdb_domain);
sh->fdb_domain = NULL;
}
net/mlx5: fix drop action for Direct Rules/Verbs There are multiple branches in rdma-core library backing the rte flows: - Verbs - Direct Verbs (DV) - Direct Rules (DR) The Verbs API always requires the specifying the queue even if there is the drop action in the flow, though the kernel optimizes out the actual queue usage for the flows containing the drop action. The PMD handles the dedicated Rx queue to provide Verbs API compatibility. The DV/DR API does not require explicit specifying the queue at the flow creation, but PMD still specified the dedicated drop queue as action. It performed the packet forwarding to the dummy queue (that was not polled at all) causing the steering pipeline resources usage and degrading the overall packet processing rate. For example, with inserted flow to drop all the ingress packets the statistics reported only 15Mpps of 64B packets were received over 100Gbps line. Since the Direct Rule API for E-Switch was introduced the rdma-core supports the dedicated drop action, that is recognized both for DV and DR and can be used for the entire device in unified fashion, regardless of steering domain. The similar drop action was introduced for E-Switch, the usage of this one can be extended for other steering domains, not for E-Switch's one only. This patch: - renames esw_drop_action to dr_drop_action to emphasize the global nature of the variable (not only E-Switch domain) - specifies this global drop action instead of dedicated drop queue for the DR/DV flows Fixes: 34fa7c0268e7 ("net/mlx5: add drop action to Direct Verbs E-Switch") Fixes: 65b3cd0dc39b ("net/mlx5: create global drop action") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-05 09:59:01 +00:00
if (sh->dr_drop_action) {
mlx5_glue->destroy_flow_action(sh->dr_drop_action);
sh->dr_drop_action = NULL;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
if (sh->pop_vlan_action) {
mlx5_glue->destroy_flow_action(sh->pop_vlan_action);
sh->pop_vlan_action = NULL;
}
net/mlx5: manage header reformat actions with hashed list To manage encap decap header format actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-09-16 10:19:48 +00:00
if (sh->encaps_decaps) {
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->encaps_decaps);
net/mlx5: manage header reformat actions with hashed list To manage encap decap header format actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-09-16 10:19:48 +00:00
sh->encaps_decaps = NULL;
}
net/mlx5: manage modify actions with hashed list To manage header modify actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-31 03:34:18 +00:00
if (sh->modify_cmds) {
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->modify_cmds);
net/mlx5: manage modify actions with hashed list To manage header modify actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-31 03:34:18 +00:00
sh->modify_cmds = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (sh->tag_table) {
/* tags should be destroyed with flow before. */
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->tag_table);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
sh->tag_table = NULL;
}
net/mlx5: implement tunnel offload Tunnel Offload API provides hardware independent, unified model to offload tunneled traffic. Key model elements are: - apply matches to both outer and inner packet headers during entire offload procedure; - restore outer header of partially offloaded packet; - model is implemented as a set of helper functions. Implementation details: * tunnel_offload PMD parameter must be set to 1 to enable the feature. * application cannot use MARK and META flow actions with tunnel. * offload JUMP action is restricted to steering tunnel rule only. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-25 14:08:09 +00:00
if (sh->tunnel_hub) {
mlx5_release_tunnel_hub(sh, priv->dev_port);
sh->tunnel_hub = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_free_table_hash_list(priv);
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
if (sh->port_id_action_list) {
mlx5_list_destroy(sh->port_id_action_list);
sh->port_id_action_list = NULL;
}
if (sh->push_vlan_action_list) {
mlx5_list_destroy(sh->push_vlan_action_list);
sh->push_vlan_action_list = NULL;
}
if (sh->sample_action_list) {
mlx5_list_destroy(sh->sample_action_list);
sh->sample_action_list = NULL;
}
if (sh->dest_array_list) {
mlx5_list_destroy(sh->dest_array_list);
sh->dest_array_list = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return err;
}
/**
* Destroy DR related data within private structure.
*
* @param[in] priv
* Pointer to the private device data structure.
*/
void
mlx5_os_free_shared_dr(struct mlx5_priv *priv)
{
net/mlx5: fix port shared data reference count When probe a representor, tag cache hash table and modification cache hash table allocated memory upon each port, overwrote previous existing cache in shared context data. This patch moves reference check of shared data prior to hash table allocation to avoid such issue. Fixes: 6801116688fe ("net/mlx5: fix multiple flow table hash list") Fixes: 1ef4cdef2682 ("net/mlx5: fix flow tag hash list conversion") Cc: stable@dpdk.org Acked-by: Matan Azrad <matan@nvidia.com> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
2020-10-21 11:15:23 +00:00
struct mlx5_dev_ctx_shared *sh = priv->sh;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
net/mlx5: fix port shared data reference count When probe a representor, tag cache hash table and modification cache hash table allocated memory upon each port, overwrote previous existing cache in shared context data. This patch moves reference check of shared data prior to hash table allocation to avoid such issue. Fixes: 6801116688fe ("net/mlx5: fix multiple flow table hash list") Fixes: 1ef4cdef2682 ("net/mlx5: fix flow tag hash list conversion") Cc: stable@dpdk.org Acked-by: Matan Azrad <matan@nvidia.com> Signed-off-by: Xueming Li <xuemingl@nvidia.com>
2020-10-21 11:15:23 +00:00
MLX5_ASSERT(sh && sh->refcnt);
if (sh->refcnt > 1)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return;
net/mlx5: support shared Rx queue This patch introduces shared RxQ. All shared Rx queues with same group and queue ID share the same rxq_ctrl. Rxq_ctrl and rxq_data are shared, all queues from different member port share same WQ and CQ, essentially one Rx WQ, mbufs are filled into this singleton WQ. Shared rxq_data is set into device Rx queues of all member ports as RxQ object, used for receiving packets. Polling queue of any member ports returns packets of any member, mbuf->port is used to identify source port. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-04 12:33:19 +00:00
MLX5_ASSERT(LIST_EMPTY(&sh->shared_rxqs));
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifdef HAVE_MLX5DV_DR
if (sh->rx_domain) {
mlx5_glue->dr_destroy_domain(sh->rx_domain);
sh->rx_domain = NULL;
}
if (sh->tx_domain) {
mlx5_glue->dr_destroy_domain(sh->tx_domain);
sh->tx_domain = NULL;
}
#ifdef HAVE_MLX5DV_DR_ESWITCH
if (sh->fdb_domain) {
mlx5_glue->dr_destroy_domain(sh->fdb_domain);
sh->fdb_domain = NULL;
}
net/mlx5: fix drop action for Direct Rules/Verbs There are multiple branches in rdma-core library backing the rte flows: - Verbs - Direct Verbs (DV) - Direct Rules (DR) The Verbs API always requires the specifying the queue even if there is the drop action in the flow, though the kernel optimizes out the actual queue usage for the flows containing the drop action. The PMD handles the dedicated Rx queue to provide Verbs API compatibility. The DV/DR API does not require explicit specifying the queue at the flow creation, but PMD still specified the dedicated drop queue as action. It performed the packet forwarding to the dummy queue (that was not polled at all) causing the steering pipeline resources usage and degrading the overall packet processing rate. For example, with inserted flow to drop all the ingress packets the statistics reported only 15Mpps of 64B packets were received over 100Gbps line. Since the Direct Rule API for E-Switch was introduced the rdma-core supports the dedicated drop action, that is recognized both for DV and DR and can be used for the entire device in unified fashion, regardless of steering domain. The similar drop action was introduced for E-Switch, the usage of this one can be extended for other steering domains, not for E-Switch's one only. This patch: - renames esw_drop_action to dr_drop_action to emphasize the global nature of the variable (not only E-Switch domain) - specifies this global drop action instead of dedicated drop queue for the DR/DV flows Fixes: 34fa7c0268e7 ("net/mlx5: add drop action to Direct Verbs E-Switch") Fixes: 65b3cd0dc39b ("net/mlx5: create global drop action") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-05 09:59:01 +00:00
if (sh->dr_drop_action) {
mlx5_glue->destroy_flow_action(sh->dr_drop_action);
sh->dr_drop_action = NULL;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
#endif
if (sh->pop_vlan_action) {
mlx5_glue->destroy_flow_action(sh->pop_vlan_action);
sh->pop_vlan_action = NULL;
}
#endif /* HAVE_MLX5DV_DR */
net/mlx5: create global default miss action This commit creates the global default miss action instead of maintain it in flow insertion time. This makes the action to be thread safe. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:29 +00:00
if (sh->default_miss_action)
mlx5_glue->destroy_flow_action
(sh->default_miss_action);
net/mlx5: manage header reformat actions with hashed list To manage encap decap header format actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-09-16 10:19:48 +00:00
if (sh->encaps_decaps) {
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->encaps_decaps);
net/mlx5: manage header reformat actions with hashed list To manage encap decap header format actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-09-16 10:19:48 +00:00
sh->encaps_decaps = NULL;
}
net/mlx5: manage modify actions with hashed list To manage header modify actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-31 03:34:18 +00:00
if (sh->modify_cmds) {
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->modify_cmds);
net/mlx5: manage modify actions with hashed list To manage header modify actions mlx5 PMD used the single linked list and lookup and insertion operations took too long times if there were millions of objects and this impacted the flow insertion/deletion rate. In order to optimize the performance the hashed list is engaged. The list implementation is updated to support non-unique keys with few collisions. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-31 03:34:18 +00:00
sh->modify_cmds = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (sh->tag_table) {
/* tags should be destroyed with flow before. */
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(sh->tag_table);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
sh->tag_table = NULL;
}
net/mlx5: implement tunnel offload Tunnel Offload API provides hardware independent, unified model to offload tunneled traffic. Key model elements are: - apply matches to both outer and inner packet headers during entire offload procedure; - restore outer header of partially offloaded packet; - model is implemented as a set of helper functions. Implementation details: * tunnel_offload PMD parameter must be set to 1 to enable the feature. * application cannot use MARK and META flow actions with tunnel. * offload JUMP action is restricted to steering tunnel rule only. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-25 14:08:09 +00:00
if (sh->tunnel_hub) {
mlx5_release_tunnel_hub(sh, priv->dev_port);
sh->tunnel_hub = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_free_table_hash_list(priv);
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
if (sh->port_id_action_list) {
mlx5_list_destroy(sh->port_id_action_list);
sh->port_id_action_list = NULL;
}
if (sh->push_vlan_action_list) {
mlx5_list_destroy(sh->push_vlan_action_list);
sh->push_vlan_action_list = NULL;
}
if (sh->sample_action_list) {
mlx5_list_destroy(sh->sample_action_list);
sh->sample_action_list = NULL;
}
if (sh->dest_array_list) {
mlx5_list_destroy(sh->dest_array_list);
sh->dest_array_list = NULL;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: refactor multi-process communication 1. The shared data communication between the primary and the secondary processes is implemented using Linux API. Move the Linux API code under linux directory (file linux/mlx5_os.c). 2. File net/mlx5/mlx5_mp.c handles requests to the primary and secondary processes (e.g. start_rxtx, stop_rxtx). It is Linux based so it is moved under linux (new file linux/mlx5_mp_os.c). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:15 +00:00
/**
* Initialize shared data between primary and secondary process.
*
* A memzone is reserved by primary process and secondary processes attach to
* the memzone.
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
static int
mlx5_init_shared_data(void)
{
const struct rte_memzone *mz;
int ret = 0;
rte_spinlock_lock(&mlx5_shared_data_lock);
if (mlx5_shared_data == NULL) {
if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
/* Allocate shared memory. */
mz = rte_memzone_reserve(MZ_MLX5_PMD_SHARED_DATA,
sizeof(*mlx5_shared_data),
SOCKET_ID_ANY, 0);
if (mz == NULL) {
DRV_LOG(ERR,
"Cannot allocate mlx5 shared data");
ret = -rte_errno;
goto error;
}
mlx5_shared_data = mz->addr;
memset(mlx5_shared_data, 0, sizeof(*mlx5_shared_data));
rte_spinlock_init(&mlx5_shared_data->lock);
} else {
/* Lookup allocated shared memory. */
mz = rte_memzone_lookup(MZ_MLX5_PMD_SHARED_DATA);
if (mz == NULL) {
DRV_LOG(ERR,
"Cannot attach mlx5 shared data");
ret = -rte_errno;
goto error;
}
mlx5_shared_data = mz->addr;
memset(&mlx5_local_data, 0, sizeof(mlx5_local_data));
}
}
error:
rte_spinlock_unlock(&mlx5_shared_data_lock);
return ret;
}
/**
* PMD global initialization.
*
* Independent from individual device, this function initializes global
* per-PMD data structures distinguishing primary and secondary processes.
* Hence, each initialization is called once per a process.
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
static int
mlx5_init_once(void)
{
struct mlx5_shared_data *sd;
struct mlx5_local_data *ld = &mlx5_local_data;
int ret = 0;
if (mlx5_init_shared_data())
return -rte_errno;
sd = mlx5_shared_data;
MLX5_ASSERT(sd);
rte_spinlock_lock(&sd->lock);
switch (rte_eal_process_type()) {
case RTE_PROC_PRIMARY:
if (sd->init_done)
break;
ret = mlx5_mp_init_primary(MLX5_MP_NAME,
mlx5_mp_os_primary_handle);
if (ret)
goto out;
sd->init_done = true;
break;
case RTE_PROC_SECONDARY:
if (ld->init_done)
break;
ret = mlx5_mp_init_secondary(MLX5_MP_NAME,
mlx5_mp_os_secondary_handle);
if (ret)
goto out;
++sd->secondary_cnt;
ld->init_done = true;
break;
default:
break;
}
out:
rte_spinlock_unlock(&sd->lock);
return ret;
}
net/mlx5: workaround drop action with old kernel Currently, there are two types of drop action implementation in the PMD. One is the DR (Direct Rules) dummy placeholder drop action and another is the dedicated dummy queue drop action. When creates flow on the root table with DR drop action, the action will be converted to MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute in rdma-core. In some inbox systems, MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute may not be supported in the kernel driver. Create flow with drop action on the root table will be failed as it is not supported. In this case, the dummy queue drop action should be used instead of DR dummy placeholder drop action. This commit adds the DR drop action support detect on the root table. If MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs is not supported in the system, a dummy queue will be used as drop action. Fixes: da845ae9d7c1 ("net/mlx5: fix drop action for Direct Rules/Verbs") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-08-02 14:30:24 +00:00
/**
* DR flow drop action support detect.
*
* @param dev
* Pointer to rte_eth_dev structure.
*
*/
static void
mlx5_flow_drop_action_config(struct rte_eth_dev *dev __rte_unused)
{
#ifdef HAVE_MLX5DV_DR
struct mlx5_priv *priv = dev->data->dev_private;
if (!priv->config.dv_flow_en || !priv->sh->dr_drop_action)
return;
/**
* DR supports drop action placeholder when it is supported;
* otherwise, use the queue drop action.
*/
net/mlx5: optimize device spawn time with representors During the device spawn process, mlx5 PMD queried the available flow priorities by calling mlx5_flow_discover_priorities, queried if the DR drop action was supported on the root table by calling the mlx5_flow_discover_dr_action_support routine, and queried the availability of metadata register C by calling mlx5_flow_discover_mreg_c These functions created the test flows to get the supported fields, and at the end destroyed the test flows. The test flows in the first two functions was created on the root table. If the device was spawned with multiple representors, these test flows were created and destroyed on each representor as well. The above operations took a significant amount of init time during the device spawn. This patch optimizes the device discover functions, if there is the device with multiple representors (VF/SF) being spawned, the priority and drop action and metadata register support check can be done only ones and check results can be shared for all representors. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-27 10:35:10 +00:00
if (!priv->sh->drop_action_check_flag) {
if (!mlx5_flow_discover_dr_action_support(dev))
priv->sh->dr_drop_action_en = 1;
priv->sh->drop_action_check_flag = 1;
}
if (priv->sh->dr_drop_action_en)
net/mlx5: workaround drop action with old kernel Currently, there are two types of drop action implementation in the PMD. One is the DR (Direct Rules) dummy placeholder drop action and another is the dedicated dummy queue drop action. When creates flow on the root table with DR drop action, the action will be converted to MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute in rdma-core. In some inbox systems, MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute may not be supported in the kernel driver. Create flow with drop action on the root table will be failed as it is not supported. In this case, the dummy queue drop action should be used instead of DR dummy placeholder drop action. This commit adds the DR drop action support detect on the root table. If MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs is not supported in the system, a dummy queue will be used as drop action. Fixes: da845ae9d7c1 ("net/mlx5: fix drop action for Direct Rules/Verbs") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-08-02 14:30:24 +00:00
priv->root_drop_action = priv->sh->dr_drop_action;
net/mlx5: optimize device spawn time with representors During the device spawn process, mlx5 PMD queried the available flow priorities by calling mlx5_flow_discover_priorities, queried if the DR drop action was supported on the root table by calling the mlx5_flow_discover_dr_action_support routine, and queried the availability of metadata register C by calling mlx5_flow_discover_mreg_c These functions created the test flows to get the supported fields, and at the end destroyed the test flows. The test flows in the first two functions was created on the root table. If the device was spawned with multiple representors, these test flows were created and destroyed on each representor as well. The above operations took a significant amount of init time during the device spawn. This patch optimizes the device discover functions, if there is the device with multiple representors (VF/SF) being spawned, the priority and drop action and metadata register support check can be done only ones and check results can be shared for all representors. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-27 10:35:10 +00:00
else
priv->root_drop_action = priv->drop_queue.hrxq->action;
net/mlx5: workaround drop action with old kernel Currently, there are two types of drop action implementation in the PMD. One is the DR (Direct Rules) dummy placeholder drop action and another is the dedicated dummy queue drop action. When creates flow on the root table with DR drop action, the action will be converted to MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute in rdma-core. In some inbox systems, MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute may not be supported in the kernel driver. Create flow with drop action on the root table will be failed as it is not supported. In this case, the dummy queue drop action should be used instead of DR dummy placeholder drop action. This commit adds the DR drop action support detect on the root table. If MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs is not supported in the system, a dummy queue will be used as drop action. Fixes: da845ae9d7c1 ("net/mlx5: fix drop action for Direct Rules/Verbs") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-08-02 14:30:24 +00:00
#endif
}
net/mlx5: fix imissed statistics The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-25 10:45:01 +00:00
static void
mlx5_queue_counter_id_prepare(struct rte_eth_dev *dev)
{
struct mlx5_priv *priv = dev->data->dev_private;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
void *ctx = priv->sh->cdev->ctx;
net/mlx5: fix imissed statistics The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-25 10:45:01 +00:00
priv->q_counters = mlx5_devx_cmd_queue_counter_alloc(ctx);
if (!priv->q_counters) {
struct ibv_cq *cq = mlx5_glue->create_cq(ctx, 1, NULL, NULL, 0);
struct ibv_wq *wq;
DRV_LOG(DEBUG, "Port %d queue counter object cannot be created "
"by DevX - fall-back to use the kernel driver global "
"queue counter.", dev->data->port_id);
/* Create WQ by kernel and query its queue counter ID. */
if (cq) {
wq = mlx5_glue->create_wq(ctx,
&(struct ibv_wq_init_attr){
.wq_type = IBV_WQT_RQ,
.max_wr = 1,
.max_sge = 1,
common/mlx5: share protection domain object Create shared Protection Domain in common area and add it and its PDN as fields of common device structure. Use this Protection Domain in all drivers and remove the PD and PDN fields from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:54 +00:00
.pd = priv->sh->cdev->pd,
net/mlx5: fix imissed statistics The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-25 10:45:01 +00:00
.cq = cq,
});
if (wq) {
/* Counter is assigned only on RDY state. */
int ret = mlx5_glue->modify_wq(wq,
&(struct ibv_wq_attr){
.attr_mask = IBV_WQ_ATTR_STATE,
.wq_state = IBV_WQS_RDY,
});
if (ret == 0)
mlx5_devx_cmd_wq_query(wq,
&priv->counter_set_id);
claim_zero(mlx5_glue->destroy_wq(wq));
}
claim_zero(mlx5_glue->destroy_cq(cq));
}
} else {
priv->counter_set_id = priv->q_counters->id;
}
if (priv->counter_set_id == 0)
DRV_LOG(INFO, "Part of the port %d statistics will not be "
"available.", dev->data->port_id);
}
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
/**
* Check if representor spawn info match devargs.
*
* @param spawn
* Verbs device parameters (name, port, switch_info) to spawn.
* @param eth_da
* Device devargs to probe.
*
* @return
* Match result.
*/
static bool
mlx5_representor_match(struct mlx5_dev_spawn_data *spawn,
struct rte_eth_devargs *eth_da)
{
struct mlx5_switch_info *switch_info = &spawn->info;
unsigned int p, f;
uint16_t id;
net/mlx5: probe host PF representor with sub-function To simplify BlueField HPF representor(vf[-1]) probe, this patch allows probe it with "sf" syntax: "sf[-1]". Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:15 +00:00
uint16_t repr_id = mlx5_representor_id_encode(switch_info,
eth_da->type);
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
switch (eth_da->type) {
case RTE_ETH_REPRESENTOR_SF:
net/mlx5: probe host PF representor with sub-function To simplify BlueField HPF representor(vf[-1]) probe, this patch allows probe it with "sf" syntax: "sf[-1]". Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:15 +00:00
if (!(spawn->info.port_name == -1 &&
switch_info->name_type ==
MLX5_PHYS_PORT_NAME_TYPE_PFHPF) &&
switch_info->name_type != MLX5_PHYS_PORT_NAME_TYPE_PFSF) {
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
rte_errno = EBUSY;
return false;
}
break;
case RTE_ETH_REPRESENTOR_VF:
/* Allows HPF representor index -1 as exception. */
if (!(spawn->info.port_name == -1 &&
switch_info->name_type ==
MLX5_PHYS_PORT_NAME_TYPE_PFHPF) &&
switch_info->name_type != MLX5_PHYS_PORT_NAME_TYPE_PFVF) {
rte_errno = EBUSY;
return false;
}
break;
case RTE_ETH_REPRESENTOR_NONE:
rte_errno = EBUSY;
return false;
default:
rte_errno = ENOTSUP;
DRV_LOG(ERR, "unsupported representor type");
return false;
}
/* Check representor ID: */
for (p = 0; p < eth_da->nb_ports; ++p) {
if (spawn->pf_bond < 0) {
/* For non-LAG mode, allow and ignore pf. */
switch_info->pf_num = eth_da->ports[p];
net/mlx5: probe host PF representor with sub-function To simplify BlueField HPF representor(vf[-1]) probe, this patch allows probe it with "sf" syntax: "sf[-1]". Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:15 +00:00
repr_id = mlx5_representor_id_encode(switch_info,
eth_da->type);
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
}
for (f = 0; f < eth_da->nb_representor_ports; ++f) {
id = MLX5_REPRESENTOR_ID
(eth_da->ports[p], eth_da->type,
eth_da->representor_ports[f]);
if (repr_id == id)
return true;
}
}
rte_errno = EBUSY;
return false;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/**
* Spawn an Ethernet device from Verbs information.
*
* @param dpdk_dev
* Backing DPDK device.
* @param spawn
* Verbs device parameters (name, port, switch_info) to spawn.
* @param config
* Device configuration parameters.
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
* @param eth_da
net/mlx5: support sub-function representor This patch adds support for SF representor. Similar to VF representor, switch port name of SF representor in phys_port_name sysfs key is "pf<x>sf<y>". Device representor argument is "representors=sf[list]", list member could be mix of instance and range. Example: representors=sf[0,2,4,8-12,-1] To probe VF representor and SF representor, need to separate into 2 devices: -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:08 +00:00
* Device arguments.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*
* @return
* A valid Ethernet device object on success, NULL otherwise and rte_errno
* is set. The following errors are defined:
*
* EBUSY: device is not supposed to be spawned.
* EEXIST: device is already spawned
*/
static struct rte_eth_dev *
mlx5_dev_spawn(struct rte_device *dpdk_dev,
struct mlx5_dev_spawn_data *spawn,
net/mlx5: support sub-function representor This patch adds support for SF representor. Similar to VF representor, switch port name of SF representor in phys_port_name sysfs key is "pf<x>sf<y>". Device representor argument is "representors=sf[list]", list member could be mix of instance and range. Example: representors=sf[0,2,4,8-12,-1] To probe VF representor and SF representor, need to separate into 2 devices: -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:08 +00:00
struct mlx5_dev_config *config,
struct rte_eth_devargs *eth_da)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
{
const struct mlx5_switch_info *switch_info = &spawn->info;
struct mlx5_dev_ctx_shared *sh = NULL;
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
struct mlx5_hca_attr *hca_attr = &spawn->cdev->config.hca_attr;
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
struct ibv_port_attr port_attr = { .state = IBV_PORT_NOP };
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
struct rte_eth_dev *eth_dev = NULL;
struct mlx5_priv *priv = NULL;
int err = 0;
struct rte_ether_addr mac;
char name[RTE_ETH_NAME_MAX_LEN];
int own_domain_id = 0;
uint16_t port_id;
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
struct mlx5_port_info vport_info = { .query_flags = 0 };
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
int nl_rdma = -1;
net/mlx5: replace flow list with indexed pool The flow list is used to save the create flows and to be used only when port closes all the flows need to be flushed. This commit takes advantage of the index pool foreach operation to flush all the allocated flows. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:39 +00:00
int i;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Determine if this port representor is supposed to be spawned. */
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
if (switch_info->representor && dpdk_dev->devargs &&
!mlx5_representor_match(spawn, eth_da))
return NULL;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Build device name. */
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
if (spawn->pf_bond < 0) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Single device. */
if (!switch_info->representor)
strlcpy(name, dpdk_dev->name, sizeof(name));
else
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
err = snprintf(name, sizeof(name), "%s_representor_%s%u",
net/mlx5: support sub-function representor This patch adds support for SF representor. Similar to VF representor, switch port name of SF representor in phys_port_name sysfs key is "pf<x>sf<y>". Device representor argument is "representors=sf[list]", list member could be mix of instance and range. Example: representors=sf[0,2,4,8-12,-1] To probe VF representor and SF representor, need to separate into 2 devices: -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:08 +00:00
dpdk_dev->name,
switch_info->name_type ==
MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
switch_info->port_name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
} else {
/* Bonding device. */
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
if (!switch_info->representor) {
err = snprintf(name, sizeof(name), "%s_%s",
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
dpdk_dev->name, spawn->phys_dev_name);
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
} else {
err = snprintf(name, sizeof(name), "%s_%s_representor_c%dpf%d%s%u",
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
dpdk_dev->name, spawn->phys_dev_name,
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
switch_info->ctrl_num,
switch_info->pf_num,
switch_info->name_type ==
MLX5_PHYS_PORT_NAME_TYPE_PFSF ? "sf" : "vf",
switch_info->port_name);
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
if (err >= (int)sizeof(name))
DRV_LOG(WARNING, "device name overflow %s", name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* check if the device is already spawned */
if (rte_eth_dev_get_port_by_name(name, &port_id) == 0) {
rte_errno = EEXIST;
return NULL;
}
DRV_LOG(DEBUG, "naming Ethernet device \"%s\"", name);
if (rte_eal_process_type() == RTE_PROC_SECONDARY) {
struct mlx5_mp_id mp_id;
eth_dev = rte_eth_dev_attach_secondary(name);
if (eth_dev == NULL) {
DRV_LOG(ERR, "can not attach rte ethdev");
rte_errno = ENOMEM;
return NULL;
}
net/mlx5: revert setting bonding representor to first PF With kernel bonding, representors on second PF are being probed by devargs: <primary_bdf>,representor=pf1vf<N> No need to save primary PF port ID and lookup when probing sibling ports, revert patch [1] [1]: commit e6818853c022 ("net/mlx5: set representor to first PF in bonding mode") Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:09 +00:00
eth_dev->device = dpdk_dev;
net/mlx5: unify operations for all OS There are three types of eth_dev_ops: primary, secondary and isolate represented in three callback tables per OS. In this commit the OS specific eth dev tables are unified into shared tables in file mlx5.c. Starting from this commit all operating systems must implement the same eth dev APIs. In case an OS does not support an API - it can return in its implementation an error ENOTSUP. Fixes: 042f5c94fd3a ("net/mlx5: refactor device operations for Linux") Cc: stable@dpdk.org Signed-off-by: Ophir Munk <ophirmu@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-12-28 12:32:56 +00:00
eth_dev->dev_ops = &mlx5_dev_sec_ops;
ethdev: move inline device operations This patch is a preparation to hide the 'struct eth_dev_ops' from applications by moving some device operations from 'struct eth_dev_ops' to 'struct rte_eth_dev'. Mentioned ethdev APIs are in the data path and implemented as inline because of performance reasons. Exposing 'struct eth_dev_ops' to applications is bad because it is a contract between ethdev and PMDs, not really needs to be known by applications, also changes in the struct causing ABI breakages which shouldn't. To be able to both keep APIs inline and hide the 'struct eth_dev_ops', moving device operations used in ethdev inline APIs to 'struct rte_eth_dev' to the same level with Rx/Tx burst functions. The list of dev_ops moved: eth_rx_queue_count_t rx_queue_count; eth_rx_descriptor_done_t rx_descriptor_done; eth_rx_descriptor_status_t rx_descriptor_status; eth_tx_descriptor_status_t tx_descriptor_status; Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com> Acked-by: David Marchand <david.marchand@redhat.com> Acked-by: Sachin Saxena <sachin.saxena@nxp.com>
2020-09-09 13:01:43 +00:00
eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
err = mlx5_proc_priv_init(eth_dev);
if (err)
return NULL;
net/mlx5: support mempool registration When the first port in a given protection domain (PD) starts, install a mempool event callback for this PD and register all existing memory regions (MR) for it. When the last port in a PD closes, remove the callback and unregister all mempools for this PD. This behavior can be switched off with a new devarg: mr_mempool_reg_en. On TX slow path, i.e. when an MR key for the address of the buffer to send is not in the local cache, first try to retrieve it from the database of registered mempools. Supported are direct and indirect mbufs, as well as externally-attached ones from MLX5 MPRQ feature. Lookup in the database of non-mempool memory is used as the last resort. RX mempools are registered regardless of the devarg value. On RX data path only the local cache and the mempool database is used. If implicit mempool registration is disabled, these mempools are unregistered at port stop, releasing the MRs. Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-18 22:43:53 +00:00
mlx5_mp_id_init(&mp_id, eth_dev->data->port_id);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Receive command fd from primary process */
err = mlx5_mp_req_verbs_cmd_fd(&mp_id);
if (err < 0)
goto err_secondary;
/* Remap UAR for Tx queues. */
err = mlx5_tx_uar_init_secondary(eth_dev, err);
if (err)
goto err_secondary;
/*
* Ethdev pointer is still required as input since
* the primary device is not accessible from the
* secondary process.
*/
eth_dev->rx_pkt_burst = mlx5_select_rx_function(eth_dev);
eth_dev->tx_pkt_burst = mlx5_select_tx_function(eth_dev);
return eth_dev;
err_secondary:
mlx5_dev_close(eth_dev);
return NULL;
}
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
/* Process parameters. */
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
err = mlx5_args(config, dpdk_dev->devargs);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (err) {
DRV_LOG(ERR, "failed to process device arguments: %s",
strerror(rte_errno));
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
return NULL;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
sh = mlx5_alloc_shared_dev_ctx(spawn, config);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!sh)
return NULL;
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
/* Update final values for devargs before check sibling config. */
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->dv_flow_en && !sh->dev_cap.dv_flow_en) {
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
DRV_LOG(WARNING, "DV flow is not supported.");
config->dv_flow_en = 0;
}
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->dv_esw_en && !sh->dev_cap.dv_esw_en) {
DRV_LOG(WARNING, "E-Switch DV flow is not supported.");
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
config->dv_esw_en = 0;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
}
net/mlx5: add E-Switch mode flag This patch adds in SH structure a flag which indicates whether is E-Switch mode. When configure "dv_esw_en" from devargs, it is enabled only when is E-switch mode. So, since dv_esw_en has been configure, it is enough to check if "dv_esw_en" is valid. This patch also removes E-Switch mode check when "dv_esw_en" is checked too. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:04 +00:00
if (config->dv_miss_info && config->dv_esw_en)
config->dv_xmeta_en = MLX5_XMETA_MODE_META16;
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
if (!config->dv_esw_en &&
config->dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) {
DRV_LOG(WARNING,
"Metadata mode %u is not supported (no E-Switch).",
config->dv_xmeta_en);
config->dv_xmeta_en = MLX5_XMETA_MODE_LEGACY;
}
/* Check sibling device configurations. */
err = mlx5_dev_check_sibling_config(sh, config, dpdk_dev);
if (err)
goto error;
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
nl_rdma = mlx5_nl_init(NETLINK_RDMA);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Check port status. */
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
if (spawn->phys_port <= UINT8_MAX) {
/* Legacy Verbs api only support u8 port number. */
err = mlx5_glue->query_port(sh->cdev->ctx, spawn->phys_port,
&port_attr);
if (err) {
DRV_LOG(ERR, "port query failed: %s", strerror(err));
goto error;
}
if (port_attr.link_layer != IBV_LINK_LAYER_ETHERNET) {
DRV_LOG(ERR, "port is not configured in Ethernet mode");
err = EINVAL;
goto error;
}
} else if (nl_rdma >= 0) {
/* IB doesn't allow more than 255 ports, must be Ethernet. */
err = mlx5_nl_port_state(nl_rdma,
spawn->phys_dev_name,
spawn->phys_port);
if (err < 0) {
DRV_LOG(INFO, "Failed to get netlink port state: %s",
strerror(rte_errno));
err = -rte_errno;
goto error;
}
port_attr.state = (enum ibv_port_state)err;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
if (port_attr.state != IBV_PORT_ACTIVE)
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
DRV_LOG(INFO, "port is not active: \"%s\" (%d)",
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_glue->port_state_str(port_attr.state),
port_attr.state);
/* Allocate private eth device data. */
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
priv = mlx5_malloc(MLX5_MEM_ZERO | MLX5_MEM_RTE,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
sizeof(*priv),
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (priv == NULL) {
DRV_LOG(ERR, "priv allocation failure");
err = ENOMEM;
goto error;
}
priv->sh = sh;
net/mlx5: rename ib in names Renames in this commit: mlx5_ibv_list -> mlx5_dev_ctx_list mlx5_alloc_shared_ibctx -> mlx5_alloc_shared_dev_ctx mlx5_free_shared_ibctx -> mlx5_free_shared_dev_ctx mlx5_ibv_shared_port -> mlx5_dev_shared_port ibv_port -> dev_port Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:27 +00:00
priv->dev_port = spawn->phys_port;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
priv->pci_dev = spawn->pci_dev;
priv->mtu = RTE_ETHER_MTU;
/* Some internal functions rely on Netlink sockets, open them now. */
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
priv->nl_socket_rdma = nl_rdma;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
priv->nl_socket_route = mlx5_nl_init(NETLINK_ROUTE);
priv->representor = !!switch_info->representor;
priv->master = !!switch_info->master;
priv->domain_id = RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID;
priv->vport_meta_tag = 0;
priv->vport_meta_mask = 0;
priv->pf_bond = spawn->pf_bond;
net/mlx5: fix port initialization of switch domain All active ports that belong to the same E-switch share domain_id value. Port initialization procedure searches through a database for existing port with matching properties. New domain_id allocated if match was not located. Otherwise, new port inherits existing domain_id. Port initialization did not pass enough info to search procedure to find existing matches. Therefore, each port was created with a private domain_id value. As the result, port_id flow action failed because it could not match ports in a rule to E-switch. The patch adds dpdk_dev with port properties to device search. Fixes: 56bb3c84e982 ("net/mlx5: reduce PCI dependency") Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-08-02 14:55:24 +00:00
DRV_LOG(DEBUG,
"dev_port=%u bus=%s pci=%s master=%d representor=%d pf_bond=%d\n",
priv->dev_port, dpdk_dev->bus->name,
priv->pci_dev ? priv->pci_dev->name : "NONE",
priv->master, priv->representor, priv->pf_bond);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
* If we have E-Switch we should determine the vport attributes.
* E-Switch may use either source vport field or reg_c[0] metadata
* register to match on vport index. The engaged part of metadata
* register is defined by mask.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*/
net/mlx5: add E-Switch mode flag This patch adds in SH structure a flag which indicates whether is E-Switch mode. When configure "dv_esw_en" from devargs, it is enabled only when is E-switch mode. So, since dv_esw_en has been configure, it is enough to check if "dv_esw_en" is valid. This patch also removes E-Switch mode check when "dv_esw_en" is checked too. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:04 +00:00
if (sh->esw_mode) {
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
err = mlx5_glue->devx_port_query(sh->cdev->ctx,
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
spawn->phys_port,
&vport_info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (err) {
DRV_LOG(WARNING,
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
"Cannot query devx port %d on device %s",
spawn->phys_port, spawn->phys_dev_name);
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
vport_info.query_flags = 0;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
}
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
if (vport_info.query_flags & MLX5_PORT_QUERY_REG_C0) {
priv->vport_meta_tag = vport_info.vport_meta_tag;
priv->vport_meta_mask = vport_info.vport_meta_mask;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!priv->vport_meta_mask) {
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR,
"vport zero mask for port %d on bonding device %s",
spawn->phys_port, spawn->phys_dev_name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
err = ENOTSUP;
goto error;
}
if (priv->vport_meta_tag & ~priv->vport_meta_mask) {
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR,
"Invalid vport tag for port %d on bonding device %s",
spawn->phys_port, spawn->phys_dev_name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
err = ENOTSUP;
goto error;
}
}
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
if (vport_info.query_flags & MLX5_PORT_QUERY_VPORT) {
priv->vport_id = vport_info.vport_id;
net/mlx5: add E-Switch mode flag This patch adds in SH structure a flag which indicates whether is E-Switch mode. When configure "dv_esw_en" from devargs, it is enabled only when is E-switch mode. So, since dv_esw_en has been configure, it is enough to check if "dv_esw_en" is valid. This patch also removes E-Switch mode check when "dv_esw_en" is checked too. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:04 +00:00
} else if (spawn->pf_bond >= 0 && sh->esw_mode) {
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR,
"Cannot deduce vport index for port %d on bonding device %s",
spawn->phys_port, spawn->phys_dev_name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
err = ENOTSUP;
goto error;
} else {
common/mlx5: use new port query API if available In order to get E-Switch vport identifiers the mlx5 PMD relies on two approaches: [a] use port query API if it is provided by rdma-core library [b] otherwise, deduce vport ids from the related VF index The latter is not reliable and may not work with newer kernel drivers and in some configurations (LAG), causing E-Switch malfunction. Hence, engaging the port query API is highly desirable. Depending on rdma-core version the port query API is: - very old OFED versions have no query API (approach [b]) - rdma-core OFED < 5.5 provides mlx5dv_query_devx_port, HAVE_MLX5DV_DR_DEVX_PORT flag is defined (approach [a]) - rdma-core OFED >= 5.5 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) - future OFED versions might remove mlx5dv_query_devx_port and HAVE_MLX5DV_DR_DEVX_PORT will not be defined - Upstream rdma-core < v35 has no port query API (approach [b]) - Upstream rdma-core >= v35 has mlx5dv_query_port, flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is defined (approach [a]) In order to support the new mlx5dv_query_port routine, the conditional compilation flag HAVE_MLX5DV_DR_DEVX_PORT_V35 is introduced by this patch. The flag HAVE_MLX5DV_DR_DEVX_PORT is kept for compatibility with previous rdma-core versions. Despite this patch is not a bugfix (it follows the introduced API variation in underlying library), it resolves the compatibility issue and is highly desired to be ported to DPDK LTS. Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-07 15:54:27 +00:00
/*
* Suppose vport index in compatible way. Kernel/rdma_core
* support single E-Switch per PF configurations only and
* vport_id field contains the vport index for associated VF,
* which is deduced from representor port name.
* For example, let's have the IB device port 10, it has
* attached network device eth0, which has port name attribute
* pf0vf2, we can deduce the VF number as 2, and set vport index
* as 3 (2+1). This assigning schema should be changed if the
* multiple E-Switch instances per PF configurations or/and PCI
* subfunctions are added.
*/
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
priv->vport_id = switch_info->representor ?
switch_info->port_name + 1 : -1;
}
net/mlx5: probe host PF representor with sub-function To simplify BlueField HPF representor(vf[-1]) probe, this patch allows probe it with "sf" syntax: "sf[-1]". Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:15 +00:00
priv->representor_id = mlx5_representor_id_encode(switch_info,
eth_da->type);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
* Look for sibling devices in order to reuse their switch domain
* if any, otherwise allocate one.
*/
net/mlx5: fix port initialization of switch domain All active ports that belong to the same E-switch share domain_id value. Port initialization procedure searches through a database for existing port with matching properties. New domain_id allocated if match was not located. Otherwise, new port inherits existing domain_id. Port initialization did not pass enough info to search procedure to find existing matches. Therefore, each port was created with a private domain_id value. As the result, port_id flow action failed because it could not match ports in a rule to E-switch. The patch adds dpdk_dev with port properties to device search. Fixes: 56bb3c84e982 ("net/mlx5: reduce PCI dependency") Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-08-02 14:55:24 +00:00
MLX5_ETH_FOREACH_DEV(port_id, dpdk_dev) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
const struct mlx5_priv *opriv =
rte_eth_devices[port_id].data->dev_private;
if (!opriv ||
opriv->sh != priv->sh ||
opriv->domain_id ==
RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID)
continue;
priv->domain_id = opriv->domain_id;
net/mlx5: fix port initialization of switch domain All active ports that belong to the same E-switch share domain_id value. Port initialization procedure searches through a database for existing port with matching properties. New domain_id allocated if match was not located. Otherwise, new port inherits existing domain_id. Port initialization did not pass enough info to search procedure to find existing matches. Therefore, each port was created with a private domain_id value. As the result, port_id flow action failed because it could not match ports in a rule to E-switch. The patch adds dpdk_dev with port properties to device search. Fixes: 56bb3c84e982 ("net/mlx5: reduce PCI dependency") Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-08-02 14:55:24 +00:00
DRV_LOG(DEBUG, "dev_port-%u inherit domain_id=%u\n",
priv->dev_port, priv->domain_id);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
break;
}
if (priv->domain_id == RTE_ETH_DEV_SWITCH_DOMAIN_ID_INVALID) {
err = rte_eth_switch_domain_alloc(&priv->domain_id);
if (err) {
err = rte_errno;
DRV_LOG(ERR, "unable to allocate switch domain: %s",
strerror(rte_errno));
goto error;
}
own_domain_id = 1;
net/mlx5: fix port initialization of switch domain All active ports that belong to the same E-switch share domain_id value. Port initialization procedure searches through a database for existing port with matching properties. New domain_id allocated if match was not located. Otherwise, new port inherits existing domain_id. Port initialization did not pass enough info to search procedure to find existing matches. Therefore, each port was created with a private domain_id value. As the result, port_id flow action failed because it could not match ports in a rule to E-switch. The patch adds dpdk_dev with port properties to device search. Fixes: 56bb3c84e982 ("net/mlx5: reduce PCI dependency") Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-08-02 14:55:24 +00:00
DRV_LOG(DEBUG, "dev_port-%u new domain_id=%u\n",
priv->dev_port, priv->domain_id);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->hw_padding && !sh->dev_cap.hw_padding) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(DEBUG, "Rx end alignment padding isn't supported");
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
config->hw_padding = 0;
} else if (config->hw_padding) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(DEBUG, "Rx end alignment padding is enabled");
}
/*
* MPW is disabled by default, while the Enhanced MPW is enabled
* by default.
*/
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
if (config->mps == MLX5_ARG_UNSET)
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
config->mps = (sh->dev_cap.mps == MLX5_MPW_ENHANCED) ?
MLX5_MPW_ENHANCED : MLX5_MPW_DISABLED;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
else
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
config->mps = config->mps ? sh->dev_cap.mps : MLX5_MPW_DISABLED;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(INFO, "%sMPS is %s",
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
config->mps == MLX5_MPW_ENHANCED ? "enhanced " :
config->mps == MLX5_MPW ? "legacy " : "",
config->mps != MLX5_MPW_DISABLED ? "enabled" : "disabled");
net/mlx5: remove DevX flag duplication The sharing device context structure has a field named "devx" which indicates if DevX is supported. The common configure structure has also field named "devx" with the same meaning. There is no need for this duplication, because there is a reference to the common structure from within the sharing device context structure. This patch removes it from sharing device context structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:59 +00:00
if (sh->cdev->config.devx) {
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
sh->steering_format_version = hca_attr->steering_format_version;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
/* LRO is supported only when DV flow enabled. */
if (sh->dev_cap.lro_supported && config->dv_flow_en)
sh->dev_cap.lro_supported = 0;
if (sh->dev_cap.lro_supported) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
* If LRO timeout is not configured by application,
* use the minimal supported value.
*/
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (!config->lro_timeout)
config->lro_timeout =
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
hca_attr->lro_timer_supported_periods[0];
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(DEBUG, "LRO session timeout set to %d usec",
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
config->lro_timeout);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: support ASO meter action When ASO action is available, use it as the meter action Signed-off-by: Shun Hao <shunh@nvidia.com> Signed-off-by: Li Zhang <lizh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-20 10:55:19 +00:00
#if defined(HAVE_MLX5DV_DR) && \
(defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_METER) || \
defined(HAVE_MLX5_DR_CREATE_ACTION_ASO))
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->qos.sup && hca_attr->qos.flow_meter_old &&
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
config->dv_flow_en) {
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
uint8_t reg_c_mask = hca_attr->qos.flow_meter_reg_c_ids;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
* Meter needs two REG_C's for color match and pre-sfx
* flow match. Here get the REG_C for color match.
* REG_C_0 and REG_C_1 is reserved for metadata feature.
*/
reg_c_mask &= 0xfc;
if (__builtin_popcount(reg_c_mask) < 1) {
priv->mtr_en = 0;
DRV_LOG(WARNING, "No available register for"
" meter.");
} else {
net/mlx5: fix input register for ASO object Existing code uses the hard-coded value REG_C_5 as input for function mlx5dv_dr_action_create_flow_hit(). This patch updates function mlx5_flow_get_reg_id() to return the selected REG_C value for ASO Flow Hit operation. The returned value is used, after reducing offset REG_C_0, as input for function mlx5dv_dr_action_create_flow_hit(). Fixes: f935ed4b645a ("net/mlx5: support flow hit action for aging") Signed-off-by: Dekel Peled <dekelp@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-11-18 08:59:56 +00:00
/*
* The meter color register is used by the
* flow-hit feature as well.
* The flow-hit feature must use REG_C_3
* Prefer REG_C_3 if it is available.
*/
if (reg_c_mask & (1 << (REG_C_3 - REG_C_0)))
priv->mtr_color_reg = REG_C_3;
else
priv->mtr_color_reg = ffs(reg_c_mask)
- 1 + REG_C_0;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
priv->mtr_en = 1;
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
priv->mtr_reg_share = hca_attr->qos.flow_meter;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(DEBUG, "The REG_C meter uses is %d",
priv->mtr_color_reg);
}
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
}
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->qos.sup && hca_attr->qos.flow_meter_aso_sup) {
net/mlx5: initialize flow meter ASO SQ Initialize the flow meter ASO SQ WQEs with all the constant data that should not be updated per enqueue operation. Signed-off-by: Li Zhang <lizh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-20 10:55:17 +00:00
uint32_t log_obj_size =
rte_log2_u32(MLX5_ASO_MTRS_PER_POOL >> 1);
if (log_obj_size >=
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
hca_attr->qos.log_meter_aso_granularity &&
log_obj_size <=
hca_attr->qos.log_meter_aso_max_alloc)
net/mlx5: initialize flow meter ASO SQ Initialize the flow meter ASO SQ WQEs with all the constant data that should not be updated per enqueue operation. Signed-off-by: Li Zhang <lizh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-20 10:55:17 +00:00
sh->meter_aso_en = 1;
net/mlx5: support meter creation with policy Create a meter with the new pre-defined policy. The following cases to be considered: 1.Add entry match with meter_id in global drop table. 2.For non-termination policy (policy id 0), add jump rule to suffix table for green and jump rule to drop table for red. 3.Allocate counter per meter in drop table. 4.Allocate meter resource per domain per color. 5.It can work with both ASO and legacy meter HW objects. Signed-off-by: Li Zhang <lizh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-27 10:43:52 +00:00
}
if (priv->mtr_en) {
err = mlx5_aso_flow_mtrs_mng_init(priv->sh);
if (err) {
err = -err;
goto error;
net/mlx5: initialize flow meter ASO SQ Initialize the flow meter ASO SQ WQEs with all the constant data that should not be updated per enqueue operation. Signed-off-by: Li Zhang <lizh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-04-20 10:55:17 +00:00
}
}
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->flow.tunnel_header_0_1)
net/mlx5: support matching on VXLAN reserved field This adds matching on the reserved field of VXLAN header (the last 8-bits). The capability from rdma-core is detected by creating a dummy matcher using misc5 when the device is probed. For non-zero groups and FDB domain, the capability is detected from rdma-core, meanwhile for NIC domain group zero it's relying on the HCA_CAP from FW. Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Raslan Darawsheh <rasland@nvidia.com>
2021-07-13 12:09:19 +00:00
sh->tunnel_header_0_1 = 1;
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
#endif
common/mlx5: move to formal ASO action API Existing code uses the previous API offered by rdma-core in order to create ASO Flow Hit action. A general API is now formally released, to create ASO action of any type. This patch moves the MLX5 PMD code to use the formal API. Signed-off-by: Dekel Peled <dekelp@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-11-18 08:59:57 +00:00
#ifdef HAVE_MLX5_DR_CREATE_ACTION_ASO
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->flow_hit_aso && priv->mtr_color_reg == REG_C_3) {
net/mlx5: fix input register for ASO object Existing code uses the hard-coded value REG_C_5 as input for function mlx5dv_dr_action_create_flow_hit(). This patch updates function mlx5_flow_get_reg_id() to return the selected REG_C value for ASO Flow Hit operation. The returned value is used, after reducing offset REG_C_0, as input for function mlx5dv_dr_action_create_flow_hit(). Fixes: f935ed4b645a ("net/mlx5: support flow hit action for aging") Signed-off-by: Dekel Peled <dekelp@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-11-18 08:59:56 +00:00
sh->flow_hit_aso_en = 1;
err = mlx5_flow_aso_age_mng_init(sh);
if (err) {
err = -err;
goto error;
}
DRV_LOG(DEBUG, "Flow Hit ASO is supported.");
}
common/mlx5: move to formal ASO action API Existing code uses the previous API offered by rdma-core in order to create ASO Flow Hit action. A general API is now formally released, to create ASO action of any type. This patch moves the MLX5 PMD code to use the formal API. Signed-off-by: Dekel Peled <dekelp@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-11-18 08:59:57 +00:00
#endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO */
net/mlx5: initialize connection tracking management The definitions of ASO connection tracking objects management structures are added. Considering performance, the bulk allocation of ASO CT objects should be used. The maximal value per bulk and the granularity could be fetched from HCA capabilities 2. Right now, a fixed number of 64 is used for each bulk for a better management purpose. The ASO QP for CT is initialized, the SQ will be used for both modify and query command. Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-05-05 12:23:15 +00:00
#if defined(HAVE_MLX5_DR_CREATE_ACTION_ASO) && \
defined(HAVE_MLX5_DR_ACTION_ASO_CT)
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->ct_offload && priv->mtr_color_reg == REG_C_3) {
net/mlx5: initialize connection tracking management The definitions of ASO connection tracking objects management structures are added. Considering performance, the bulk allocation of ASO CT objects should be used. The maximal value per bulk and the granularity could be fetched from HCA capabilities 2. Right now, a fixed number of 64 is used for each bulk for a better management purpose. The ASO QP for CT is initialized, the SQ will be used for both modify and query command. Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-05-05 12:23:15 +00:00
err = mlx5_flow_aso_ct_mng_init(sh);
if (err) {
err = -err;
goto error;
}
DRV_LOG(DEBUG, "CT ASO is supported.");
sh->ct_aso_en = 1;
}
#endif /* HAVE_MLX5_DR_CREATE_ACTION_ASO && HAVE_MLX5_DR_ACTION_ASO_CT */
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
#if defined(HAVE_MLX5DV_DR) && defined(HAVE_MLX5_DR_CREATE_ACTION_FLOW_SAMPLE)
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (hca_attr->log_max_ft_sampler_num > 0 &&
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
config->dv_flow_en) {
priv->sampler_en = 1;
common/mlx5: remove extra line feed in log messages The macro DRV_LOG already includes a terminating line feed character defined in PMD_DRV_LOG_. The extra line feeds added in some messages are removed. Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Matan Azrad <matan@nvidia.com>
2021-03-09 09:48:35 +00:00
DRV_LOG(DEBUG, "Sampler enabled!");
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
} else {
priv->sampler_en = 0;
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (!hca_attr->log_max_ft_sampler_num)
common/mlx5: remove extra line feed in log messages The macro DRV_LOG already includes a terminating line feed character defined in PMD_DRV_LOG_. The extra line feeds added in some messages are removed. Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Matan Azrad <matan@nvidia.com>
2021-03-09 09:48:35 +00:00
DRV_LOG(WARNING,
"No available register for sampler.");
net/mlx5: validate sample action Add sample action validate function. Sample Flow is supported in NIC-RX and FDB domains. For the NIC-RX the Sample Flow action list must include the destination queue action. Only NIC-RX domain supports the optional actions list. FDB doesn't support any optional actions, the sampled packets is always forwarded to the E-Switch manager port. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2020-10-13 14:11:45 +00:00
else
common/mlx5: remove extra line feed in log messages The macro DRV_LOG already includes a terminating line feed character defined in PMD_DRV_LOG_. The extra line feeds added in some messages are removed. Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Matan Azrad <matan@nvidia.com>
2021-03-09 09:48:35 +00:00
DRV_LOG(DEBUG, "DV flow is not supported!");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: introduce send scheduling devargs This patch introduces the new devargs: tx_pp - enables accurate packet send scheduling on mbuf timestamps in the PMD. On the device start if "rte_dynflag_timestamp" dynamic flag is registered and this devarg non-zero value is specified, the driver initializes all necessary internal infrastructure to provide packet scheduling. The parameter value specifies scheduling granularity in nanoseconds. tx_skew - the parameter adjusts the send packet scheduling on timestamps and represents the average delay between beginning of the transmitting descriptor processing by the hardware and appearance of actual packet data on the wire. The value should be provided in nanoseconds and is valid only if tx_pp parameter is specified. The default value is zero. Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-16 08:23:05 +00:00
#endif
}
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->cqe_comp && !sh->dev_cap.cqe_comp) {
DRV_LOG(WARNING, "Rx CQE 128B compression is not supported.");
net/mlx5: check FW miniCQE format capabilities miniCQE formats for Flow Tag and L3/L4 Header compression are only supported by Mellanox FW starting version 16.29.392. There is no point to allow user to enable these formats if FW cannot provide them. Check FW capabilities and deny user requests if the selected miniCQE format is not supported by an underlying NIC. Fixes: 54c2d46b160f ("net/mlx5: support flow tag and packet header miniCQEs") Cc: stable@dpdk.org Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-02 02:07:37 +00:00
config->cqe_comp = 0;
}
if (config->cqe_comp_fmt == MLX5_CQE_RESP_FORMAT_FTAG_STRIDX &&
net/mlx5: remove DevX flag duplication The sharing device context structure has a field named "devx" which indicates if DevX is supported. The common configure structure has also field named "devx" with the same meaning. There is no need for this duplication, because there is a reference to the common structure from within the sharing device context structure. This patch removes it from sharing device context structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:59 +00:00
(!sh->cdev->config.devx || !hca_attr->mini_cqe_resp_flow_tag)) {
net/mlx5: check FW miniCQE format capabilities miniCQE formats for Flow Tag and L3/L4 Header compression are only supported by Mellanox FW starting version 16.29.392. There is no point to allow user to enable these formats if FW cannot provide them. Check FW capabilities and deny user requests if the selected miniCQE format is not supported by an underlying NIC. Fixes: 54c2d46b160f ("net/mlx5: support flow tag and packet header miniCQEs") Cc: stable@dpdk.org Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-02 02:07:37 +00:00
DRV_LOG(WARNING, "Flow Tag CQE compression"
" format isn't supported.");
config->cqe_comp = 0;
}
if (config->cqe_comp_fmt == MLX5_CQE_RESP_FORMAT_L34H_STRIDX &&
net/mlx5: remove DevX flag duplication The sharing device context structure has a field named "devx" which indicates if DevX is supported. The common configure structure has also field named "devx" with the same meaning. There is no need for this duplication, because there is a reference to the common structure from within the sharing device context structure. This patch removes it from sharing device context structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:59 +00:00
(!sh->cdev->config.devx || !hca_attr->mini_cqe_resp_l3_l4_tag)) {
net/mlx5: check FW miniCQE format capabilities miniCQE formats for Flow Tag and L3/L4 Header compression are only supported by Mellanox FW starting version 16.29.392. There is no point to allow user to enable these formats if FW cannot provide them. Check FW capabilities and deny user requests if the selected miniCQE format is not supported by an underlying NIC. Fixes: 54c2d46b160f ("net/mlx5: support flow tag and packet header miniCQEs") Cc: stable@dpdk.org Signed-off-by: Alexander Kozyrev <akozyrev@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-02 02:07:37 +00:00
DRV_LOG(WARNING, "L3/L4 Header CQE compression"
" format isn't supported.");
config->cqe_comp = 0;
}
DRV_LOG(DEBUG, "Rx CQE compression is %ssupported",
config->cqe_comp ? "" : "not ");
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->tx_pp && !sh->dev_cap.txpp_en) {
DRV_LOG(ERR, "Packet pacing is not supported.");
net/mlx5: introduce send scheduling devargs This patch introduces the new devargs: tx_pp - enables accurate packet send scheduling on mbuf timestamps in the PMD. On the device start if "rte_dynflag_timestamp" dynamic flag is registered and this devarg non-zero value is specified, the driver initializes all necessary internal infrastructure to provide packet scheduling. The parameter value specifies scheduling granularity in nanoseconds. tx_skew - the parameter adjusts the send packet scheduling on timestamps and represents the average delay between beginning of the transmitting descriptor processing by the hardware and appearance of actual packet data on the wire. The value should be provided in nanoseconds and is valid only if tx_pp parameter is specified. The default value is zero. Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-16 08:23:05 +00:00
err = ENODEV;
goto error;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: support Rx queue delay drop For the Ethernet RQs, if there all receiving descriptors are exhausted, the packets being received will be dropped. This behavior prevents slow or malicious software entities at the host from affecting the network. While for hairpin cases, even if there is no software involved during the packet forwarding from Rx to Tx side, some hiccup in the hardware or back pressure from Tx side may still cause the descriptors to be exhausted. In certain scenarios it may be preferred to configure the device to avoid such packet drops, assuming the posting of descriptors will resume shortly. To support this, a new devarg "delay_drop" is introduced. By default, the delay drop is enabled for hairpin Rx queues and disabled for standard Rx queues. This value is used as a bit mask: - bit 0: enablement of standard Rx queue - bit 1: enablement of hairpin Rx queue And this attribute will be applied to all Rx queues of a device. The "rq_delay_drop" capability in the HCA_CAP is checked before creating any queue. If the hardware capabilities do not support this delay drop, all the Rx queues will still be created without this attribute, and the devarg setting will be ignored even if it is specified explicitly. A warning log is used to notify the application when this occurs. Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-05 15:30:38 +00:00
if (config->std_delay_drop || config->hp_delay_drop) {
net/mlx5: remove HCA attribute structure duplication The HCA attribute structure is field of net configure structure. It is also field of common configure structure. There is no need for this duplication, because there is a reference to the common structure from within the net structures. This patch removes it from net configure structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:58 +00:00
if (!hca_attr->rq_delay_drop) {
net/mlx5: support Rx queue delay drop For the Ethernet RQs, if there all receiving descriptors are exhausted, the packets being received will be dropped. This behavior prevents slow or malicious software entities at the host from affecting the network. While for hairpin cases, even if there is no software involved during the packet forwarding from Rx to Tx side, some hiccup in the hardware or back pressure from Tx side may still cause the descriptors to be exhausted. In certain scenarios it may be preferred to configure the device to avoid such packet drops, assuming the posting of descriptors will resume shortly. To support this, a new devarg "delay_drop" is introduced. By default, the delay drop is enabled for hairpin Rx queues and disabled for standard Rx queues. This value is used as a bit mask: - bit 0: enablement of standard Rx queue - bit 1: enablement of hairpin Rx queue And this attribute will be applied to all Rx queues of a device. The "rq_delay_drop" capability in the HCA_CAP is checked before creating any queue. If the hardware capabilities do not support this delay drop, all the Rx queues will still be created without this attribute, and the devarg setting will be ignored even if it is specified explicitly. A warning log is used to notify the application when this occurs. Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-05 15:30:38 +00:00
config->std_delay_drop = 0;
config->hp_delay_drop = 0;
DRV_LOG(WARNING,
"dev_port-%u: Rxq delay drop is not supported",
priv->dev_port);
}
}
net/mlx5: add option to configure FCS or decapsulation There are some limitations on some NICs (at least on ConnectX-6 Dx and BlueField 2) with supporting FCS (frame checksum) scattering for the tunnel decapsulated packets. For the case only one of the features can be supported in the same time, and the new devarg "decap_en" is introduced to provide the choice to the users. If FCS scattering feature is not supposed to be engaged by application, this new devarg should be specified as "decap_en=0", forcing the FCS feature enable and rejecting tunnel decap actions in the rte_flow engine. If FCS scatter is not needed and application supposes to use tunnel decapsulation in rte_flow, the devarg can be omitted or set to non-zero value (this is default settings). Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-15 13:10:21 +00:00
/*
* If HW has bug working with tunnel packet decapsulation and
* scatter FCS, and decapsulation is needed, clear the hw_fcs_strip
ethdev: add namespace Add 'RTE_ETH' namespace to all enums & macros in a backward compatible way. The macros for backward compatibility can be removed in next LTS. Also updated some struct names to have 'rte_eth' prefix. All internal components switched to using new names. Syntax fixed on lines that this patch touches. Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Acked-by: Tyler Retzlaff <roretzla@linux.microsoft.com> Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru> Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com> Acked-by: Jerin Jacob <jerinj@marvell.com> Acked-by: Wisam Jaddo <wisamm@nvidia.com> Acked-by: Rosen Xu <rosen.xu@intel.com> Acked-by: Chenbo Xia <chenbo.xia@intel.com> Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com> Acked-by: Somnath Kotur <somnath.kotur@broadcom.com>
2021-10-22 11:03:12 +00:00
* bit. Then RTE_ETH_RX_OFFLOAD_KEEP_CRC bit will not be set anymore.
net/mlx5: add option to configure FCS or decapsulation There are some limitations on some NICs (at least on ConnectX-6 Dx and BlueField 2) with supporting FCS (frame checksum) scattering for the tunnel decapsulated packets. For the case only one of the features can be supported in the same time, and the new devarg "decap_en" is introduced to provide the choice to the users. If FCS scattering feature is not supposed to be engaged by application, this new devarg should be specified as "decap_en=0", forcing the FCS feature enable and rejecting tunnel decap actions in the rte_flow engine. If FCS scatter is not needed and application supposes to use tunnel decapsulation in rte_flow, the devarg can be omitted or set to non-zero value (this is default settings). Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-15 13:10:21 +00:00
*/
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (sh->dev_cap.scatter_fcs_w_decap_disable && config->decap_en)
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
config->hw_fcs_strip = 0;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
else
config->hw_fcs_strip = sh->dev_cap.hw_fcs_strip;
net/mlx5: add option to configure FCS or decapsulation There are some limitations on some NICs (at least on ConnectX-6 Dx and BlueField 2) with supporting FCS (frame checksum) scattering for the tunnel decapsulated packets. For the case only one of the features can be supported in the same time, and the new devarg "decap_en" is introduced to provide the choice to the users. If FCS scattering feature is not supposed to be engaged by application, this new devarg should be specified as "decap_en=0", forcing the FCS feature enable and rejecting tunnel decap actions in the rte_flow engine. If FCS scatter is not needed and application supposes to use tunnel decapsulation in rte_flow, the devarg can be omitted or set to non-zero value (this is default settings). Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-15 13:10:21 +00:00
DRV_LOG(DEBUG, "FCS stripping configuration is %ssupported",
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
(config->hw_fcs_strip ? "" : "not "));
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (config->mprq.enabled && !sh->dev_cap.mprq.enabled) {
DRV_LOG(WARNING, "Multi-Packet RQ isn't supported.");
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
config->mprq.enabled = 0;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
if (config->max_dump_files_num == 0)
config->max_dump_files_num = 128;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
eth_dev = rte_eth_dev_allocate(name);
if (eth_dev == NULL) {
DRV_LOG(ERR, "can not allocate rte ethdev");
err = ENOMEM;
goto error;
}
if (priv->representor) {
eth_dev->data->dev_flags |= RTE_ETH_DEV_REPRESENTOR;
eth_dev->data->representor_id = priv->representor_id;
ethdev: fix representor port ID search by name The patch is required for all PMDs which do not provide representors info on the representor itself. The function, rte_eth_representor_id_get(), is used in eth_representor_cmp() which is required in ethdev class iterator to search ethdev port ID by name (representor case). Before the patch the function is called on the representor itself and tries to get representors info to match. Search of port ID by name is used after hotplug to find out port ID of the just plugged device. Getting a list of representors from a representor does not make sense. Instead, a backer device should be used. To this end, extend the rte_eth_dev_data structure to include the port ID of the backing device for representors. Signed-off-by: Viacheslav Galaktionov <viacheslav.galaktionov@oktetlabs.ru> Signed-off-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru> Acked-by: Haiyue Wang <haiyue.wang@intel.com> Acked-by: Beilei Xing <beilei.xing@intel.com> Reviewed-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-11 12:53:06 +00:00
MLX5_ETH_FOREACH_DEV(port_id, dpdk_dev) {
struct mlx5_priv *opriv =
rte_eth_devices[port_id].data->dev_private;
if (opriv &&
opriv->master &&
opriv->domain_id == priv->domain_id &&
opriv->sh == priv->sh) {
eth_dev->data->backer_port_id = port_id;
break;
}
}
if (port_id >= RTE_MAX_ETHPORTS)
eth_dev->data->backer_port_id = eth_dev->data->port_id;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: fix multi-process port ID The device port_id is used for inter-process communication and must be the same both for primary and secondary process This IPC port_id was configured with the invalid temporary value in port spawn routine. This temporary value was used by the function rte_eth_dev_get_port_by_name() to check whether the port exists. This commit corrects the mp port_id with rte_eth_dev port_id. Fixes: 2eb4d0107acc ("net/mlx5: refactor PCI probing on Linux") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-01-24 11:02:03 +00:00
priv->mp_id.port_id = eth_dev->data->port_id;
strlcpy(priv->mp_id.name, MLX5_MP_NAME, RTE_MP_MAX_NAME_LEN);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
* Store associated network device interface index. This index
* is permanent throughout the lifetime of device. So, we may store
* the ifindex here and use the cached value further.
*/
MLX5_ASSERT(spawn->ifindex);
priv->if_index = spawn->ifindex;
net/mlx5: set Tx queue affinity in round-robin Previously, we set txq affinity to 0 and let firmware to perform round-robin when bonding. Firmware uses a global counter to assign txq affinity to different physical ports accord to remainder after division. There are three dis-advantages: 1. The global counter is shared between kernel and dpdk. 2. After restarting pmd or port, the previous counter value is reused, so the new affinity is unpredictable. 3. There is no way to get what affinity is set by firmware. In this update, we will create several TISs up to the number of bonding ports and bind each TIS to one PF port. For each port, it will start to pick up TIS using its port index. Upper layer application can quickly calculate each txq's affinity without querying. At DPDK layer, when creating txq with 2 bonding ports, the affinity is set like: port 0: 1-->2-->1-->2 port 1: 2-->1-->2-->1 port 2: 1-->2-->1-->2 Note: Only applicable to DevX api. This affinity subjects to HW hash. Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-21 08:56:36 +00:00
priv->lag_affinity_idx = sh->refcnt - 1;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
eth_dev->data->dev_private = priv;
priv->dev_data = eth_dev->data;
eth_dev->data->mac_addrs = priv->mac;
net/mlx5: revert setting bonding representor to first PF With kernel bonding, representors on second PF are being probed by devargs: <primary_bdf>,representor=pf1vf<N> No need to save primary PF port ID and lookup when probing sibling ports, revert patch [1] [1]: commit e6818853c022 ("net/mlx5: set representor to first PF in bonding mode") Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:09 +00:00
eth_dev->device = dpdk_dev;
ethdev: add device flag to bypass auto-filled queue xstats Queue stats are stored in 'struct rte_eth_stats' as array and array size is defined by 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag. As a result of technical board discussion, decided to remove the queue statistics from 'struct rte_eth_stats' in the long term. Instead PMDs should represent the queue statistics via xstats, this gives more flexibility on the number of the queues supported. Currently queue stats in the xstats are filled by ethdev layer, using some basic stats, when queue stats removed from basic stats the responsibility to fill the relevant xstats will be pushed to the PMDs. During the switch period, temporary 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' device flag is created. Initially all PMDs using xstats set this flag. The PMDs implemented queue stats in the xstats should clear the flag. When all PMDs switch to the xstats for the queue stats, queue stats related fields from 'struct rte_eth_stats' will be removed, as well as 'RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS' flag. Later 'RTE_ETHDEV_QUEUE_STAT_CNTRS' compile time flag also can be removed. Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Acked-by: Haiyue Wang <haiyue.wang@intel.com> Acked-by: Xiao Wang <xiao.w.wang@intel.com> Acked-by: Thomas Monjalon <thomas@monjalon.net>
2020-10-14 02:26:47 +00:00
eth_dev->data->dev_flags |= RTE_ETH_DEV_AUTOFILL_QUEUE_XSTATS;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Configure the first MAC address by default. */
if (mlx5_get_mac(eth_dev, &mac.addr_bytes)) {
DRV_LOG(ERR,
"port %u cannot get MAC address, is mlx5_en"
" loaded? (errno: %s)",
eth_dev->data->port_id, strerror(rte_errno));
err = ENODEV;
goto error;
}
DRV_LOG(INFO,
net: add macro for MAC address print Added macro to print six bytes of MAC address. The MAC addresses will be printed in upper case hexadecimal format. In case there is a specific check for lower case MAC address, the user may need to make a change in such test case after this patch. Signed-off-by: Aman Deep Singh <aman.deep.singh@intel.com> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-08-25 17:27:33 +00:00
"port %u MAC address is " RTE_ETHER_ADDR_PRT_FMT,
net: add macro to extract MAC address bytes Added macros to simplify print of MAC address. The six bytes of a MAC address are extracted in a macro here, to improve code readablity. Signed-off-by: Aman Deep Singh <aman.deep.singh@intel.com> Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2021-08-25 17:27:34 +00:00
eth_dev->data->port_id, RTE_ETHER_ADDR_BYTES(&mac));
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifdef RTE_LIBRTE_MLX5_DEBUG
{
net/mlx5: fix device name size on Windows Windows Devx interface name is the same as device name with different size then IF_NAMESIZE. To support it MLX5_NAMESIZE is defined with IF_NAMESIZE value for Linux and MLX5_FS_NAME_MAX value for Windows. Fixes: e9c0b96e3526 ("net/mlx5: move Linux ifname function") Cc: stable@dpdk.org Signed-off-by: Tal Shnaiderman <talshn@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-12-28 12:32:57 +00:00
char ifname[MLX5_NAMESIZE];
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (mlx5_get_ifname(eth_dev, &ifname) == 0)
DRV_LOG(DEBUG, "port %u ifname is \"%s\"",
eth_dev->data->port_id, ifname);
else
DRV_LOG(DEBUG, "port %u ifname is unknown",
eth_dev->data->port_id);
}
#endif
/* Get actual MTU if possible. */
err = mlx5_get_mtu(eth_dev, &priv->mtu);
if (err) {
err = rte_errno;
goto error;
}
DRV_LOG(DEBUG, "port %u MTU is %u", eth_dev->data->port_id,
priv->mtu);
/* Initialize burst functions to prevent crashes before link-up. */
ethdev: introduce generic dummy packet burst function Multiple PMDs have dummy/noop Rx/Tx packet burst functions. These dummy functions are very simple, introduce a common function in the ethdev and update drivers to use it instead of each driver having its own functions. Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Acked-by: Morten Brørup <mb@smartsharesystems.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Thomas Monjalon <thomas@monjalon.net>
2022-02-11 19:11:42 +00:00
eth_dev->rx_pkt_burst = rte_eth_pkt_burst_dummy;
eth_dev->tx_pkt_burst = rte_eth_pkt_burst_dummy;
net/mlx5: unify operations for all OS There are three types of eth_dev_ops: primary, secondary and isolate represented in three callback tables per OS. In this commit the OS specific eth dev tables are unified into shared tables in file mlx5.c. Starting from this commit all operating systems must implement the same eth dev APIs. In case an OS does not support an API - it can return in its implementation an error ENOTSUP. Fixes: 042f5c94fd3a ("net/mlx5: refactor device operations for Linux") Cc: stable@dpdk.org Signed-off-by: Ophir Munk <ophirmu@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-12-28 12:32:56 +00:00
eth_dev->dev_ops = &mlx5_dev_ops;
ethdev: move inline device operations This patch is a preparation to hide the 'struct eth_dev_ops' from applications by moving some device operations from 'struct eth_dev_ops' to 'struct rte_eth_dev'. Mentioned ethdev APIs are in the data path and implemented as inline because of performance reasons. Exposing 'struct eth_dev_ops' to applications is bad because it is a contract between ethdev and PMDs, not really needs to be known by applications, also changes in the struct causing ABI breakages which shouldn't. To be able to both keep APIs inline and hide the 'struct eth_dev_ops', moving device operations used in ethdev inline APIs to 'struct rte_eth_dev' to the same level with Rx/Tx burst functions. The list of dev_ops moved: eth_rx_queue_count_t rx_queue_count; eth_rx_descriptor_done_t rx_descriptor_done; eth_rx_descriptor_status_t rx_descriptor_status; eth_tx_descriptor_status_t tx_descriptor_status; Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com> Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com> Acked-by: David Marchand <david.marchand@redhat.com> Acked-by: Sachin Saxena <sachin.saxena@nxp.com>
2020-09-09 13:01:43 +00:00
eth_dev->rx_descriptor_status = mlx5_rx_descriptor_status;
eth_dev->tx_descriptor_status = mlx5_tx_descriptor_status;
eth_dev->rx_queue_count = mlx5_rx_queue_count;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Register MAC address. */
claim_zero(mlx5_mac_addr_add(eth_dev, &mac, 0, 0));
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (sh->dev_cap.vf && config->vf_nl_en)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_nl_mac_addr_sync(priv->nl_socket_route,
mlx5_ifindex(eth_dev),
eth_dev->data->mac_addrs,
MLX5_MAX_MAC_ADDRESSES);
priv->ctrl_flows = 0;
net/mlx5: make flow list thread safe To support multi-thread flow operations, this patch introduces list lock for the rte_flow list manages all the rte_flow handlers. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:25 +00:00
rte_spinlock_init(&priv->flow_list_lock);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
TAILQ_INIT(&priv->flow_meters);
net/mlx5: optimize meter profile lookup Currently a list is used to save all meter profile ids, which is not efficient when looking up profile from huge amount of profiles. This changes to use an l3 table instead to save meter profile ids, so as to improve the lookup performance. Signed-off-by: Shun Hao <shunh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-02 09:14:43 +00:00
priv->mtr_profile_tbl = mlx5_l3t_create(MLX5_L3T_TYPE_PTR);
if (!priv->mtr_profile_tbl)
goto error;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Bring Ethernet device up. */
DRV_LOG(DEBUG, "port %u forcing Ethernet interface up",
eth_dev->data->port_id);
mlx5_set_link_up(eth_dev);
/*
* Even though the interrupt handler is not installed yet,
* interrupts will still trigger on the async_fd from
* Verbs context returned by ibv_open_device().
*/
mlx5_link_update(eth_dev, 0);
/* Detect minimal data bytes to inline. */
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
mlx5_set_min_inline(spawn, config);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Store device configuration on private structure. */
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
priv->config = *config;
net/mlx5: replace flow list with indexed pool The flow list is used to save the create flows and to be used only when port closes all the flows need to be flushed. This commit takes advantage of the index pool foreach operation to flush all the allocated flows. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:39 +00:00
for (i = 0; i < MLX5_FLOW_TYPE_MAXI; i++) {
icfg[i].release_mem_en = !!config->reclaim_mode;
if (config->reclaim_mode)
icfg[i].per_core_cache = 0;
priv->flows[i] = mlx5_ipool_create(&icfg[i]);
if (!priv->flows[i])
goto error;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Create context for virtual machine VLAN workaround. */
priv->vmwa_context = mlx5_vlan_vmwa_init(eth_dev, spawn->ifindex);
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
if (config->dv_flow_en) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
err = mlx5_alloc_shared_dr(priv);
if (err)
goto error;
net/mlx5: add flex item operations This patch is a preparation step of implementing flex item feature in driver and it provides: - external entry point routines for flex item creation/deletion - flex item objects management over the ports. The flex item object keeps information about the item created over the port - reference counter to track whether item is in use by some active flows and the pointer to underlying shared DevX object, providing all the data needed to translate the flow flex pattern into matcher fields according hardware configuration. There is not too many flex items supposed to be created on the port, the design is optimized rather for flow insertion rate than memory savings. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-02 08:53:42 +00:00
if (mlx5_flex_item_port_init(eth_dev) < 0)
goto error;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
if (sh->cdev->config.devx && config->dv_flow_en &&
sh->dev_cap.dest_tir) {
net/mlx5: separate Rx queue drop Separate Rx queue drop creation into both Verbs and DevX modules. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-09-03 10:13:48 +00:00
priv->obj_ops = devx_obj_ops;
net/mlx5: fix imissed statistics The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-25 10:45:01 +00:00
mlx5_queue_counter_id_prepare(eth_dev);
net/mlx5: fix loopback for Direct Verbs queue In the past, all the queues and other hardware objects were created through Verbs interface. Currently, most of the objects creation are migrated to Devx interface by default, including queues. Only when the DV is disabled by device arg or eswitch is enabled, all or some of the objects are created through Verbs interface. When using Devx interface to create queues, the kernel driver behavior is different from the case using Verbs. The Tx loopback cannot work properly even if the Tx and Rx queues are configured with loopback attribute. To fix the support self loopback for Tx, a Verbs dummy queue pair needs to be created to trigger the kernel to enable the global loopback capability. This is only required when TIR is created for Rx and loopback is needed. Only CQ and QP are needed for this case, no WQ(RQ) needs to be created. Bugzilla ID: 645 Fixes: 6deb19e1b2d2 ("net/mlx5: separate Rx queue object creations") Cc: stable@dpdk.org Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-05-17 15:18:41 +00:00
priv->obj_ops.lb_dummy_queue_create =
mlx5_rxq_ibv_obj_dummy_lb_create;
priv->obj_ops.lb_dummy_queue_release =
mlx5_rxq_ibv_obj_dummy_lb_release;
net/mlx5: check DevX to support more Verbs ports Verbs API doesn't support device port number larger than 255 by design. To support more VF or SubFunction port representors, forces DevX API check when max Verbs device link ports larger than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:35:01 +00:00
} else if (spawn->max_port > UINT8_MAX) {
/* Verbs can't support ports larger than 255 by design. */
DRV_LOG(ERR, "must enable DV and ESW when RDMA link ports > 255");
err = ENOTSUP;
goto error;
net/mlx5: separate Rx queue drop Separate Rx queue drop creation into both Verbs and DevX modules. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-09-03 10:13:48 +00:00
} else {
priv->obj_ops = ibv_obj_ops;
}
net/mlx5: add Tx scheduling check on queue creation The send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX, the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there is no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction. If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via tx_pp devarg the device probing is rejected as device configuration can't satisfy the requirements. Fixes: 3ec73abeed52 ("net/mlx5/linux: fix Tx queue operations decision") Fixes: 8f848f32fc24 ("net/mlx5: introduce send scheduling devargs") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-29 12:26:43 +00:00
if (config->tx_pp &&
net/mlx5: fix Tx scheduling check There was a redundant check for the enabled E-Switch, this resulted in device probing failure if the Tx scheduling was requested and E-Switch was enabled. Fixes: f17e4b4ffef9 ("net/mlx5: add Tx scheduling check on queue creation") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-10 13:05:52 +00:00
priv->obj_ops.txq_obj_new != mlx5_txq_devx_obj_new) {
net/mlx5: add Tx scheduling check on queue creation The send scheduling on timestamp offload requires the Send Queue (SQ) shares its User Access Region (UAR) with the pacing Clock Queue. The SQ can be created by mlx5 PMD either with DevX or with Verbs. If the SQ is being created with DevX, the dedicated UAR can be specified and all the SQs share the single UAR. Once SQ is being created with Verbs the SQ's UAR is allocated by the rdma-core library internally on its own and there is no UAR sharing. This caused hardware errors on WAIT WQEs and overall send scheduling malfunction. If SQs are going to be created with Verbs and the send scheduling offload is explicitly requested via tx_pp devarg the device probing is rejected as device configuration can't satisfy the requirements. Fixes: 3ec73abeed52 ("net/mlx5/linux: fix Tx queue operations decision") Fixes: 8f848f32fc24 ("net/mlx5: introduce send scheduling devargs") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-29 12:26:43 +00:00
/*
* HAVE_MLX5DV_DEVX_UAR_OFFSET is required to support
* packet pacing and already checked above.
* Hence, we should only make sure the SQs will be created
* with DevX, not with Verbs.
* Verbs allocates the SQ UAR on its own and it can't be shared
* with Clock Queue UAR as required for Tx scheduling.
*/
DRV_LOG(ERR, "Verbs SQs, UAR can't be shared as required for packet pacing");
err = ENODEV;
goto error;
}
net/mlx5: create global drop action This commit creates the global drop action for flows instead of maintain it in flow insertion time. The uniqueu global drop action makes it thread safe. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:30 +00:00
priv->drop_queue.hrxq = mlx5_drop_action_create(eth_dev);
if (!priv->drop_queue.hrxq)
goto error;
fix spelling in comments and strings The tool comes from https://github.com/jsoref Signed-off-by: Josh Soref <jsoref@gmail.com> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-11-29 16:08:02 +00:00
/* Port representor shares the same max priority with pf port. */
net/mlx5: optimize device spawn time with representors During the device spawn process, mlx5 PMD queried the available flow priorities by calling mlx5_flow_discover_priorities, queried if the DR drop action was supported on the root table by calling the mlx5_flow_discover_dr_action_support routine, and queried the availability of metadata register C by calling mlx5_flow_discover_mreg_c These functions created the test flows to get the supported fields, and at the end destroyed the test flows. The test flows in the first two functions was created on the root table. If the device was spawned with multiple representors, these test flows were created and destroyed on each representor as well. The above operations took a significant amount of init time during the device spawn. This patch optimizes the device discover functions, if there is the device with multiple representors (VF/SF) being spawned, the priority and drop action and metadata register support check can be done only ones and check results can be shared for all representors. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-27 10:35:10 +00:00
if (!priv->sh->flow_priority_check_flag) {
/* Supported Verbs flow priority number detection. */
err = mlx5_flow_discover_priorities(eth_dev);
priv->sh->flow_max_priority = err;
priv->sh->flow_priority_check_flag = 1;
} else {
err = priv->sh->flow_max_priority;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (err < 0) {
err = -err;
goto error;
}
mlx5_set_metadata_mask(eth_dev);
if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY &&
!priv->sh->dv_regc0_mask) {
DRV_LOG(ERR, "metadata mode %u is not supported "
"(no metadata reg_c[0] is available)",
priv->config.dv_xmeta_en);
err = ENOTSUP;
goto error;
}
common/mlx5: add per-lcore sharing flag in object list Without lcores_share flag, mlx5 PMD was sharing the rdma-core objects between all lcores. Having lcores_share flag disabled, means each lcore will have its own objects, which will eventually lead to increased insertion/deletion rates. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-13 08:44:48 +00:00
priv->hrxqs = mlx5_list_create("hrxq", eth_dev, true,
mlx5_hrxq_create_cb,
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
mlx5_hrxq_match_cb,
mlx5_hrxq_remove_cb,
mlx5_hrxq_clone_cb,
mlx5_hrxq_clone_free_cb);
if (!priv->hrxqs)
goto error;
net/mlx5: add per-lcore cache to the list utility When mlx5 list object is accessed by multiple cores, the list lock counter is all the time written by all the cores what increases cache misses in the memory caches. In addition, when one thread accesses the list for add\remove\lookup operation, all the other threads coming to do an operation in the list are stuck in the lock. Add per lcore cache to allow thread manipulations to be lockless when the list objects are mostly reused. Synchronization with atomic operations should be done in order to allow threads to unregister an entry from other thread cache. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:42 +00:00
rte_rwlock_init(&priv->ind_tbls_lock);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Query availability of metadata reg_c's. */
net/mlx5: optimize device spawn time with representors During the device spawn process, mlx5 PMD queried the available flow priorities by calling mlx5_flow_discover_priorities, queried if the DR drop action was supported on the root table by calling the mlx5_flow_discover_dr_action_support routine, and queried the availability of metadata register C by calling mlx5_flow_discover_mreg_c These functions created the test flows to get the supported fields, and at the end destroyed the test flows. The test flows in the first two functions was created on the root table. If the device was spawned with multiple representors, these test flows were created and destroyed on each representor as well. The above operations took a significant amount of init time during the device spawn. This patch optimizes the device discover functions, if there is the device with multiple representors (VF/SF) being spawned, the priority and drop action and metadata register support check can be done only ones and check results can be shared for all representors. Signed-off-by: Jiawei Wang <jiaweiw@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-27 10:35:10 +00:00
if (!priv->sh->metadata_regc_check_flag) {
err = mlx5_flow_discover_mreg_c(eth_dev);
if (err < 0) {
err = -err;
goto error;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
if (!mlx5_flow_ext_mreg_supported(eth_dev)) {
DRV_LOG(DEBUG,
"port %u extensive metadata register is not supported",
eth_dev->data->port_id);
if (priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY) {
DRV_LOG(ERR, "metadata mode %u is not supported "
"(no metadata registers available)",
priv->config.dv_xmeta_en);
err = ENOTSUP;
goto error;
}
}
if (priv->config.dv_flow_en &&
priv->config.dv_xmeta_en != MLX5_XMETA_MODE_LEGACY &&
mlx5_flow_ext_mreg_supported(eth_dev) &&
priv->sh->dv_regc0_mask) {
priv->mreg_cp_tbl = mlx5_hlist_create(MLX5_FLOW_MREG_HNAME,
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
MLX5_FLOW_MREG_HTABLE_SZ,
common/mlx5: add per-lcore cache to hash list utility Using the mlx5 list utility object in the hlist buckets. This patch moves the list utility object to the common utility, creates all the clone operations for all the hlist instances in the driver. Also adjust all the utility callbacks to be generic for both list and hlist. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:50 +00:00
false, true, eth_dev,
net/mlx5: make metadata copy flow list thread safe To support multi-thread flow insertion, this patch updates metadata copy flow list to use thread safe hash list. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:38 +00:00
flow_dv_mreg_create_cb,
net/mlx5: optimize hash list entry memory Currently, the hash list saves the hash key in the hash entry. And the key is mostly used to get the bucket index only. Save the entire 64 bits key to the entry will not be a good option if the key is only used to get the bucket index. Since 64 bits costs more memory for the entry, mostly the signature data in the key only uses 32 bits. And in the unregister function, the key in the entry causes extra bucket index calculation. This commit saves the bucket index to the entry instead of the hash key. For the hash list like table, tag and mreg_copy which save the signature data in the key, the signature data is moved to the resource data struct itself. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-12-03 02:18:52 +00:00
flow_dv_mreg_match_cb,
common/mlx5: add per-lcore cache to hash list utility Using the mlx5 list utility object in the hlist buckets. This patch moves the list utility object to the common utility, creates all the clone operations for all the hlist instances in the driver. Also adjust all the utility callbacks to be generic for both list and hlist. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:50 +00:00
flow_dv_mreg_remove_cb,
flow_dv_mreg_clone_cb,
flow_dv_mreg_clone_free_cb);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!priv->mreg_cp_tbl) {
err = ENOMEM;
goto error;
}
}
net/mlx5: make shared action list thread safe This commit uses spinlock to protect the shared action list in multiple thread. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:52 +00:00
rte_spinlock_init(&priv->shared_act_sl);
net/mlx5: remove single counter container A flow counter which was allocated by a batch API couldn't be assigned to a flow in the root table (group 0) in old rdma-core version. Hence, a root table flow counter required PMD mechanism to manage counters which were allocated singly. Currently, the batch counters have already been supported in root table includes a new rdma-core version with MLX5_FLOW_ACTION_COUNTER_OFFSET enum and with a kernel driver includes MLX5_IB_ATTR_CREATE_FLOW_ARR_COUNTERS_DEVX_OFFSET enum. When the PMD uses rdma-core API to assign a batch counter to a root table flow using invalid counter offset, it should get an error only if the batch counter assignment for root table is supported. Using this trial in the initialization time can help to detect the support. Using the above trial, if the support is valid, remove the management of single counter container in the fast counter mechanism. Otherwise, move the counter mechanism to fallback mode. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-20 03:02:23 +00:00
mlx5_flow_counter_mode_config(eth_dev);
net/mlx5: workaround drop action with old kernel Currently, there are two types of drop action implementation in the PMD. One is the DR (Direct Rules) dummy placeholder drop action and another is the dedicated dummy queue drop action. When creates flow on the root table with DR drop action, the action will be converted to MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute in rdma-core. In some inbox systems, MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs attribute may not be supported in the kernel driver. Create flow with drop action on the root table will be failed as it is not supported. In this case, the dummy queue drop action should be used instead of DR dummy placeholder drop action. This commit adds the DR drop action support detect on the root table. If MLX5_IB_ATTR_CREATE_FLOW_FLAGS_DROP Verbs is not supported in the system, a dummy queue will be used as drop action. Fixes: da845ae9d7c1 ("net/mlx5: fix drop action for Direct Rules/Verbs") Cc: stable@dpdk.org Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-08-02 14:30:24 +00:00
mlx5_flow_drop_action_config(eth_dev);
net/mlx5: remove shared context lock To support multi-thread flow insertion, this patch removes shared data lock since all resources should support concurrent protection. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:53 +00:00
if (priv->config.dv_flow_en)
eth_dev->data->dev_flags |= RTE_ETH_DEV_FLOW_OPS_THREAD_SAFE;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return eth_dev;
error:
if (priv) {
if (priv->mreg_cp_tbl)
net/mlx5: support concurrent access for hash list In order to support hash list concurrent access, adding next: 1. List level read/write lock. 2. Entry reference counter. 3. Entry create/match/remove callback. 4. Remove insert/lookup/remove function which are not thread safe. 5. Add register/unregister function to support entry reuse. For better performance, lookup function uses read lock to allow concurrent lookup from different thread, all other hash list modification functions uses write lock which blocks concurrent modification and lookups from other thread. The exact objects change will be applied in the next patches. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:31 +00:00
mlx5_hlist_destroy(priv->mreg_cp_tbl);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (priv->sh)
mlx5_os_free_shared_dr(priv);
if (priv->nl_socket_route >= 0)
close(priv->nl_socket_route);
if (priv->vmwa_context)
mlx5_vlan_vmwa_exit(priv->vmwa_context);
net/mlx5: create global drop action This commit creates the global drop action for flows instead of maintain it in flow insertion time. The uniqueu global drop action makes it thread safe. Signed-off-by: Suanming Mou <suanmingm@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-28 09:33:30 +00:00
if (eth_dev && priv->drop_queue.hrxq)
mlx5_drop_action_destroy(eth_dev);
net/mlx5: optimize meter profile lookup Currently a list is used to save all meter profile ids, which is not efficient when looking up profile from huge amount of profiles. This changes to use an l3 table instead to save meter profile ids, so as to improve the lookup performance. Signed-off-by: Shun Hao <shunh@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-02 09:14:43 +00:00
if (priv->mtr_profile_tbl)
mlx5_l3t_destroy(priv->mtr_profile_tbl);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (own_domain_id)
claim_zero(rte_eth_switch_domain_free(priv->domain_id));
net/mlx5: allocate list memory in create function Currently, the list memory was allocated by the list API caller. Move it to be allocated by the create API in order to save consistence with the hlist utility. Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Suanming Mou <suanmingm@nvidia.com>
2021-07-13 08:44:46 +00:00
if (priv->hrxqs)
mlx5_list_destroy(priv->hrxqs);
net/mlx5: add flex item operations This patch is a preparation step of implementing flex item feature in driver and it provides: - external entry point routines for flex item creation/deletion - flex item objects management over the ports. The flex item object keeps information about the item created over the port - reference counter to track whether item is in use by some active flows and the pointer to underlying shared DevX object, providing all the data needed to translate the flow flex pattern into matcher fields according hardware configuration. There is not too many flex items supposed to be created on the port, the design is optimized rather for flow insertion rate than memory savings. Signed-off-by: Gregory Etelson <getelson@nvidia.com> Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-02 08:53:42 +00:00
if (eth_dev && priv->flex_item_map)
mlx5_flex_item_port_cleanup(eth_dev);
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
mlx5_free(priv);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (eth_dev != NULL)
eth_dev->data->dev_private = NULL;
}
if (eth_dev != NULL) {
/* mac_addrs must not be freed alone because part of
* dev_private
**/
eth_dev->data->mac_addrs = NULL;
rte_eth_dev_release_port(eth_dev);
}
if (sh)
net/mlx5: rename ib in names Renames in this commit: mlx5_ibv_list -> mlx5_dev_ctx_list mlx5_alloc_shared_ibctx -> mlx5_alloc_shared_dev_ctx mlx5_free_shared_ibctx -> mlx5_free_shared_dev_ctx mlx5_ibv_shared_port -> mlx5_dev_shared_port ibv_port -> dev_port Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:27 +00:00
mlx5_free_shared_dev_ctx(sh);
net/mlx5: use Netlink when IB port greater than 255 IB spec doesn't allow 255 ports on a single HCA, port number of 256 was cast to u8 value 0 which invalid to ibv_query_port() This patch invokes Netlink API to query port state when port number greater than 255. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-19 10:34:55 +00:00
if (nl_rdma >= 0)
close(nl_rdma);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
MLX5_ASSERT(err > 0);
rte_errno = err;
return NULL;
}
/**
* Comparison callback to sort device data.
*
* This is meant to be used with qsort().
*
* @param a[in]
* Pointer to pointer to first data object.
* @param b[in]
* Pointer to pointer to second data object.
*
* @return
* 0 if both objects are equal, less than 0 if the first argument is less
* than the second, greater than 0 otherwise.
*/
static int
mlx5_dev_spawn_data_cmp(const void *a, const void *b)
{
const struct mlx5_switch_info *si_a =
&((const struct mlx5_dev_spawn_data *)a)->info;
const struct mlx5_switch_info *si_b =
&((const struct mlx5_dev_spawn_data *)b)->info;
int ret;
/* Master device first. */
ret = si_b->master - si_a->master;
if (ret)
return ret;
/* Then representor devices. */
ret = si_b->representor - si_a->representor;
if (ret)
return ret;
/* Unidentified devices come last in no specific order. */
if (!si_a->representor)
return 0;
/* Order representors by name. */
return si_a->port_name - si_b->port_name;
}
/**
* Match PCI information for possible slaves of bonding device.
*
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
* @param[in] ibdev_name
* Name of Infiniband device.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
* @param[in] pci_dev
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
* Pointer to primary PCI address structure to match.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
* @param[in] nl_rdma
* Netlink RDMA group socket handle.
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
* @param[in] owner
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
* Representor owner PF index.
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
* @param[out] bond_info
* Pointer to bonding information.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*
* @return
* negative value if no bonding device found, otherwise
* positive index of slave PF in bonding.
*/
static int
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
mlx5_device_bond_pci_match(const char *ibdev_name,
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
const struct rte_pci_addr *pci_dev,
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
int nl_rdma, uint16_t owner,
struct mlx5_bond_info *bond_info)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
{
char ifname[IF_NAMESIZE + 1];
unsigned int ifindex;
unsigned int np, i;
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
FILE *bond_file = NULL, *file;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
int pf = -1;
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
int ret;
net/mlx5: support socket direct mode bonding In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-26 08:48:30 +00:00
uint8_t cur_guid[32] = {0};
uint8_t guid[32] = {0};
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
* Try to get master device name. If something goes wrong suppose
* the lack of kernel support and no bonding devices.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*/
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
memset(bond_info, 0, sizeof(*bond_info));
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (nl_rdma < 0)
return -1;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
if (!strstr(ibdev_name, "bond"))
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return -1;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
np = mlx5_nl_portnum(nl_rdma, ibdev_name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!np)
return -1;
net/mlx5: support socket direct mode bonding In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-26 08:48:30 +00:00
if (mlx5_get_device_guid(pci_dev, cur_guid, sizeof(cur_guid)) < 0)
return -1;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/*
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
* The master device might not be on the predefined port(not on port
* index 1, it is not guaranteed), we have to scan all Infiniband
* device ports and find master.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*/
for (i = 1; i <= np; ++i) {
/* Check whether Infiniband port is populated. */
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
ifindex = mlx5_nl_ifindex(nl_rdma, ibdev_name, i);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!ifindex)
continue;
if (!if_indextoname(ifindex, ifname))
continue;
/* Try to read bonding slave names from sysfs. */
MKSTR(slaves,
"/sys/class/net/%s/master/bonding/slaves", ifname);
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
bond_file = fopen(slaves, "r");
if (bond_file)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
break;
}
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
if (!bond_file)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return -1;
/* Use safe format to check maximal buffer length. */
MLX5_ASSERT(atol(RTE_STR(IF_NAMESIZE)) == IF_NAMESIZE);
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
while (fscanf(bond_file, "%" RTE_STR(IF_NAMESIZE) "s", ifname) == 1) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
char tmp_str[IF_NAMESIZE + 32];
struct rte_pci_addr pci_addr;
struct mlx5_switch_info info;
net/mlx5: support socket direct mode bonding In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-26 08:48:30 +00:00
int ret;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Process slave interface names in the loop. */
snprintf(tmp_str, sizeof(tmp_str),
"/sys/class/net/%s", ifname);
common/mlx5: get PCI device address from any bus A function is exported to allow retrieving the PCI address of the parent PCI device of a Sub-Function in auxiliary bus sysfs. The function mlx5_dev_to_pci_str() is accepting both PCI and auxiliary devices. In case of a PCI device, it is simply using the device name. The function mlx5_dev_to_pci_addr(), which is based on sysfs path and do not use any device object, is renamed to mlx5_get_pci_addr() for clarity purpose. Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:32 +00:00
if (mlx5_get_pci_addr(tmp_str, &pci_addr)) {
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
DRV_LOG(WARNING,
"Cannot get PCI address for netdev \"%s\".",
ifname);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
continue;
}
/* Slave interface PCI address match found. */
snprintf(tmp_str, sizeof(tmp_str),
"/sys/class/net/%s/phys_port_name", ifname);
file = fopen(tmp_str, "rb");
if (!file)
break;
info.name_type = MLX5_PHYS_PORT_NAME_TYPE_NOTSET;
if (fscanf(file, "%32s", tmp_str) == 1)
mlx5_translate_port_name(tmp_str, &info);
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
fclose(file);
/* Only process PF ports. */
if (info.name_type != MLX5_PHYS_PORT_NAME_TYPE_LEGACY &&
info.name_type != MLX5_PHYS_PORT_NAME_TYPE_UPLINK)
continue;
/* Check max bonding member. */
if (info.port_name >= MLX5_BOND_MAX_PORTS) {
DRV_LOG(WARNING, "bonding index out of range, "
"please increase MLX5_BOND_MAX_PORTS: %s",
tmp_str);
break;
}
/* Get ifindex. */
snprintf(tmp_str, sizeof(tmp_str),
"/sys/class/net/%s/ifindex", ifname);
file = fopen(tmp_str, "rb");
if (!file)
break;
ret = fscanf(file, "%u", &ifindex);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
fclose(file);
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
if (ret != 1)
break;
/* Save bonding info. */
strncpy(bond_info->ports[info.port_name].ifname, ifname,
sizeof(bond_info->ports[0].ifname));
bond_info->ports[info.port_name].pci_addr = pci_addr;
bond_info->ports[info.port_name].ifindex = ifindex;
bond_info->n_port++;
net/mlx5: support socket direct mode bonding In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-26 08:48:30 +00:00
/*
* Under socket direct mode, bonding will use
* system_image_guid as identification.
* After OFED 5.4, guid is readable (ret >= 0) under sysfs.
* All bonding members should have the same guid even if driver
* is using PCIe BDF.
*/
ret = mlx5_get_device_guid(&pci_addr, guid, sizeof(guid));
if (ret < 0)
break;
else if (ret > 0) {
if (!memcmp(guid, cur_guid, sizeof(guid)) &&
owner == info.port_name &&
(owner != 0 || (owner == 0 &&
!rte_pci_addr_cmp(pci_dev, &pci_addr))))
pf = info.port_name;
} else if (pci_dev->domain == pci_addr.domain &&
pci_dev->bus == pci_addr.bus &&
pci_dev->devid == pci_addr.devid &&
((pci_dev->function == 0 &&
pci_dev->function + owner == pci_addr.function) ||
(pci_dev->function == owner &&
pci_addr.function == owner)))
pf = info.port_name;
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
}
if (pf >= 0) {
/* Get bond interface info */
ret = mlx5_sysfs_bond_info(ifindex, &bond_info->ifindex,
bond_info->ifname);
if (ret)
DRV_LOG(ERR, "unable to get bond info: %s",
strerror(rte_errno));
else
DRV_LOG(INFO, "PF device %u, bond device %u(%s)",
ifindex, bond_info->ifindex, bond_info->ifname);
}
net/mlx5: support socket direct mode bonding In socket direct mode, it's possible to bind any two (maybe four in future) PCIe devices with IDs like xxxx:xx:xx.x and yyyy:yy:yy.y. Bonding member interfaces are unnecessary to have the same PCIe domain/bus/device ID anymore, Kernel driver uses "system_image_guid" to identify if devices can be bound together or not. Sysfs "phys_switch_id" is used to get "system_image_guid" of each network interface. OFED 5.4+ is required to support "phys_switch_id". Signed-off-by: Rongwei Liu <rongweil@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-10-26 08:48:30 +00:00
if (owner == 0 && pf != 0) {
DRV_LOG(INFO, "PCIe instance %04x:%02x:%02x.%x isn't bonding owner",
pci_dev->domain, pci_dev->bus, pci_dev->devid,
pci_dev->function);
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return pf;
}
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
static void
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
mlx5_os_config_default(struct mlx5_dev_config *config)
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
{
memset(config, 0, sizeof(*config));
config->mps = MLX5_ARG_UNSET;
net/mlx5: remove redundant check of devargs The device arguments are parsed and updated twice during spawning. First time before creating the share device context, and again later after updating a default value to one of the arguments. This patch consolidates them into one parsing and updates the default values before it. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:57 +00:00
config->cqe_comp = 1;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
config->rx_vec_en = 1;
config->txq_inline_max = MLX5_ARG_UNSET;
config->txq_inline_min = MLX5_ARG_UNSET;
config->txq_inline_mpw = MLX5_ARG_UNSET;
config->txqs_inline = MLX5_ARG_UNSET;
config->vf_nl_en = 1;
config->mprq.max_memcpy_len = MLX5_MPRQ_MEMCPY_DEFAULT_LEN;
config->mprq.min_rxqs_num = MLX5_MPRQ_MIN_RXQS;
net/mlx5: fix MPRQ stride devargs adjustment In Multi-Packet RQ creation, the user can choose the number of strides and their size in bytes. The user updates it using specific devargs for both of these parameters. The above two parameters determine the size of the WQE which is actually their product of multiplication. If the user selects values that are not in the supported range, the PMD changes them to default values. However, apart from the range limitations for each parameter individually there is also a minimum value on their multiplication. When the user selects values that their multiplication are lower than minimum value, no adjustment is made and the creation of the WQE fails. This patch adds an adjustment in these cases as well. When the user selects values whose multiplication is lower than the minimum, they are replaced with the default values. Fixes: ecb160456aed ("net/mlx5: add device parameter for MPRQ stride size") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-11-23 18:38:05 +00:00
config->mprq.log_stride_num = MLX5_MPRQ_DEFAULT_LOG_STRIDE_NUM;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
config->dv_esw_en = 1;
config->dv_flow_en = 1;
config->decap_en = 1;
config->log_hp_size = MLX5_ARG_UNSET;
net/mlx5: fix duplicate pattern option default In order to allow/disallow configuring rules with identical patterns, the new device argument 'allow_duplicate_pattern' was introduced. The default is to allow, and it is initialized to 1 in PCI probe function. However, on auxiliary bus probing (for Sub-Function) it is not initialized at all, so it's actually initialized to 0. Move the initialization to default config function which is called from both. Fixes: 919488fbfa71 ("net/mlx5: support Sub-Function") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-09-12 10:36:25 +00:00
config->allow_duplicate_pattern = 1;
net/mlx5: support Rx queue delay drop For the Ethernet RQs, if there all receiving descriptors are exhausted, the packets being received will be dropped. This behavior prevents slow or malicious software entities at the host from affecting the network. While for hairpin cases, even if there is no software involved during the packet forwarding from Rx to Tx side, some hiccup in the hardware or back pressure from Tx side may still cause the descriptors to be exhausted. In certain scenarios it may be preferred to configure the device to avoid such packet drops, assuming the posting of descriptors will resume shortly. To support this, a new devarg "delay_drop" is introduced. By default, the delay drop is enabled for hairpin Rx queues and disabled for standard Rx queues. This value is used as a bit mask: - bit 0: enablement of standard Rx queue - bit 1: enablement of hairpin Rx queue And this attribute will be applied to all Rx queues of a device. The "rq_delay_drop" capability in the HCA_CAP is checked before creating any queue. If the hardware capabilities do not support this delay drop, all the Rx queues will still be created without this attribute, and the devarg setting will be ignored even if it is specified explicitly. A warning log is used to notify the application when this occurs. Signed-off-by: Bing Zhao <bingz@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-11-05 15:30:38 +00:00
config->std_delay_drop = 0;
config->hp_delay_drop = 0;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/**
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
* Register a PCI device within bonding.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
* This function spawns Ethernet devices out of a given PCI device and
* bonding owner PF index.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
* @param[in] cdev
* Pointer to common mlx5 device structure.
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
* @param[in] req_eth_da
* Requested ethdev device argument.
* @param[in] owner_id
* Requested owner PF port ID within bonding device, default to 0.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
static int
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
mlx5_os_pci_probe_pf(struct mlx5_common_device *cdev,
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
struct rte_eth_devargs *req_eth_da,
uint16_t owner_id)
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
{
struct ibv_device **ibv_list;
/*
* Number of found IB Devices matching with requested PCI BDF.
* nd != 1 means there are multiple IB devices over the same
* PCI device and we have representors and master.
*/
unsigned int nd = 0;
/*
* Number of found IB device Ports. nd = 1 and np = 1..n means
* we have the single multiport IB device, and there may be
* representors attached to some of found ports.
*/
unsigned int np = 0;
/*
* Number of DPDK ethernet devices to Spawn - either over
* multiple IB devices or multiple ports of single IB device.
* Actually this is the number of iterations to spawn.
*/
unsigned int ns = 0;
/*
* Bonding device
* < 0 - no bonding device (single one)
* >= 0 - bonding device (value is slave PF index)
*/
int bd = -1;
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
struct mlx5_dev_spawn_data *list = NULL;
struct mlx5_dev_config dev_config;
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
struct rte_eth_devargs eth_da = *req_eth_da;
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
struct rte_pci_addr owner_pci = pci_dev->addr; /* Owner PF. */
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
struct mlx5_bond_info bond_info;
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
int ret = -1;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
errno = 0;
ibv_list = mlx5_glue->get_device_list(&ret);
if (!ibv_list) {
rte_errno = errno ? errno : ENOSYS;
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR, "Cannot list devices, is ib_uverbs loaded?");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
return -rte_errno;
}
/*
* First scan the list of all Infiniband devices to find
* matching ones, gathering into the list.
*/
struct ibv_device *ibv_match[ret + 1];
int nl_route = mlx5_nl_init(NETLINK_ROUTE);
int nl_rdma = mlx5_nl_init(NETLINK_RDMA);
unsigned int i;
while (ret-- > 0) {
struct rte_pci_addr pci_addr;
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(DEBUG, "Checking device \"%s\"", ibv_list[ret]->name);
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
bd = mlx5_device_bond_pci_match(ibv_list[ret]->name, &owner_pci,
nl_rdma, owner_id, &bond_info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (bd >= 0) {
/*
* Bonding device detected. Only one match is allowed,
* the bonding is supported over multi-port IB device,
* there should be no matches on representor PCI
* functions or non VF LAG bonding devices with
* specified address.
*/
if (nd) {
DRV_LOG(ERR,
"multiple PCI match on bonding device"
"\"%s\" found", ibv_list[ret]->name);
rte_errno = ENOENT;
ret = -rte_errno;
goto exit;
}
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
/* Amend owner pci address if owner PF ID specified. */
if (eth_da.nb_representor_ports)
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
owner_pci.function += owner_id;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
DRV_LOG(INFO,
"PCI information matches for slave %d bonding device \"%s\"",
bd, ibv_list[ret]->name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
ibv_match[nd++] = ibv_list[ret];
break;
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
} else {
/* Bonding device not found. */
common/mlx5: get PCI device address from any bus A function is exported to allow retrieving the PCI address of the parent PCI device of a Sub-Function in auxiliary bus sysfs. The function mlx5_dev_to_pci_str() is accepting both PCI and auxiliary devices. In case of a PCI device, it is simply using the device name. The function mlx5_dev_to_pci_addr(), which is based on sysfs path and do not use any device object, is renamed to mlx5_get_pci_addr() for clarity purpose. Signed-off-by: Thomas Monjalon <thomas@monjalon.net> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:32 +00:00
if (mlx5_get_pci_addr(ibv_list[ret]->ibdev_path,
&pci_addr))
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
continue;
if (owner_pci.domain != pci_addr.domain ||
owner_pci.bus != pci_addr.bus ||
owner_pci.devid != pci_addr.devid ||
owner_pci.function != pci_addr.function)
continue;
DRV_LOG(INFO, "PCI information matches for device \"%s\"",
ibv_list[ret]->name);
ibv_match[nd++] = ibv_list[ret];
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
}
ibv_match[nd] = NULL;
if (!nd) {
/* No device matches, just complain and bail out. */
DRV_LOG(WARNING,
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
"No Verbs device matches PCI device " PCI_PRI_FMT ","
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
" are kernel drivers loaded?",
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
owner_pci.domain, owner_pci.bus,
owner_pci.devid, owner_pci.function);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
rte_errno = ENOENT;
ret = -rte_errno;
goto exit;
}
if (nd == 1) {
/*
* Found single matching device may have multiple ports.
* Each port may be representor, we have to check the port
* number and check the representors existence.
*/
if (nl_rdma >= 0)
np = mlx5_nl_portnum(nl_rdma, ibv_match[0]->name);
if (!np)
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(WARNING,
"Cannot get IB device \"%s\" ports number.",
ibv_match[0]->name);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (bd >= 0 && !np) {
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR, "Cannot get ports for bonding device.");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
rte_errno = ENOENT;
ret = -rte_errno;
goto exit;
}
}
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
/* Now we can determine the maximal amount of devices to be spawned. */
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
list = mlx5_malloc(MLX5_MEM_ZERO,
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
sizeof(struct mlx5_dev_spawn_data) * (np ? np : nd),
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
RTE_CACHE_LINE_SIZE, SOCKET_ID_ANY);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!list) {
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(ERR, "Spawn data array allocation failure.");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
rte_errno = ENOMEM;
ret = -rte_errno;
goto exit;
}
if (bd >= 0 || np > 1) {
/*
* Single IB device with multiple ports found,
* it may be E-Switch master device and representors.
* We have to perform identification through the ports.
*/
MLX5_ASSERT(nl_rdma >= 0);
MLX5_ASSERT(ns == 0);
MLX5_ASSERT(nd == 1);
MLX5_ASSERT(np);
for (i = 1; i <= np; ++i) {
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
list[ns].bond_info = &bond_info;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].max_port = np;
net/mlx5: remove Verbs dependency in spawn struct 1. Replace 'struct ibv_device *' with 'void *' in 'struct mlx5_dev_spawn_data'. Define a getter function to retrieve the device name. 2. Rename ibv_dev and ibv_port as phys_dev and phys_port respectively. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:02 +00:00
list[ns].phys_port = i;
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
list[ns].phys_dev_name = ibv_match[0]->name;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].eth_dev = NULL;
list[ns].pci_dev = pci_dev;
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
list[ns].cdev = cdev;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].pf_bond = bd;
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
list[ns].ifindex = mlx5_nl_ifindex(nl_rdma,
ibv_match[0]->name,
i);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!list[ns].ifindex) {
/*
* No network interface index found for the
* specified port, it means there is no
* representor on this port. It's OK,
* there can be disabled ports, for example
* if sriov_numvfs < sriov_totalvfs.
*/
continue;
}
ret = -1;
if (nl_route >= 0)
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
ret = mlx5_nl_switch_info(nl_route,
list[ns].ifindex,
&list[ns].info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (ret || (!list[ns].info.representor &&
!list[ns].info.master)) {
/*
* We failed to recognize representors with
* Netlink, let's try to perform the task
* with sysfs.
*/
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
ret = mlx5_sysfs_switch_info(list[ns].ifindex,
&list[ns].info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
if (!ret && bd >= 0) {
switch (list[ns].info.name_type) {
case MLX5_PHYS_PORT_NAME_TYPE_UPLINK:
net/mlx5: fix RoCE LAG bond device probing The RoCE LAG bond device requires neither E-Switch nor SR-IOV configurations. It means the RoCE LAG bond device might be presented as a single port Infiniband device. The mlx5 PMD wrongly recognized standalone RoCE LAG bond device as E-Switch configuration, this triggered the calls of E-Switch ports related API and the latter failed (over the new OFED kernel driver, starting since 5.4.1), causing the overall device probe failure. If there is a single port Infiniband bond device found the E-Switch related flags must be cleared indicating standalone configuration. Also, it is not true anymore the bond device can exist over E-Switch configurations only (as it was claimed for VF LAG bond devices). The related checks are not relevant anymore and removed. Fixes: 790164ce1d2d ("net/mlx5: check kernel support for VF LAG bonding") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-21 08:31:40 +00:00
if (np == 1) {
/*
* Force standalone bonding
* device for ROCE LAG
fix spelling in comments and strings The tool comes from https://github.com/jsoref Signed-off-by: Josh Soref <jsoref@gmail.com> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-11-29 16:08:02 +00:00
* configurations.
net/mlx5: fix RoCE LAG bond device probing The RoCE LAG bond device requires neither E-Switch nor SR-IOV configurations. It means the RoCE LAG bond device might be presented as a single port Infiniband device. The mlx5 PMD wrongly recognized standalone RoCE LAG bond device as E-Switch configuration, this triggered the calls of E-Switch ports related API and the latter failed (over the new OFED kernel driver, starting since 5.4.1), causing the overall device probe failure. If there is a single port Infiniband bond device found the E-Switch related flags must be cleared indicating standalone configuration. Also, it is not true anymore the bond device can exist over E-Switch configurations only (as it was claimed for VF LAG bond devices). The related checks are not relevant anymore and removed. Fixes: 790164ce1d2d ("net/mlx5: check kernel support for VF LAG bonding") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-07-21 08:31:40 +00:00
*/
list[ns].info.master = 0;
list[ns].info.representor = 0;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (list[ns].info.port_name == bd)
ns++;
break;
net/mlx5: fix host physical function representor naming The new kernel adds the names like "pf0" for Host PCI physical function representor on Bluefield SmartNIC hosts. This patch provides correct HPF representor recognition over the kernel versions 5.7 and laters. The following port naming formats are supported: - missing physical port name (no sysfs/netlink key) at all, master is assumed - decimal digits (for example "12"), representor is assumed, the value is the index of attached VF - "p" followed by decimal digits, for example "p2", master is assumed - "pf" followed by PF index, for example "pf0", Host PF representor is assumed on SmartNIC systems. - "pf" followed by PF index concatenated with "vf" followed by VF index, for example "pf0vf1", representor is assumed. If index of VF is "-1" it is a special case of Host PF representor, this representor must be indexed in devargs as 65535, for example representor=[0-3,65535] will allow representors for VF0, VF1, VF2, VF3 and for host PF. Fixes: 79aa430721b1 ("common/mlx5: split common file under Linux directory") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-23 07:48:34 +00:00
case MLX5_PHYS_PORT_NAME_TYPE_PFHPF:
/* Fallthrough */
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
case MLX5_PHYS_PORT_NAME_TYPE_PFVF:
net/mlx5: support sub-function representor This patch adds support for SF representor. Similar to VF representor, switch port name of SF representor in phys_port_name sysfs key is "pf<x>sf<y>". Device representor argument is "representors=sf[list]", list member could be mix of instance and range. Example: representors=sf[0,2,4,8-12,-1] To probe VF representor and SF representor, need to separate into 2 devices: -a <BDF>,representor=vf[list] -a <BDF>,representor=sf[list] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:08 +00:00
/* Fallthrough */
case MLX5_PHYS_PORT_NAME_TYPE_PFSF:
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (list[ns].info.pf_num == bd)
ns++;
break;
default:
break;
}
continue;
}
if (!ret && (list[ns].info.representor ^
list[ns].info.master))
ns++;
}
if (!ns) {
DRV_LOG(ERR,
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
"Unable to recognize master/representors on the IB device with multiple ports.");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
rte_errno = ENOENT;
ret = -rte_errno;
goto exit;
}
} else {
/*
* The existence of several matching entries (nd > 1) means
* port representors have been instantiated. No existing Verbs
* call nor sysfs entries can tell them apart, this can only
* be done through Netlink calls assuming kernel drivers are
* recent enough to support them.
*
* In the event of identification failure through Netlink,
* try again through sysfs, then:
*
* 1. A single IB device matches (nd == 1) with single
* port (np=0/1) and is not a representor, assume
* no switch support.
*
* 2. Otherwise no safe assumptions can be made;
* complain louder and bail out.
*/
for (i = 0; i != nd; ++i) {
memset(&list[ns].info, 0, sizeof(list[ns].info));
net/mlx5: save bonding member ports information Since kernel bonding netdev doesn't provide statistics counter that reflects all member ports, PMD has to manually summarize counters from each member ports. As a preparation, this patch collects bonding member port information and saves to shared context data. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:12 +00:00
list[ns].bond_info = NULL;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].max_port = 1;
net/mlx5: remove Verbs dependency in spawn struct 1. Replace 'struct ibv_device *' with 'void *' in 'struct mlx5_dev_spawn_data'. Define a getter function to retrieve the device name. 2. Rename ibv_dev and ibv_port as phys_dev and phys_port respectively. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:02 +00:00
list[ns].phys_port = 1;
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
list[ns].phys_dev_name = ibv_match[i]->name;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].eth_dev = NULL;
list[ns].pci_dev = pci_dev;
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
list[ns].cdev = cdev;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
list[ns].pf_bond = -1;
list[ns].ifindex = 0;
if (nl_rdma >= 0)
list[ns].ifindex = mlx5_nl_ifindex
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
(nl_rdma,
ibv_match[i]->name,
1);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!list[ns].ifindex) {
char ifname[IF_NAMESIZE];
/*
* Netlink failed, it may happen with old
* ib_core kernel driver (before 4.16).
* We can assume there is old driver because
* here we are processing single ports IB
* devices. Let's try sysfs to retrieve
* the ifindex. The method works for
* master device only.
*/
if (nd > 1) {
/*
* Multiple devices found, assume
* representors, can not distinguish
* master/representor and retrieve
* ifindex via sysfs.
*/
continue;
}
common/mlx5: share kernel interface name getter Some configuration of the mlx5 port are done by the kernel net device associated to the IB device represents the PCI device. The DPDK mlx5 driver uses Linux system calls, for example ioctl, in order to configure per port configurations requested by the DPDK user. One of the basic knowledges required to access the correct kernel net device is its name. Move function to get interface name from IB device path to the common library. Signed-off-by: Matan Azrad <matan@mellanox.com> Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-06-18 19:06:02 +00:00
ret = mlx5_get_ifname_sysfs
(ibv_match[i]->ibdev_path, ifname);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!ret)
list[ns].ifindex =
if_nametoindex(ifname);
if (!list[ns].ifindex) {
/*
* No network interface index found
* for the specified device, it means
* there it is neither representor
* nor master.
*/
continue;
}
}
ret = -1;
if (nl_route >= 0)
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
ret = mlx5_nl_switch_info(nl_route,
list[ns].ifindex,
&list[ns].info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (ret || (!list[ns].info.representor &&
!list[ns].info.master)) {
/*
* We failed to recognize representors with
* Netlink, let's try to perform the task
* with sysfs.
*/
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
ret = mlx5_sysfs_switch_info(list[ns].ifindex,
&list[ns].info);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
if (!ret && (list[ns].info.representor ^
list[ns].info.master)) {
ns++;
} else if ((nd == 1) &&
!list[ns].info.representor &&
!list[ns].info.master) {
/*
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
* Single IB device with one physical port and
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
* attached network device.
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
* May be SRIOV is not enabled or there is no
* representors.
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
*/
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
DRV_LOG(INFO, "No E-Switch support detected.");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
ns++;
break;
}
}
if (!ns) {
DRV_LOG(ERR,
common/mlx5: move basic probing functions to common Move open IBV/DevX device function to common. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:50 +00:00
"Unable to recognize master/representors on the multiple IB devices.");
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
rte_errno = ENOENT;
ret = -rte_errno;
goto exit;
}
net/mlx5: fix switchdev mode recognition The new kernels might add the switch_id attribute to the Netlink replies and this caused the wrong recognition of the E-Switch presence. The single uplink device was erroneously recognized as master and it caused the extending match for source vport index on all installed flows, including the default ones, and adding extra hops in the steering engine, that affected the maximal throughput packet rate. The extra check for the new device name format (it supposes the new kernel) and the device is only one is added. If this check succeeds the E-Switch presence is considered as wrongly detected and overridden. Fixes: 30a86157f6d5 ("net/mlx5: support PF representor") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-06-11 15:37:19 +00:00
/*
* New kernels may add the switch_id attribute for the case
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
* there is no E-Switch and we wrongly recognized the only
* device as master. Override this if there is the single
* device with single port and new device name format present.
net/mlx5: fix switchdev mode recognition The new kernels might add the switch_id attribute to the Netlink replies and this caused the wrong recognition of the E-Switch presence. The single uplink device was erroneously recognized as master and it caused the extending match for source vport index on all installed flows, including the default ones, and adding extra hops in the steering engine, that affected the maximal throughput packet rate. The extra check for the new device name format (it supposes the new kernel) and the device is only one is added. If this check succeeds the E-Switch presence is considered as wrongly detected and overridden. Fixes: 30a86157f6d5 ("net/mlx5: support PF representor") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-06-11 15:37:19 +00:00
*/
if (nd == 1 &&
list[0].info.name_type == MLX5_PHYS_PORT_NAME_TYPE_UPLINK) {
list[0].info.master = 0;
list[0].info.representor = 0;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
MLX5_ASSERT(ns);
/*
* Sort list to probe devices in natural order for users convenience
* (i.e. master first, then representors from lowest to highest ID).
*/
qsort(list, ns, sizeof(*list), mlx5_dev_spawn_data_cmp);
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
if (eth_da.type != RTE_ETH_REPRESENTOR_NONE) {
/* Set devargs default values. */
if (eth_da.nb_mh_controllers == 0) {
eth_da.nb_mh_controllers = 1;
eth_da.mh_controllers[0] = 0;
}
if (eth_da.nb_ports == 0 && ns > 0) {
if (list[0].pf_bond >= 0 && list[0].info.representor)
DRV_LOG(WARNING, "Representor on Bonding device should use pf#vf# syntax: %s",
pci_dev->device.devargs->args);
eth_da.nb_ports = 1;
eth_da.ports[0] = list[0].info.pf_num;
}
if (eth_da.nb_representor_ports == 0) {
eth_da.nb_representor_ports = 1;
eth_da.representor_ports[0] = 0;
}
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
for (i = 0; i != ns; ++i) {
uint32_t restore;
net/mlx5: optimize stack memory in probe The device configuration struct is not small enough to be used as function argument by value. Call spawn function with device configuration by reference. Signed-off-by: Michael Baum <michaelba@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-21 12:05:16 +00:00
/* Default configuration. */
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
mlx5_os_config_default(&dev_config);
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
list[i].eth_dev = mlx5_dev_spawn(cdev->dev, &list[i],
&dev_config, &eth_da);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (!list[i].eth_dev) {
if (rte_errno != EBUSY && rte_errno != EEXIST)
break;
/* Device is disabled or already spawned. Ignore it. */
continue;
}
restore = list[i].eth_dev->data->dev_flags;
rte_eth_copy_pci_info(list[i].eth_dev, pci_dev);
net/mlx5: fix representor interrupt handler In mlx5 PMD the PCI device interrupt vector was used by Uplink representor exclusively and other VF representors did not support interrupt mode. All the VFs and Uplink representors are separate ethernet devices and must have dedicated interrupt vectors. The fix provides each representor with a dedicated interrupt vector. Fixes: 5882bde88da2 ("net/mlx5: fix representor interrupts handler") Cc: stable@dpdk.org Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-20 15:38:19 +00:00
/**
* Each representor has a dedicated interrupts vector.
* rte_eth_copy_pci_info() assigns PF interrupts handle to
* representor eth_dev object because representor and PF
* share the same PCI address.
* Override representor device with a dedicated
* interrupts handle here.
* Representor interrupts handle is released in mlx5_dev_stop().
*/
if (list[i].info.representor) {
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
struct rte_intr_handle *intr_handle =
rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
if (intr_handle == NULL) {
net/mlx5: fix representor interrupt handler In mlx5 PMD the PCI device interrupt vector was used by Uplink representor exclusively and other VF representors did not support interrupt mode. All the VFs and Uplink representors are separate ethernet devices and must have dedicated interrupt vectors. The fix provides each representor with a dedicated interrupt vector. Fixes: 5882bde88da2 ("net/mlx5: fix representor interrupts handler") Cc: stable@dpdk.org Signed-off-by: Gregory Etelson <getelson@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-20 15:38:19 +00:00
DRV_LOG(ERR,
"port %u failed to allocate memory for interrupt handler "
"Rx interrupts will not be supported",
i);
rte_errno = ENOMEM;
ret = -rte_errno;
goto exit;
}
list[i].eth_dev->intr_handle = intr_handle;
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/* Restore non-PCI flags cleared by the above call. */
list[i].eth_dev->data->dev_flags |= restore;
rte_eth_dev_probing_finish(list[i].eth_dev);
}
if (i != ns) {
DRV_LOG(ERR,
"probe of PCI device " PCI_PRI_FMT " aborted after"
" encountering an error: %s",
net/mlx5: refactor bonding representor probing To probe representor on 2nd PF of kernel bonding device, had to specify PF1 BDF in devarg: <PF1_BDF>,representor=0 When closing bonding device, all representors had to be closed together and this implies all representors have to use primary PF of bonding device. So after probing representor port on 2nd PF, when locating new probed device using device argument, the filter used 2nd PF as PCI address and failed to locate new device. Conflict happened by using current representor devargs: - Use PCI BDF to specify representor owner PF - Use PCI BDF to locate probed representor device. - PMD uses primary PCI BDF as PCI device. To resolve such conflicts, new representor syntax is introduced here: <primary BDF>,representor=pfXvfY All representors must use primary PF as owner PCI device, PMD internally locate owner PCI address by checking representor "pfX" part. To EAL, all representors are registered to primary PCI device, the 2nd PF is hidden to EAL, thus all search should be consistent. Same to VF representor, HPF (host PF on BlueField) uses same syntax to probe, example: representor=pf1vf[0-3,-1] This patch also adds pf index into kernel bonding representor port name: <BDF>_<ib_name>_representor_pf<X>vf<Y> Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:10 +00:00
owner_pci.domain, owner_pci.bus,
owner_pci.devid, owner_pci.function,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
strerror(rte_errno));
ret = -rte_errno;
/* Roll back. */
while (i--) {
if (!list[i].eth_dev)
continue;
mlx5_dev_close(list[i].eth_dev);
/* mac_addrs must not be freed because in dev_private */
list[i].eth_dev->data->mac_addrs = NULL;
claim_zero(rte_eth_dev_release_port(list[i].eth_dev));
}
/* Restore original error. */
rte_errno = -ret;
} else {
ret = 0;
}
exit:
/*
* Do the routine cleanup:
* - close opened Netlink sockets
* - free allocated spawn data array
* - free the Infiniband device list
*/
if (nl_rdma >= 0)
close(nl_rdma);
if (nl_route >= 0)
close(nl_route);
if (list)
net/mlx5: convert configuration objects to unified malloc This commit allocates the miscellaneous configuration objects from the unified malloc function. Signed-off-by: Suanming Mou <suanmingm@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-28 09:02:44 +00:00
mlx5_free(list);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
MLX5_ASSERT(ibv_list);
mlx5_glue->free_device_list(ibv_list);
return ret;
}
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
static int
mlx5_os_parse_eth_devargs(struct rte_device *dev,
struct rte_eth_devargs *eth_da)
{
int ret = 0;
if (dev->devargs == NULL)
return 0;
memset(eth_da, 0, sizeof(*eth_da));
/* Parse representor information first from class argument. */
if (dev->devargs->cls_str)
ret = rte_eth_devargs_parse(dev->devargs->cls_str, eth_da);
if (ret != 0) {
DRV_LOG(ERR, "failed to parse device arguments: %s",
dev->devargs->cls_str);
return -rte_errno;
}
if (eth_da->type == RTE_ETH_REPRESENTOR_NONE) {
/* Parse legacy device argument */
ret = rte_eth_devargs_parse(dev->devargs->args, eth_da);
if (ret) {
DRV_LOG(ERR, "failed to parse device arguments: %s",
dev->devargs->args);
return -rte_errno;
}
}
return 0;
}
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
/**
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
* Callback to register a PCI device.
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
*
* This function spawns Ethernet devices out of a given PCI device.
*
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
* @param[in] cdev
* Pointer to common mlx5 device structure.
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
*
* @return
* 0 on success, a negative errno value otherwise and rte_errno is set.
*/
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
static int
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
mlx5_os_pci_probe(struct mlx5_common_device *cdev)
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
{
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
struct rte_pci_device *pci_dev = RTE_DEV_TO_PCI(cdev->dev);
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
struct rte_eth_devargs eth_da = { .nb_ports = 0 };
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
int ret = 0;
uint16_t p;
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
ret = mlx5_os_parse_eth_devargs(cdev->dev, &eth_da);
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
if (ret != 0)
return ret;
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
if (eth_da.nb_ports > 0) {
/* Iterate all port if devargs pf is range: "pf[0-1]vf[...]". */
net/mlx5: fix PF leak on PCI probing failure During PCI probe, the internal probe function is called per PF. If one of them fails, it was missing a proper destroy for the previously probed PFs. This fixes the behavior by destroying all previously probed PFs. Fixes: 08c2772fc747 ("net/mlx5: support list of representor PF") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-09-12 10:36:24 +00:00
for (p = 0; p < eth_da.nb_ports; p++) {
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
ret = mlx5_os_pci_probe_pf(cdev, &eth_da,
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
eth_da.ports[p]);
net/mlx5: fix PF leak on PCI probing failure During PCI probe, the internal probe function is called per PF. If one of them fails, it was missing a proper destroy for the previously probed PFs. This fixes the behavior by destroying all previously probed PFs. Fixes: 08c2772fc747 ("net/mlx5: support list of representor PF") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-09-12 10:36:24 +00:00
if (ret)
break;
}
if (ret) {
DRV_LOG(ERR, "Probe of PCI device " PCI_PRI_FMT " "
fix spelling in comments and strings The tool comes from https://github.com/jsoref Signed-off-by: Josh Soref <jsoref@gmail.com> Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2021-11-29 16:08:02 +00:00
"aborted due to prodding failure of PF %u",
net/mlx5: fix PF leak on PCI probing failure During PCI probe, the internal probe function is called per PF. If one of them fails, it was missing a proper destroy for the previously probed PFs. This fixes the behavior by destroying all previously probed PFs. Fixes: 08c2772fc747 ("net/mlx5: support list of representor PF") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-09-12 10:36:24 +00:00
pci_dev->addr.domain, pci_dev->addr.bus,
pci_dev->addr.devid, pci_dev->addr.function,
eth_da.ports[p]);
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
mlx5_net_remove(cdev);
net/mlx5: fix PF leak on PCI probing failure During PCI probe, the internal probe function is called per PF. If one of them fails, it was missing a proper destroy for the previously probed PFs. This fixes the behavior by destroying all previously probed PFs. Fixes: 08c2772fc747 ("net/mlx5: support list of representor PF") Cc: stable@dpdk.org Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-09-12 10:36:24 +00:00
}
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
} else {
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
ret = mlx5_os_pci_probe_pf(cdev, &eth_da, 0);
net/mlx5: support list of representor PF To probe representors from different kernel bonding PFs, had to specify 2 separate devargs like this: -a 03:00.0,representor=pf0vf[0-3] -a 03:00.0,representor=pf1vf[0-3] This patch supports range or list of PF section in devargs, so the alternative short devargs of above is: -a 03:00.0,representor=pf[0-1]vf[0-3] Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-03-28 13:48:11 +00:00
}
return ret;
}
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
/* Probe a single SF device on auxiliary bus, no representor support. */
static int
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
mlx5_os_auxiliary_probe(struct mlx5_common_device *cdev)
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
{
struct rte_eth_devargs eth_da = { .nb_ports = 0 };
struct mlx5_dev_config config;
struct mlx5_dev_spawn_data spawn = { .pf_bond = -1 };
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
struct rte_device *dev = cdev->dev;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
struct rte_auxiliary_device *adev = RTE_DEV_TO_AUXILIARY(dev);
struct rte_eth_dev *eth_dev;
int ret = 0;
/* Parse ethdev devargs. */
ret = mlx5_os_parse_eth_devargs(dev, &eth_da);
if (ret != 0)
return ret;
/* Set default config data. */
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
mlx5_os_config_default(&config);
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
/* Init spawn data. */
spawn.max_port = 1;
spawn.phys_port = 1;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
spawn.phys_dev_name = mlx5_os_get_ctx_device_name(cdev->ctx);
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
ret = mlx5_auxiliary_get_ifindex(dev->name);
if (ret < 0) {
DRV_LOG(ERR, "failed to get ethdev ifindex: %s", dev->name);
return ret;
}
spawn.ifindex = ret;
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
spawn.cdev = cdev;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
/* Spawn device. */
eth_dev = mlx5_dev_spawn(dev, &spawn, &config, &eth_da);
if (eth_dev == NULL)
return -rte_errno;
/* Post create. */
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
eth_dev->intr_handle = adev->intr_handle;
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
if (rte_eal_process_type() == RTE_PROC_PRIMARY) {
eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_LSC;
eth_dev->data->dev_flags |= RTE_ETH_DEV_INTR_RMV;
eth_dev->data->numa_node = dev->numa_node;
}
rte_eth_dev_probing_finish(eth_dev);
return 0;
}
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
/**
* Net class driver callback to probe a device.
*
net/mlx5: support Sub-Function Introduce SF support. Similar to VF, SF on auxiliary bus is a portion of hardware PF, no representor or bonding parameters for SF. Devargs to support SF: -a auxiliary:mlx5_core.sf.8,dv_flow_en=1 New global syntax to support SF: -a bus=auxiliary,name=mlx5_core.sf.8/class=eth/driver=mlx5,dv_flow_en=1 Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:35 +00:00
* This function probe PCI bus device(s) or a single SF on auxiliary bus.
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
*
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
* @param[in] cdev
* Pointer to the common mlx5 device.
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
*
* @return
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
* 0 on success, a negative errno value otherwise and rte_errno is set.
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
*/
int
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
mlx5_os_net_probe(struct mlx5_common_device *cdev)
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
{
int ret;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
if (rte_eal_process_type() == RTE_PROC_PRIMARY)
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
mlx5_pmd_socket_init();
ret = mlx5_init_once();
if (ret) {
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
DRV_LOG(ERR, "Unable to init PMD global data: %s",
net/mlx5: migrate to bus-agnostic common interface To support SubFunction based on auxiliary bus, common driver supports new bus-agnostic driver. This patch migrates net driver to new common driver. Signed-off-by: Xueming Li <xuemingl@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-21 14:37:34 +00:00
strerror(rte_errno));
return -rte_errno;
}
common/mlx5: share basic probing with internal drivers Create common probing structure that includes, for now, basic probing information detected by the common driver and share it with all the internal drivers. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:46 +00:00
if (mlx5_dev_is_pci(cdev->dev))
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
return mlx5_os_pci_probe(cdev);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
else
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
return mlx5_os_auxiliary_probe(cdev);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
net/mlx5: close tools socket with last device MLX5 PMD exposes a socket for external tools to dump port state. Socket events are listened using an interrupt source of EXT type. The socket was closed and the interrupt callback was unregistered at program exit, which is incorrect because DPDK could be already shut down at this point. Move actions performed at program exit to the moment the last MLX5 port is closed. The socket will be opened again if later a new MLX5 device is plugged in and probed. Also fix comments that were decisively talking about secondary processes instead of external tools. Fixes: e6cdc54cc0ef ("net/mlx5: add socket server for external tools") Cc: stable@dpdk.org Reported-by: Harman Kalra <hkalra@marvell.com> Signed-off-by: Dmitry Kozlyuk <dkozlyuk@nvidia.com> Acked-by: Thomas Monjalon <thomas@monjalon.net>
2021-10-14 08:55:28 +00:00
/**
* Cleanup resources when the last device is closed.
*/
void
mlx5_os_net_cleanup(void)
{
mlx5_pmd_socket_uninit();
}
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
/**
* Install shared asynchronous device events handler.
* This function is implemented to support event sharing
* between multiple ports of single IB device.
*
* @param sh
* Pointer to mlx5_dev_ctx_shared object.
*/
void
mlx5_os_dev_shared_handler_install(struct mlx5_dev_ctx_shared *sh)
{
int ret;
int flags;
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
struct ibv_context *ctx = sh->cdev->ctx;
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
sh->intr_handle = rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
if (sh->intr_handle == NULL) {
DRV_LOG(ERR, "Fail to allocate intr_handle");
rte_errno = ENOMEM;
return;
}
rte_intr_fd_set(sh->intr_handle, -1);
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
flags = fcntl(ctx->async_fd, F_GETFL);
ret = fcntl(ctx->async_fd, F_SETFL, flags | O_NONBLOCK);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (ret) {
DRV_LOG(INFO, "failed to change file descriptor async event"
" queue");
} else {
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_fd_set(sh->intr_handle, ctx->async_fd);
rte_intr_type_set(sh->intr_handle, RTE_INTR_HANDLE_EXT);
if (rte_intr_callback_register(sh->intr_handle,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_dev_interrupt_handler, sh)) {
DRV_LOG(INFO, "Fail to install the shared interrupt.");
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_fd_set(sh->intr_handle, -1);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
}
net/mlx5: remove DevX flag duplication The sharing device context structure has a field named "devx" which indicates if DevX is supported. The common configure structure has also field named "devx" with the same meaning. There is no need for this duplication, because there is a reference to the common structure from within the sharing device context structure. This patch removes it from sharing device context structure and uses the common config structure instead. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:34:59 +00:00
if (sh->cdev->config.devx) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifdef HAVE_IBV_DEVX_ASYNC
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
sh->intr_handle_devx =
rte_intr_instance_alloc(RTE_INTR_INSTANCE_F_SHARED);
if (!sh->intr_handle_devx) {
DRV_LOG(ERR, "Fail to allocate intr_handle");
rte_errno = ENOMEM;
return;
}
rte_intr_fd_set(sh->intr_handle_devx, -1);
common/mlx5: share device context object Create shared context device in common area and add it as a field of common device. Use this context device in all drivers and remove the ctx field from their private structure. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2021-10-19 20:55:52 +00:00
sh->devx_comp = (void *)mlx5_glue->devx_create_cmd_comp(ctx);
net/mlx5: remove completion object dependency on DV Replace 'struct mlx5dv_devx_cmd_comp *' with 'void *' in 'struct mlx5_dev_ctx_shared'. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:26 +00:00
struct mlx5dv_devx_cmd_comp *devx_comp = sh->devx_comp;
if (!devx_comp) {
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
DRV_LOG(INFO, "failed to allocate devx_comp.");
return;
}
net/mlx5: remove completion object dependency on DV Replace 'struct mlx5dv_devx_cmd_comp *' with 'void *' in 'struct mlx5_dev_ctx_shared'. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:26 +00:00
flags = fcntl(devx_comp->fd, F_GETFL);
ret = fcntl(devx_comp->fd, F_SETFL, flags | O_NONBLOCK);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (ret) {
DRV_LOG(INFO, "failed to change file descriptor"
" devx comp");
return;
}
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_fd_set(sh->intr_handle_devx, devx_comp->fd);
rte_intr_type_set(sh->intr_handle_devx,
RTE_INTR_HANDLE_EXT);
if (rte_intr_callback_register(sh->intr_handle_devx,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_dev_interrupt_handler_devx, sh)) {
DRV_LOG(INFO, "Fail to install the devx shared"
" interrupt.");
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_fd_set(sh->intr_handle_devx, -1);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
}
#endif /* HAVE_IBV_DEVX_ASYNC */
}
}
/**
* Uninstall shared asynchronous device events handler.
* This function is implemented to support event sharing
* between multiple ports of single IB device.
*
* @param dev
* Pointer to mlx5_dev_ctx_shared object.
*/
void
mlx5_os_dev_shared_handler_uninstall(struct mlx5_dev_ctx_shared *sh)
{
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
if (rte_intr_fd_get(sh->intr_handle) >= 0)
mlx5_intr_callback_unregister(sh->intr_handle,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_dev_interrupt_handler, sh);
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_instance_free(sh->intr_handle);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
#ifdef HAVE_IBV_DEVX_ASYNC
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
if (rte_intr_fd_get(sh->intr_handle_devx) >= 0)
rte_intr_callback_unregister(sh->intr_handle_devx,
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
mlx5_dev_interrupt_handler_devx, sh);
drivers: remove direct access to interrupt handle Removing direct access to interrupt handle structure fields, rather use respective get set APIs for the same. Making changes to all the drivers access the interrupt handle fields. Signed-off-by: Harman Kalra <hkalra@marvell.com> Acked-by: Hyong Youb Kim <hyonkim@cisco.com> Signed-off-by: David Marchand <david.marchand@redhat.com> Tested-by: Raslan Darawsheh <rasland@nvidia.com>
2021-10-22 20:49:32 +00:00
rte_intr_instance_free(sh->intr_handle_devx);
net/mlx5: refactor PCI probing on Linux Refactor PCI probing related code. Move Linux specific functions (as well as verbs and dv related code) from mlx5.c file to linux/mlx5_os.c file. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-03 15:06:00 +00:00
if (sh->devx_comp)
mlx5_glue->devx_destroy_cmd_comp(sh->devx_comp);
#endif
}
net/mlx5: refactor device operations for Linux There are three types of eth_dev_ops: primary, secondary and isolate. Their function calls assignments are moved from common file mlx5.c to the Linux specific file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:30 +00:00
net/mlx5: refactor statistics mlx5 statistics are calculated by several methods: 1. In software when packets go through datapath. 2. Calling ioctl with ETHTOOL command (Linux specific). 3. Reading counters from SYSFS device path (Linux specific). The Linux related functions are moved to file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:33 +00:00
/**
* Read statistics by a named counter.
*
* @param[in] priv
* Pointer to the private device data structure.
* @param[in] ctr_name
* Pointer to the name of the statistic counter to read
* @param[out] stat
* Pointer to read statistic value.
* @return
* 0 on success and stat is valud, 1 if failed to read the value
* rte_errno is set.
*
*/
int
mlx5_os_read_dev_stat(struct mlx5_priv *priv, const char *ctr_name,
uint64_t *stat)
{
int fd;
if (priv->sh) {
net/mlx5: fix imissed statistics The imissed port statistic counts packets that were dropped by the device Rx queues. In mlx5, the imissed counter summarizes 2 counters: - packets dropped by the SW queue handling counted by SW. - packets dropped by the HW queues due to "out of buffer" events detected when no SW buffer is available for the incoming packets. There is HW counter object that should be created per device, and all the Rx queues should be assigned to this counter in configuration time. This part was missed when the Rx queues were created by DevX what remained the "out of buffer" counter clean forever in this case. Add 2 options to assign the DevX Rx queues to queue counter: - Create queue counter per device by DevX and assign all the queues to it. - Query the kernel counter and assign all the queues to it. Use the first option by default and if it is failed, fallback to the second option. Fixes: e79c9be91515 ("net/mlx5: support Rx hairpin queues") Fixes: dc9ceff73c99 ("net/mlx5: create advanced RxQ via DevX") Cc: stable@dpdk.org Signed-off-by: Matan Azrad <matan@nvidia.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-02-25 10:45:01 +00:00
if (priv->q_counters != NULL &&
strcmp(ctr_name, "out_of_buffer") == 0)
net/mlx5/linux: fix missed Rx packet stats There was a typo - the device context was wrongly provided instead of counter's one for the DevX query operation. Fixes: e6988afdc75a ("net/mlx5: fix imissed statistics") Cc: stable@dpdk.org Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-04-28 09:31:45 +00:00
return mlx5_devx_cmd_queue_counter_query
(priv->q_counters, 0, (uint32_t *)stat);
net/mlx5: refactor statistics mlx5 statistics are calculated by several methods: 1. In software when packets go through datapath. 2. Calling ioctl with ETHTOOL command (Linux specific). 3. Reading counters from SYSFS device path (Linux specific). The Linux related functions are moved to file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:33 +00:00
MKSTR(path, "%s/ports/%d/hw_counters/%s",
net/mlx5: fix HW counters path in switchdev mode When debugging performance of a DPDK application the user may need to view the different statistics of DPDK (for example out_of_buffer) This can be enabled by using testpmd command 'show port xstats <port_id>' for example. The current implementation assumes legacy mode in which the counters are at <ibdev_path>/<port_id>/hw_counters/<file_name>. In switchdev mode the counters file is located right after the device name, hence resides at <ibdev_path>/hw_counters. The fix tries to open the path in the second location after a failure to open the file from the first location. Fixes: 9c0a9eed37f1 ("net/mlx5: switch to the names in the shared IB context") Cc: stable@dpdk.org Signed-off-by: Shy Shyman <shys@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-15 10:50:55 +00:00
priv->sh->ibdev_path,
priv->dev_port,
ctr_name);
net/mlx5: refactor statistics mlx5 statistics are calculated by several methods: 1. In software when packets go through datapath. 2. Calling ioctl with ETHTOOL command (Linux specific). 3. Reading counters from SYSFS device path (Linux specific). The Linux related functions are moved to file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:33 +00:00
fd = open(path, O_RDONLY);
net/mlx5: fix HW counters path in switchdev mode When debugging performance of a DPDK application the user may need to view the different statistics of DPDK (for example out_of_buffer) This can be enabled by using testpmd command 'show port xstats <port_id>' for example. The current implementation assumes legacy mode in which the counters are at <ibdev_path>/<port_id>/hw_counters/<file_name>. In switchdev mode the counters file is located right after the device name, hence resides at <ibdev_path>/hw_counters. The fix tries to open the path in the second location after a failure to open the file from the first location. Fixes: 9c0a9eed37f1 ("net/mlx5: switch to the names in the shared IB context") Cc: stable@dpdk.org Signed-off-by: Shy Shyman <shys@mellanox.com> Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
2020-07-15 10:50:55 +00:00
/*
* in switchdev the file location is not per port
* but rather in <ibdev_path>/hw_counters/<file_name>.
*/
if (fd == -1) {
MKSTR(path1, "%s/hw_counters/%s",
priv->sh->ibdev_path,
ctr_name);
fd = open(path1, O_RDONLY);
}
net/mlx5: refactor statistics mlx5 statistics are calculated by several methods: 1. In software when packets go through datapath. 2. Calling ioctl with ETHTOOL command (Linux specific). 3. Reading counters from SYSFS device path (Linux specific). The Linux related functions are moved to file linux/mlx5_os.c. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-06-10 09:32:33 +00:00
if (fd != -1) {
char buf[21] = {'\0'};
ssize_t n = read(fd, buf, sizeof(buf));
close(fd);
if (n != -1) {
*stat = strtoull(buf, NULL, 10);
return 0;
}
}
}
*stat = 0;
return 1;
}
net/mlx5: refactor Linux MAC operations Move OS specific MAC operations add, remove, modify VF into file linux/mlx5_os.c. Remove unused function mlx5_get_mac(). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:11 +00:00
/**
* Remove a MAC address from device
*
* @param dev
* Pointer to Ethernet device structure.
* @param index
* MAC address index.
*/
void
mlx5_os_mac_addr_remove(struct rte_eth_dev *dev, uint32_t index)
{
struct mlx5_priv *priv = dev->data->dev_private;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
const int vf = priv->sh->dev_cap.vf;
net/mlx5: refactor Linux MAC operations Move OS specific MAC operations add, remove, modify VF into file linux/mlx5_os.c. Remove unused function mlx5_get_mac(). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:11 +00:00
if (vf)
mlx5_nl_mac_addr_remove(priv->nl_socket_route,
mlx5_ifindex(dev), priv->mac_own,
&dev->data->mac_addrs[index], index);
}
/**
* Adds a MAC address to the device
*
* @param dev
* Pointer to Ethernet device structure.
* @param mac_addr
* MAC address to register.
* @param index
* MAC address index.
*
* @return
* 0 on success, a negative errno value otherwise
*/
int
mlx5_os_mac_addr_add(struct rte_eth_dev *dev, struct rte_ether_addr *mac,
uint32_t index)
{
struct mlx5_priv *priv = dev->data->dev_private;
net/mlx5: concentrate all device configurations Move all device configure to be performed by mlx5_os_cap_config() function instead of the spawn function. In addition move all relevant fields from mlx5_dev_config structure to mlx5_dev_cap. Signed-off-by: Michael Baum <michaelba@nvidia.com> Acked-by: Matan Azrad <matan@nvidia.com>
2022-02-14 09:35:06 +00:00
const int vf = priv->sh->dev_cap.vf;
net/mlx5: refactor Linux MAC operations Move OS specific MAC operations add, remove, modify VF into file linux/mlx5_os.c. Remove unused function mlx5_get_mac(). Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:11 +00:00
int ret = 0;
if (vf)
ret = mlx5_nl_mac_addr_add(priv->nl_socket_route,
mlx5_ifindex(dev), priv->mac_own,
mac, index);
return ret;
}
/**
* Modify a VF MAC address
*
* @param priv
* Pointer to device private data.
* @param mac_addr
* MAC address to modify into.
* @param iface_idx
* Net device interface index
* @param vf_index
* VF index
*
* @return
* 0 on success, a negative errno value otherwise
*/
int
mlx5_os_vf_mac_addr_modify(struct mlx5_priv *priv,
unsigned int iface_idx,
struct rte_ether_addr *mac_addr,
int vf_index)
{
return mlx5_nl_vf_mac_addr_modify
(priv->nl_socket_route, iface_idx, mac_addr, vf_index);
}
net/mlx5: wrap Linux promiscuous and multicast functions This commit adds Linux implementation of routines mlx5_os_set_promisc() and mlx5_os_set_promisc(). The routines call netlink APIs. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-07-19 10:18:12 +00:00
/**
* Set device promiscuous mode
*
* @param dev
* Pointer to Ethernet device structure.
* @param enable
* 0 - promiscuous is disabled, otherwise - enabled
*
* @return
* 0 on success, a negative error value otherwise
*/
int
mlx5_os_set_promisc(struct rte_eth_dev *dev, int enable)
{
struct mlx5_priv *priv = dev->data->dev_private;
return mlx5_nl_promisc(priv->nl_socket_route,
mlx5_ifindex(dev), !!enable);
}
/**
* Set device promiscuous mode
*
* @param dev
* Pointer to Ethernet device structure.
* @param enable
* 0 - all multicase is disabled, otherwise - enabled
*
* @return
* 0 on success, a negative error value otherwise
*/
int
mlx5_os_set_allmulti(struct rte_eth_dev *dev, int enable)
{
struct mlx5_priv *priv = dev->data->dev_private;
return mlx5_nl_allmulti(priv->nl_socket_route,
mlx5_ifindex(dev), !!enable);
}
net/mlx5: remove netlink dependency in shared code This commit adds Linux implementation of routine mlx5_os_mac_addr_flush as wrapper to Netlink API to avoid direct calls under non-Linux operating systems. Signed-off-by: Ophir Munk <ophirmu@mellanox.com> Acked-by: Matan Azrad <matan@mellanox.com>
2020-08-25 09:31:09 +00:00
/**
* Flush device MAC addresses
*
* @param dev
* Pointer to Ethernet device structure.
*
*/
void
mlx5_os_mac_addr_flush(struct rte_eth_dev *dev)
{
struct mlx5_priv *priv = dev->data->dev_private;
mlx5_nl_mac_addr_flush(priv->nl_socket_route, mlx5_ifindex(dev),
dev->data->mac_addrs,
MLX5_MAX_MAC_ADDRESSES, priv->mac_own);
}
Reference in New Issue Copy Permalink