d/numam-dpdk
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

net/enic: enable GENEVE offload via VNIC configuration

Browse Source 
The admin-configured vNIC settings (i.e. via CIMC or UCSM) now include
Geneve offload. Use that setting to decide whether to enable or
disable Geneve offload and remove the devarg 'geneve-opt'.

Also, the firmware now allows the driver to change the Geneve port
number. So extend udp_tunnel_port_{add,del} to accept Geneve port, in
addition to VXLAN.

Fixes: 93fb21fdbe23 ("net/enic: enable overlay offload for VXLAN and GENEVE")
Cc: stable@dpdk.org

Signed-off-by: John Daley <johndale@cisco.com>
Reviewed-by: Hyong Youb Kim <hyonkim@cisco.com>
This commit is contained in:
John Daley 2021-05-11 12:25:26 -07:00 committed by Ferruh Yigit
parent 8eff201b00
commit 61c7b522d9
8 changed files with 162 additions and 125 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

32
doc/guides/nics/enic.rst
View File

@ -294,35 +294,31 @@ inner and outer packets can be IPv4 or IPv6.
RSS hash calculation, therefore queue selection, is done on inner packets.
In order to enable overlay offload, the 'Enable VXLAN' box should be checked
In order to enable overlay offload, enable VXLAN and/or Geneve on vNIC
via CIMC or UCSM followed by a reboot of the server. When PMD successfully
enables overlay offload, it prints the following message on the console.
enables overlay offload, it prints one of the following messages on the console.
.. code-block:: console
Overlay offload is enabled
Overlay offload is enabled (VxLAN)
Overlay offload is enabled (Geneve)
Overlay offload is enabled (VxLAN, Geneve)
By default, PMD enables overlay offload if hardware supports it. To disable
it, set ``devargs`` parameter ``disable-overlay=1``. For example::
-a 12:00.0,disable-overlay=1
By default, the NIC uses 4789 as the VXLAN port. The user may change
it through ``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as
the current NIC has a single VXLAN port number, the user cannot
configure multiple port numbers.
Geneve headers with non-zero options are not supported by default. To
use Geneve with options, update the VIC firmware to the latest version
and then set ``devargs`` parameter ``geneve-opt=1``. When Geneve with
options is enabled, flow API cannot be used as the features are
currently mutually exclusive. When this feature is successfully
enabled, PMD prints the following message.
.. code-block:: console
Geneve with options is enabled
By default, the NIC uses 4789 and 6081 as the VXLAN and Geneve ports,
respectively. The user may change them through
``rte_eth_dev_udp_tunnel_port_{add,delete}``. However, as the current
NIC has a single VXLAN port number and a single Geneve port number,
the user cannot configure multiple port numbers for each tunnel type.
Geneve offload support has evolved over VIC models. On older models,
Geneve offload and advanced filters are mutually exclusive. This is
enforced by UCSM and CIMC, which only allow one of the two features
to be selected at one time. Newer VIC models do not have this restriction.
Ingress VLAN Rewrite
--------------------

7
doc/guides/rel_notes/release_20_05.rst
View File

@ -121,6 +121,13 @@ New Features
* Added flow counters to extended stats.
* Added PCI function stats to extended stats.
* **Updated Cisco enic driver.**
Updated Cisco enic driver GENEVE tunneling support:
* Added support to control GENEVE tunneling via UCSM/CIMC and removed devarg.
* Added GENEVE port number configuration.
* **Updated Hisilicon hns3 driver.**
Updated Hisilicon hns3 driver with new features and improvements, including:

2
drivers/net/enic/base/vnic_dev.c
View File

@ -1318,7 +1318,7 @@ int vnic_dev_capable_geneve(struct vnic_dev *vdev)
int ret;
ret = vnic_dev_cmd(vdev, CMD_GET_SUPP_FEATURE_VER, &a0, &a1, wait);
return ret == 0 && (a1 & FEATURE_GENEVE_OPTIONS);
return ret == 0 && !!(a1 & FEATURE_GENEVE_OPTIONS);
}
uint64_t vnic_dev_capable_cq_entry_size(struct vnic_dev *vdev)

1
drivers/net/enic/base/vnic_enet.h
View File

@ -55,6 +55,7 @@ struct vnic_enet_config {
#define VENETF_NICSWITCH 0x80000 /* NICSWITCH enabled */
#define VENETF_RSSHASH_UDPIPV4 0x100000 /* Hash on UDP + IPv4 fields */
#define VENETF_RSSHASH_UDPIPV6 0x200000 /* Hash on UDP + IPv6 fields */
#define VENETF_GENEVE 0x400000 /* GENEVE offload */
#define VENET_INTR_TYPE_MIN 0 /* Timer specs min interrupt spacing */
#define VENET_INTR_TYPE_IDLE 1 /* Timer specs idle time before irq */

4
drivers/net/enic/enic.h
View File

@ -118,17 +118,17 @@ struct enic {
uint32_t flow_filter_mode;
uint8_t filter_actions; /* HW supported actions */
uint64_t cq_entry_sizes; /* supported CQ entry sizes */
bool geneve;
bool vxlan;
bool cq64; /* actually using 64B CQ entry */
bool cq64_request; /* devargs cq64=1 */
bool disable_overlay; /* devargs disable_overlay=1 */
uint8_t enable_avx2_rx; /* devargs enable-avx2-rx=1 */
uint8_t geneve_opt_avail; /* Geneve with options offload available */
uint8_t geneve_opt_enabled; /* Geneve with options offload enabled */
uint8_t geneve_opt_request; /* devargs geneve-opt=1 */
bool nic_cfg_chk; /* NIC_CFG_CHK available */
bool udp_rss_weak; /* Bodega style UDP RSS */
uint8_t ig_vlan_rewrite_mode; /* devargs ig-vlan-rewrite */
uint16_t geneve_port; /* current geneve port pushed to NIC */
uint16_t vxlan_port; /* current vxlan port pushed to NIC */
int use_simple_tx_handler;
int use_noscatter_vec_rx_handler;

71
drivers/net/enic/enic_ethdev.c
View File

@ -11,6 +11,7 @@
#include <rte_bus_pci.h>
#include <ethdev_driver.h>
#include <ethdev_pci.h>
#include <rte_geneve.h>
#include <rte_kvargs.h>
#include <rte_string_fns.h>
@ -67,7 +68,6 @@ static const struct vic_speed_capa {
#define ENIC_DEVARG_CQ64 "cq64"
#define ENIC_DEVARG_DISABLE_OVERLAY "disable-overlay"
#define ENIC_DEVARG_ENABLE_AVX2_RX "enable-avx2-rx"
#define ENIC_DEVARG_GENEVE_OPT "geneve-opt"
#define ENIC_DEVARG_IG_VLAN_REWRITE "ig-vlan-rewrite"
#define ENIC_DEVARG_REPRESENTOR "representor"
@ -81,13 +81,6 @@ enicpmd_dev_flow_ops_get(struct rte_eth_dev *dev,
ENICPMD_FUNC_TRACE();
/*
* Currently, when Geneve with options offload is enabled, host
* cannot insert match-action rules.
*/
if (enic->geneve_opt_enabled)
return -ENOTSUP;
if (enic->flow_filter_mode == FILTER_FLOWMAN)
*ops = &enic_fm_flow_ops;
else
@ -972,26 +965,32 @@ static int enicpmd_dev_rx_queue_intr_disable(struct rte_eth_dev *eth_dev,
static int udp_tunnel_common_check(struct enic *enic,
struct rte_eth_udp_tunnel *tnl)
{
if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN)
if (tnl->prot_type != RTE_TUNNEL_TYPE_VXLAN &&
tnl->prot_type != RTE_TUNNEL_TYPE_GENEVE)
return -ENOTSUP;
if (!enic->overlay_offload) {
ENICPMD_LOG(DEBUG, " vxlan (overlay offload) is not "
"supported\n");
ENICPMD_LOG(DEBUG, " overlay offload is not supported\n");
return -ENOTSUP;
}
return 0;
}
static int update_vxlan_port(struct enic *enic, uint16_t port)
static int update_tunnel_port(struct enic *enic, uint16_t port, bool vxlan)
{
if (vnic_dev_overlay_offload_cfg(enic->vdev,
OVERLAY_CFG_VXLAN_PORT_UPDATE,
port)) {
ENICPMD_LOG(DEBUG, " failed to update vxlan port\n");
uint8_t cfg;
cfg = vxlan ? OVERLAY_CFG_VXLAN_PORT_UPDATE :
OVERLAY_CFG_GENEVE_PORT_UPDATE;
if (vnic_dev_overlay_offload_cfg(enic->vdev, cfg, port)) {
ENICPMD_LOG(DEBUG, " failed to update tunnel port\n");
return -EINVAL;
}
ENICPMD_LOG(DEBUG, " updated vxlan port to %u\n", port);
enic->vxlan_port = port;
ENICPMD_LOG(DEBUG, " updated %s port to %u\n",
vxlan ? "vxlan" : "geneve", port);
if (vxlan)
enic->vxlan_port = port;
else
enic->geneve_port = port;
return 0;
}
@ -999,34 +998,48 @@ static int enicpmd_dev_udp_tunnel_port_add(struct rte_eth_dev *eth_dev,
struct rte_eth_udp_tunnel *tnl)
{
struct enic *enic = pmd_priv(eth_dev);
uint16_t port;
bool vxlan;
int ret;
ENICPMD_FUNC_TRACE();
ret = udp_tunnel_common_check(enic, tnl);
if (ret)
return ret;
vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
if (vxlan)
port = enic->vxlan_port;
else
port = enic->geneve_port;
/*
* The NIC has 1 configurable VXLAN port number. "Adding" a new port
* number replaces it.
* The NIC has 1 configurable port number per tunnel type.
* "Adding" a new port number replaces it.
*/
if (tnl->udp_port == enic->vxlan_port || tnl->udp_port == 0) {
if (tnl->udp_port == port || tnl->udp_port == 0) {
ENICPMD_LOG(DEBUG, " %u is already configured or invalid\n",
tnl->udp_port);
return -EINVAL;
}
return update_vxlan_port(enic, tnl->udp_port);
return update_tunnel_port(enic, tnl->udp_port, vxlan);
}
static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
struct rte_eth_udp_tunnel *tnl)
{
struct enic *enic = pmd_priv(eth_dev);
uint16_t port;
bool vxlan;
int ret;
ENICPMD_FUNC_TRACE();
ret = udp_tunnel_common_check(enic, tnl);
if (ret)
return ret;
vxlan = (tnl->prot_type == RTE_TUNNEL_TYPE_VXLAN);
if (vxlan)
port = enic->vxlan_port;
else
port = enic->geneve_port;
/*
* Clear the previously set port number and restore the
* hardware default port number. Some drivers disable VXLAN
@ -1034,12 +1047,13 @@ static int enicpmd_dev_udp_tunnel_port_del(struct rte_eth_dev *eth_dev,
* enic does not do that as VXLAN is part of overlay offload,
* which is tied to inner RSS and TSO.
*/
if (tnl->udp_port != enic->vxlan_port) {
ENICPMD_LOG(DEBUG, " %u is not a configured vxlan port\n",
if (tnl->udp_port != port) {
ENICPMD_LOG(DEBUG, " %u is not a configured tunnel port\n",
tnl->udp_port);
return -EINVAL;
}
return update_vxlan_port(enic, RTE_VXLAN_DEFAULT_PORT);
port = vxlan ? RTE_VXLAN_DEFAULT_PORT : RTE_GENEVE_DEFAULT_PORT;
return update_tunnel_port(enic, port, vxlan);
}
static int enicpmd_dev_fw_version_get(struct rte_eth_dev *eth_dev,
@ -1145,8 +1159,6 @@ static int enic_parse_zero_one(const char *key,
enic->disable_overlay = b;
if (strcmp(key, ENIC_DEVARG_ENABLE_AVX2_RX) == 0)
enic->enable_avx2_rx = b;
if (strcmp(key, ENIC_DEVARG_GENEVE_OPT) == 0)
enic->geneve_opt_request = b;
return 0;
}
@ -1189,7 +1201,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
ENIC_DEVARG_CQ64,
ENIC_DEVARG_DISABLE_OVERLAY,
ENIC_DEVARG_ENABLE_AVX2_RX,
ENIC_DEVARG_GENEVE_OPT,
ENIC_DEVARG_IG_VLAN_REWRITE,
ENIC_DEVARG_REPRESENTOR,
NULL};
@ -1201,7 +1212,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
enic->cq64_request = true; /* Use 64B entry if available */
enic->disable_overlay = false;
enic->enable_avx2_rx = false;
enic->geneve_opt_request = false;
enic->ig_vlan_rewrite_mode = IG_VLAN_REWRITE_MODE_PASS_THRU;
if (!dev->device->devargs)
return 0;
@ -1214,8 +1224,6 @@ static int enic_check_devargs(struct rte_eth_dev *dev)
enic_parse_zero_one, enic) < 0 ||
rte_kvargs_process(kvlist, ENIC_DEVARG_ENABLE_AVX2_RX,
enic_parse_zero_one, enic) < 0 ||
rte_kvargs_process(kvlist, ENIC_DEVARG_GENEVE_OPT,
enic_parse_zero_one, enic) < 0 ||
rte_kvargs_process(kvlist, ENIC_DEVARG_IG_VLAN_REWRITE,
enic_parse_ig_vlan_rewrite, enic) < 0) {
rte_kvargs_free(kvlist);
@ -1391,5 +1399,4 @@ RTE_PMD_REGISTER_PARAM_STRING(net_enic,
ENIC_DEVARG_CQ64 "=0|1"
ENIC_DEVARG_DISABLE_OVERLAY "=0|1 "
ENIC_DEVARG_ENABLE_AVX2_RX "=0|1 "
ENIC_DEVARG_GENEVE_OPT "=0|1 "
ENIC_DEVARG_IG_VLAN_REWRITE "=trunk|untag|priority|pass");

163
drivers/net/enic/enic_main.c
View File

@ -16,6 +16,7 @@
#include <rte_mbuf.h>
#include <rte_string_fns.h>
#include <ethdev_driver.h>
#include <rte_geneve.h>
#include "enic_compat.h"
#include "enic.h"
@ -1719,6 +1720,85 @@ set_mtu_done:
return rc;
}
static void
enic_disable_overlay_offload(struct enic *enic)
{
/*
* Disabling fails if the feature is provisioned but
* not enabled. So ignore result and do not log error.
*/
if (enic->vxlan) {
vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_DISABLE);
}
if (enic->geneve) {
vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_DISABLE);
}
}
static int
enic_enable_overlay_offload(struct enic *enic)
{
if (enic->vxlan && vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_VXLAN, OVERLAY_OFFLOAD_ENABLE) != 0) {
dev_err(NULL, "failed to enable VXLAN offload\n");
return -EINVAL;
}
if (enic->geneve && vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_GENEVE, OVERLAY_OFFLOAD_ENABLE) != 0) {
dev_err(NULL, "failed to enable Geneve offload\n");
return -EINVAL;
}
enic->tx_offload_capa |=
DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
(enic->geneve ? DEV_TX_OFFLOAD_GENEVE_TNL_TSO : 0) |
(enic->vxlan ? DEV_TX_OFFLOAD_VXLAN_TNL_TSO : 0);
enic->tx_offload_mask |=
PKT_TX_OUTER_IPV6 |
PKT_TX_OUTER_IPV4 |
PKT_TX_OUTER_IP_CKSUM |
PKT_TX_TUNNEL_MASK;
enic->overlay_offload = true;
if (enic->vxlan && enic->geneve)
dev_info(NULL, "Overlay offload is enabled (VxLAN, Geneve)\n");
else if (enic->vxlan)
dev_info(NULL, "Overlay offload is enabled (VxLAN)\n");
else
dev_info(NULL, "Overlay offload is enabled (Geneve)\n");
return 0;
}
static int
enic_reset_overlay_port(struct enic *enic)
{
if (enic->vxlan) {
enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
/*
* Reset the vxlan port to the default, as the NIC firmware
* does not reset it automatically and keeps the old setting.
*/
if (vnic_dev_overlay_offload_cfg(enic->vdev,
OVERLAY_CFG_VXLAN_PORT_UPDATE,
RTE_VXLAN_DEFAULT_PORT)) {
dev_err(enic, "failed to update vxlan port\n");
return -EINVAL;
}
}
if (enic->geneve) {
enic->geneve_port = RTE_GENEVE_DEFAULT_PORT;
if (vnic_dev_overlay_offload_cfg(enic->vdev,
OVERLAY_CFG_GENEVE_PORT_UPDATE,
RTE_GENEVE_DEFAULT_PORT)) {
dev_err(enic, "failed to update vxlan port\n");
return -EINVAL;
}
}
return 0;
}
static int enic_dev_init(struct enic *enic)
{
int err;
@ -1785,85 +1865,32 @@ static int enic_dev_init(struct enic *enic)
/* set up link status checking */
vnic_dev_notify_set(enic->vdev, -1); /* No Intr for notify */
/*
* When Geneve with options offload is available, always disable it
* first as it can interfere with user flow rules.
*/
if (enic->geneve_opt_avail) {
/*
* Disabling fails if the feature is provisioned but
* not enabled. So ignore result and do not log error.
*/
vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_GENEVE,
OVERLAY_OFFLOAD_DISABLE);
}
enic->overlay_offload = false;
if (enic->disable_overlay && enic->vxlan) {
/*
* Explicitly disable overlay offload as the setting is
* sticky, and resetting vNIC does not disable it.
*/
if (vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_VXLAN,
OVERLAY_OFFLOAD_DISABLE)) {
dev_err(enic, "failed to disable overlay offload\n");
} else {
dev_info(enic, "Overlay offload is disabled\n");
}
}
if (!enic->disable_overlay && enic->vxlan &&
/* 'VXLAN feature' enables VXLAN, NVGRE, and GENEVE. */
vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_VXLAN,
OVERLAY_OFFLOAD_ENABLE) == 0) {
enic->tx_offload_capa |=
DEV_TX_OFFLOAD_OUTER_IPV4_CKSUM |
DEV_TX_OFFLOAD_GENEVE_TNL_TSO |
DEV_TX_OFFLOAD_VXLAN_TNL_TSO;
enic->tx_offload_mask |=
PKT_TX_OUTER_IPV6 |
PKT_TX_OUTER_IPV4 |
PKT_TX_OUTER_IP_CKSUM |
PKT_TX_TUNNEL_MASK;
enic->overlay_offload = true;
dev_info(enic, "Overlay offload is enabled\n");
}
/* Geneve with options offload requires overlay offload */
if (enic->overlay_offload && enic->geneve_opt_avail &&
enic->geneve_opt_request) {
if (vnic_dev_overlay_offload_ctrl(enic->vdev,
OVERLAY_FEATURE_GENEVE,
OVERLAY_OFFLOAD_ENABLE)) {
dev_err(enic, "failed to enable geneve+option\n");
} else {
enic->geneve_opt_enabled = 1;
dev_info(enic, "Geneve with options is enabled\n");
/*
* First, explicitly disable overlay offload as the setting is
* sticky, and resetting vNIC may not disable it.
*/
enic_disable_overlay_offload(enic);
/* Then, enable overlay offload according to vNIC flags */
if (!enic->disable_overlay && (enic->vxlan || enic->geneve)) {
err = enic_enable_overlay_offload(enic);
if (err) {
dev_info(NULL, "failed to enable overlay offload\n");
return err;
}
}
/*
* Reset the vxlan port if HW vxlan parsing is available. It
* Reset the vxlan/geneve port if HW parsing is available. It
* is always enabled regardless of overlay offload
* enable/disable.
*/
if (enic->vxlan) {
enic->vxlan_port = RTE_VXLAN_DEFAULT_PORT;
/*
* Reset the vxlan port to the default, as the NIC firmware
* does not reset it automatically and keeps the old setting.
*/
if (vnic_dev_overlay_offload_cfg(enic->vdev,
OVERLAY_CFG_VXLAN_PORT_UPDATE,
RTE_VXLAN_DEFAULT_PORT)) {
dev_err(enic, "failed to update vxlan port\n");
return -EINVAL;
}
}
err = enic_reset_overlay_port(enic);
if (err)
return err;
if (enic_fm_init(enic))
dev_warning(enic, "Init of flowman failed.\n");
return 0;
}
static void lock_devcmd(void *priv)

7
drivers/net/enic/enic_res.c
View File

@ -179,10 +179,9 @@ int enic_get_vnic_config(struct enic *enic)
enic->vxlan = ENIC_SETTING(enic, VXLAN) &&
vnic_dev_capable_vxlan(enic->vdev);
if (vnic_dev_capable_geneve(enic->vdev)) {
dev_info(NULL, "Geneve with options offload available\n");
enic->geneve_opt_avail = 1;
}
enic->geneve = ENIC_SETTING(enic, GENEVE) &&
vnic_dev_capable_geneve(enic->vdev);
/* Supported CQ entry sizes */
enic->cq_entry_sizes = vnic_dev_capable_cq_entry_size(enic->vdev);
sizes = enic->cq_entry_sizes;