For i40e scalar Tx path, if implement FAST_FREE_MBUF mode, it means
per-queue all mbufs come from the same mempool and have refcnt = 1.
Thus we can use bulk free of the buffers when mbuf fast free mode is
enabled.
Following are the test results with this patch:
MRR L3FWD Test:
two ports & bi-directional flows & one core
RX API: i40e_recv_pkts_bulk_alloc
TX API: i40e_xmit_pkts_simple
ring_descs_size = 1024;
Ring_I40E_TX_MAX_FREE_SZ = 64;
tx_rs_thresh = I40E_DEFAULT_TX_RSBIT_THRESH = 32;
tx_free_thresh = I40E_DEFAULT_TX_FREE_THRESH = 32;
For scalar path in arm platform with default 'tx_rs_thresh':
In n1sdp, performance is improved by 7.9%;
In thunderx2, performance is improved by 7.6%.
For scalar path in x86 platform with default 'tx_rs_thresh':
performance is improved by 4.7%.
Suggested-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>
The flag use_avx2 and use_avx512 are defined as local variables, they
will not be aware by the secondary process, then wrong data path is
selected. Fix the issue by moving them into struct i40e_adapter.
Fixes: 6ada10deac66 ("net/i40e: remove devarg use-latest-supported-vec")
Fixes: e6a6a138919f ("net/i40e: add AVX512 vector path")
Cc: stable@dpdk.org
Signed-off-by: Dapeng Yu <dapengx.yu@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
The VSI pointer is always valid, so there is no need to judge its
validity.
Fixes: b6583ee40265 ("i40e: full VMDQ pools support")
Cc: stable@dpdk.org
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Introduce i40e_simple_prep_pkts() as the preparation function for
simple Tx data path, as it's for sanity check for simple Tx.
Suggested-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Rename PKT_RX_EIP_CKSUM_BAD to PKT_RX_OUTER_IP_CKSUM_BAD and
deprecate the original name. The new name is better aligned
with existing PKT_RX_OUTER_* flags, which should help reduce
confusion about its use.
Suggested-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Lance Richardson <lance.richardson@broadcom.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
The rte_ethdev_driver.h, rte_ethdev_vdev.h and rte_ethdev_pci.h files are
for drivers only and should be a private to DPDK and not installed.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Steven Webster <steven.webster@windriver.com>
The `data_sz` name is fine, but it looks out of place because nothing
else has "data" prefix in that structure. Rename it to "size", as well
as add more clarity to the comments around each struct member.
Fixes: 6a17919b0e2a ("eal: change power intrinsics API")
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
This config item is not exposed anymore now that we removed make
support.
Note: all architectures provide vectorised functions.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
The jumbo frame uses the 'RTE_ETHER_MAX_LEN' as boundary condition,
but the Ether overhead is larger than 18 when it supports dual VLAN tags.
That will cause the jumbo flag rx offload is wrong when MTU size is
'RTE_ETHER_MTU'.
This fix will change the boundary condition with 'RTE_ETHER_MTU' and
overhead, that perhaps impacts the cases of the jumbo frame related.
Fixes: c1715402df8f ("i40evf: fix jumbo frame support")
Fixes: 43e5488c0ac6 ("net/i40e: support MTU configuration")
Fixes: a778a1fa2e4e ("i40e: set up and initialize flow director")
Fixes: c3ac7c5b0b8a ("net/i40e: convert to new Rx offloads API")
Cc: stable@dpdk.org
Signed-off-by: Steve Yang <stevex.yang@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
Optimize Tx path by using AVX512 instructions and vectorize the
tx free bufs process.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Add AVX512 support for i40e PMD. This patch adds i40e_rxtx_vec_avx512.c
to support i40e AVX512 vPMD.
This patch aims to enable AVX512 on i40e vPMD. Main changes are focus
on Rx path compared with AVX2 vPMD.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
As eal parameter --force-max-simd-bitwidth is already introduced,
to make it more clear when setting rx/tx function, remove
devarg use-latest-supported-vec support.
Signed-off-by: Leyi Rong <leyi.rong@intel.com>
Acked-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Implement support for the power management API by implementing a
`get_monitor_addr` function that will return an address of an RX ring's
status bit.
Signed-off-by: Liang Ma <liang.j.ma@intel.com>
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Jeff Guo <jia.guo@intel.com>
When choosing a vector path to take, an extra condition must be
satisfied to ensure the max SIMD bitwidth allows for the CPU enabled
path.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Performance improvement: use a write combining store
instead of a regular mmio write to update queue tail
registers.
Signed-off-by: Radu Nicolau <radu.nicolau@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Wenzhuo Lu <wenzhuo.lu@intel.com>
Since the 20.08 release deprecated rte_cio_*mb APIs because these APIs
provide the same functionality as rte_io_*mb APIs on all platforms, so
remove them and use rte_io_*mb instead.
Signed-off-by: Phil Yang <phil.yang@arm.com>
Signed-off-by: Joyce Kong <joyce.kong@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: David Marchand <david.marchand@redhat.com>
Make is not supported for compiling DPDK, the config files are no
longer needed.
Signed-off-by: Ciara Power <ciara.power@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
This patch optimized the fdir update rate for i40e PF, by tracking
whether the fdir rule being inserted into the guaranteed space
or shared space.
For the flows that are inserted to the guaranteed space, we assume
that the insertion will always succeed as the hardware only report
the "no enough space left" error. In this case, the software can
directly return success and no need to retrieve the result from
the hardware. When destroying a flow, we also assume the operation
will succeed as the software has checked the flow is indeed in
the hardware.
See the fdir programming status descriptor format in the datasheet
for more details.
Signed-off-by: Chenmin Sun <chenmin.sun@intel.com>
Reviewed-by: Jingjing Wu <jingjing.wu@intel.com>
This patch introduces a FDIR flow management for guaranteed/shared
space tracking.
The fdir space is reported by the
i40e_hw_capabilities.fd_filters_guaranteed and fd_filters_best_effort.
The fdir space is managed by hardware and now is tracking in software.
The management algorithm is controlled by the GLQF_CTL.INVALPRIO.
Detailed implementation please check in the datasheet and the
description of struct i40e_fdir_info.fdir_invalprio.
This patch changes the global register GLQF_CTL. Therefore, when devarg
``support-multi-driver`` is set, the patch will not take effect to
avoid affecting the normal behavior of other i40e drivers, e.g., Linux
kernel driver.
Signed-off-by: Chenmin Sun <chenmin.sun@intel.com>
Reviewed-by: Jingjing Wu <jingjing.wu@intel.com>
There should have different behavior in queue start fail and stop fail
case. When queue start fail, all the next actions should be terminated
and then started queues should be cleared. But for queue stop stage, one
queue stop fail should not end other queues stop. This patch fixed that
issue in PF and VF.
Fixes: b6583ee40265 ("i40e: full VMDQ pools support")
Fixes: 3f6a696f1054 ("i40evf: queue start and stop")
Cc: stable@dpdk.org
Signed-off-by: Qiming Yang <qiming.yang@intel.com>
Acked-by: Xiaolong Ye <xiaolong.ye@intel.com>
The new macro __rte_cold, for compiler hinting,
is now used where appropriate for consistency.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: David Christensen <drc@linux.vnet.ibm.com>
Coverity complains of unchecked return value warning of
i40e_xmit_cleanup, while this cleanup is opportunistic and will not
cause problems if it fails. So instead of checking the return value of
i40e_xmit_cleanup and return in case of cleanup failure, we directly
cast it to void function to make the Coverity happy.
Coverity issue: 353617
Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Xiaolong Ye <xiaolong.ye@intel.com>
Add support to the i40e driver for the API rte_eth_tx_done_cleanup
to force free consumed buffers on Tx ring.
Signed-off-by: Chenxu Di <chenxux.di@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Hardware limits that max buffer size per tx descriptor should be
(16K-1)B. So when TSO enabled, the mbuf data size may exceed the
limit and cause malicious behavior to the NIC. This patch fixes
this issue by using more tx descs for this kind of large buffer.
Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Ciara Loftus <ciara.loftus@intel.com>
Change the type of burst mode information from bit field to free string
data, so that each PMD can describe the Rx/Tx busrt functions flexibly.
Fixes: eb5902504a13 ("ethdev: add API for getting burst mode information")
Fixes: 6b6609f68ccd ("net/i40e: support Rx/Tx burst mode info")
Fixes: e9a10e6c2102 ("net/ice: support Rx/Tx burst mode info")
Fixes: 7fe108edcf53 ("app/testpmd: show Rx/Tx burst mode description")
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Ray Kinsella <ray.kinsella@intel.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
This commit adds an implementation to the SSE vector implementation of
RX routine and moves some common defines from a c file to the header
file.
I40e can have 16 and 32 byte descriptors, and the Flow
Director ID data and indication-bit are in different locations
for each size descriptor. The support is implemented in two
separate functions as they require vastly different operations.
The 16B descriptor re-purposes the "filter-status" u32 field
to indicate FDIR ID when the FLM bit is set. No extra loads
are required, however we do have to store to mbuf->fdir.hi,
which is not stored to in the RX path before this patch.
The 32B descriptor requires loading the 2nd 16 bytes of each
descriptor, to get the FLEXBH_STAT and FD Filter ID from
qword3. The resulting data must also be stored to mbuf->fdir.hi,
same as the 16B code path.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tested-by: Mesut Ali Ergin <mesut.a.ergin@intel.com>
When configuring i40e rx queue, the temporary variable to store max
packet length is not big enough which leads to integer overflow issue.
This patch fixes the issue by removing the variable and using the
expression directly since the variable is only used once.
Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Xiao Zhang <xiao.zhang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
Retrieve burst mode options according to the selected Rx/Tx burst
function name.
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Reviewed-by: Xiaolong Ye <xiaolong.ye@intel.com>
To guarantee the orderings of successive stores to CIO and MMIO memory,
a lighter weight rte_io_wmb [1] can be used instead of rte_wmb, and since
the I40E_PCI_REG_WRITE API already has an inclusive rte_io_wmb, this
explicit call can be even saved.
[1] http://git.dpdk.org/dpdk/tree/lib/librte_eal/common/include/generic/
rte_atomic.h#n98
Signed-off-by: Gavin Hu <gavin.hu@arm.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
This patch fixes X722 VF problem when received packet don't have
HASH value.
1) Packet classifier types update should support X722 VF, not only
for X722 PF;
2) MAC type is invalid for X722 VF when set packet classifier type,
so move it after MAC type is set correctly;
Fixes: a286ebeb0714 ("net/i40e: add dynamic mapping of SW flow types to HW pctypes")
Cc: stable@dpdk.org
Signed-off-by: Peng Huang <peng.huang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
The default policy for offload-specific fields is that
they are undefined unless the corresponding offloads are
requested in mbuf ol_flags. This is also the case for outer
L2 and L3 length fields which must not be assumed to contain
zeros for non-tunnel packets. The patch clarifies this behaviour
in the comments and also adds appropriate checks to the PMDs which
do not check any tunnel-related offloads before using the said fields.
Signed-off-by: Ivan Malov <ivan.malov@oktetlabs.ru>
Reviewed-by: Andrew Rybchenko <arybchenko@solarflare.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Use of weak symbols can hide makefile errors especially when
custom makefiles are used. Removing the use of weak symbols
to avoid a stub function being linked in production code.
Signed-off-by: David Harton <dharton@cisco.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Add 'RTE_' prefix to defines:
- rename ETHER_ADDR_LEN as RTE_ETHER_ADDR_LEN.
- rename ETHER_TYPE_LEN as RTE_ETHER_TYPE_LEN.
- rename ETHER_CRC_LEN as RTE_ETHER_CRC_LEN.
- rename ETHER_HDR_LEN as RTE_ETHER_HDR_LEN.
- rename ETHER_MIN_LEN as RTE_ETHER_MIN_LEN.
- rename ETHER_MAX_LEN as RTE_ETHER_MAX_LEN.
- rename ETHER_MTU as RTE_ETHER_MTU.
- rename ETHER_MAX_VLAN_FRAME_LEN as RTE_ETHER_MAX_VLAN_FRAME_LEN.
- rename ETHER_MAX_VLAN_ID as RTE_ETHER_MAX_VLAN_ID.
- rename ETHER_MAX_JUMBO_FRAME_LEN as RTE_ETHER_MAX_JUMBO_FRAME_LEN.
- rename ETHER_MIN_MTU as RTE_ETHER_MIN_MTU.
- rename ETHER_LOCAL_ADMIN_ADDR as RTE_ETHER_LOCAL_ADMIN_ADDR.
- rename ETHER_GROUP_ADDR as RTE_ETHER_GROUP_ADDR.
- rename ETHER_TYPE_IPv4 as RTE_ETHER_TYPE_IPv4.
- rename ETHER_TYPE_IPv6 as RTE_ETHER_TYPE_IPv6.
- rename ETHER_TYPE_ARP as RTE_ETHER_TYPE_ARP.
- rename ETHER_TYPE_VLAN as RTE_ETHER_TYPE_VLAN.
- rename ETHER_TYPE_RARP as RTE_ETHER_TYPE_RARP.
- rename ETHER_TYPE_QINQ as RTE_ETHER_TYPE_QINQ.
- rename ETHER_TYPE_ETAG as RTE_ETHER_TYPE_ETAG.
- rename ETHER_TYPE_1588 as RTE_ETHER_TYPE_1588.
- rename ETHER_TYPE_SLOW as RTE_ETHER_TYPE_SLOW.
- rename ETHER_TYPE_TEB as RTE_ETHER_TYPE_TEB.
- rename ETHER_TYPE_LLDP as RTE_ETHER_TYPE_LLDP.
- rename ETHER_TYPE_MPLS as RTE_ETHER_TYPE_MPLS.
- rename ETHER_TYPE_MPLSM as RTE_ETHER_TYPE_MPLSM.
- rename ETHER_VXLAN_HLEN as RTE_ETHER_VXLAN_HLEN.
- rename ETHER_ADDR_FMT_SIZE as RTE_ETHER_ADDR_FMT_SIZE.
- rename VXLAN_GPE_TYPE_IPV4 as RTE_VXLAN_GPE_TYPE_IPV4.
- rename VXLAN_GPE_TYPE_IPV6 as RTE_VXLAN_GPE_TYPE_IPV6.
- rename VXLAN_GPE_TYPE_ETH as RTE_VXLAN_GPE_TYPE_ETH.
- rename VXLAN_GPE_TYPE_NSH as RTE_VXLAN_GPE_TYPE_NSH.
- rename VXLAN_GPE_TYPE_MPLS as RTE_VXLAN_GPE_TYPE_MPLS.
- rename VXLAN_GPE_TYPE_GBP as RTE_VXLAN_GPE_TYPE_GBP.
- rename VXLAN_GPE_TYPE_VBNG as RTE_VXLAN_GPE_TYPE_VBNG.
- rename ETHER_VXLAN_GPE_HLEN as RTE_ETHER_VXLAN_GPE_HLEN.
Do not update the command line library to avoid adding a dependency to
librte_net.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
Tx desc's DD status is not cleaned by NIC automatically after packets
have been transmitted until software refill a new packet during next
loop. So when tx_free_thresh + tx_rs_thresh > nb_desc, it is possible
that an outdated DD status be checked as tx_next_dd, then segment fault
happen due to free a NULL mbuf pointer.
Then patch fixes this issue by
1. try to adapt tx_rs_thresh to an aggressive tx_free_thresh.
2. queue setup fail when tx_free_thresh + tx_rs_thresh > nb_desc
Fixes: 4861cde46116 ("i40e: new poll mode driver")
Cc: stable@dpdk.org
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Acked-by: Beilei Xing <beilei.xing@intel.com>
Coverity flags that the txq variable is used before it's checked for NULL.
Also fix typo in error message.
Coverity issue: 195023
Fixes: 24853544c84c ("net/i40e: fix mbuf free in vector Tx")
Cc: stable@dpdk.org
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Rami Rosen <ramirose@gmail.com>
No need to add additional vlan tag size for max packet size, since
for i40e, the queue's Rx Max Frame Size (rxq->max_pkt_len) already
includes the vlan header size.
Fixes: a3c83a2527e1 ("net/i40e: enable runtime queue setup")
Fixes: 4861cde46116 ("i40e: new poll mode driver")
Fixes: c1715402df8f ("i40evf: fix jumbo frame support")
Cc: stable@dpdk.org
Signed-off-by: Qi Zhang <qi.z.zhang@intel.com>
Before this patch, there are two functions will call
i40e_dev_free_queues to free queues. For rte_eth_dev_close(), its
redundant because of duplication. For rte_eth_dev_reset() its
redundant because of not necessary, since following dev_configure
is required after dev_reset and it will be updated correctly.
This patch removes redundant code in i40e_dev_free_queues().
Fixes: 6b4537128394 ("i40e: free queue memory when closing")
Cc: stable@dpdk.org
Signed-off-by: Zhirun Yan <zhirun.yan@intel.com>
Signed-off-by: Haiyue Wang <haiyue.wang@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
This reverts
commit 09a62d75691f ("net/i40e: fix offload not supported mask")
Contrary to what is said in above patch commit log,
I40E_TX_OFFLOAD_NOTSUP_MASK is the mask of Tx offload bits that are part
of PKT_TX_OFFLOAD_MASK but not included in I40E_TX_OFFLOAD_MASK.
Above patch erroneously includes all PKT_RX_OFFLOAD_ bits in the
I40E_TX_OFFLOAD_NOTSUP_MASK, this is not what is expected.
Restore the initial xor that gives the expected result.
Fixes: 09a62d75691f ("net/i40e: fix offload not supported mask")
Cc: stable@dpdk.org
Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Previously, there is instability during vector Rx if descriptor
number is not power of 2, e.g. process hang and some Rx packets
are unexpectedly empty. That's because vector Rx mode assumes Rx
descriptor number is power of 2 when doing bit mask.
This patch allows vector mode only when the number of Rx descriptor
is power of 2.
Fixes: 8e109464c022 ("i40e: allow vector Rx and Tx usage")
Fixes: a3c83a2527e1 ("net/i40e: enable runtime queue setup")
Cc: stable@dpdk.org
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Just as the name I40E_TX_OFFLOAD_NOTSUP_MASK indicates, it should be the
mask of unsupported features (either not in PKT_TX_OFFLOAD_MASK or in
I40E_TX_OFFLOAD_MASK), however, xor will not get desired result here,
assume bit 0 of PKT_TX_OFFLOAD_MASK and I40E_TX_OFFLOAD_MAKS are 0 which
means corresponding feature is not supported in both sides, then we get
value of bit 0 of I40E_TX_OFFLOAD_NOTSUP_MASK which is 0 via xor, it
implies that it is supported which doesn't meet our expectation.
Correct it by a NOT-AND operation.
Fixes: 3f33e643e5c6 ("net/i40e: add Tx preparation")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
Tx offload mask is updated in following commit:
commit 1037ed842c37 ("mbuf: fix Tx offload mask").
Currently, the new added offload flags are not supported in PMD
and application will fail to call PMD transmit prepare function.
This patch updates PMD Tx offload mask.
Cc: stable@dpdk.org
Signed-off-by: Beilei Xing <beilei.xing@intel.com>
Acked-by: Qi Zhang <qi.z.zhang@intel.com>
