When bulk alloc is enabled at compile time but preconditions for
it are not met at runtime the ixgbe_reset_rx_queue() function
overrides rxq->sw_ring not allocated elements.
Fixes: 01fa1d6215 ("ixgbe: unify Rx setup")
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
If RTE_LIBRTE_IXGBE_RX_ALLOW_BULK_ALLOC is disabled in the config file,
RTE_PMD_IXGBE_RX_MAX_BURST macro and ixgbe_recv_pkts_bulk_alloc function
are not declared, and some parts of the ixgbe code were still trying to
use them.
Fixes: 01fa1d6215 ("ixgbe: unify Rx setup")
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
- Set the callback in a single function that is called from
ixgbe_dev_rx_init() for a primary process and from eth_ixgbe_dev_init()
for a secondary processes. This is instead of multiple, hard to track places.
- Added ixgbe_hw.rx_bulk_alloc_allowed - see ixgbe_hw.rx_vec_allowed description below.
- Added ixgbe_hw.rx_vec_allowed: like with Bulk Allocation, Vector Rx is
enabled or disabled on a per-port level. All queues have to meet the appropriate
preconditions and if any of them doesn't - the feature has to be disabled.
Therefore ixgbe_hw.rx_vec_allowed will be updated during each queues configuration
(rte_eth_rx_queue_setup()) and then used in rte_eth_dev_start() to configure the
appropriate callbacks. The same happens with ixgbe_hw.rx_vec_allowed in a Bulk Allocation
context.
- Bugs fixed:
- Vector scattered packets callback was called regardless the appropriate
preconditions:
- Vector Rx specific preconditions.
- Bulk Allocation preconditions.
- Vector Rx was enabled/disabled according to the last queue setting and not
based on all queues setting (which may be different for each queue).
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
According to x540 spec chapter 8.2.4.8.9 CRCSTRIP field of RDRXCTL should
be configured to the same value as HLREG0.RXCRCSTRP.
Clearing the RDRXCTL.RSCFRSTSIZE field for x540 is not required by the spec
but seems harmless.
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Use the rte_le_to_cpu_xx()/rte_cpu_to_le_xx() when reading/setting HW ring
descriptor fields.
Fixed the above in ixgbe_rx_alloc_bufs() and in ixgbe_recv_scattered_pkts().
Signed-off-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
To avoid any possible confusion or breakage, rename all the structures
of ixgbe driver to use ixgbe_ rather than igb_ because igb is a
different driver.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Changchun Ouyang <changchun.ouyang@intel.com>
All global functions in a driver should use the same prefix
to avoid any future name collisions.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Reviewed-by: Bruce Richardson <bruce.richardson@intel.com>
[Bruce: one instance was missing]
Acked-by: Changchun Ouyang <changchun.ouyang@intel.com>
Only used in this file, make it static.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Changchun Ouyang <changchun.ouyang@intel.com>
These are const data structures, just put them in txt segment
rather than having compiler emit code to set them up on the stack.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Changchun Ouyang <changchun.ouyang@intel.com>
All virtual function tables should be const so they are put
in text segment rather than data.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Changchun Ouyang <changchun.ouyang@intel.com>
X550 should use the correct macro to set the VFTDT and VFRDT register address.
This patch fixes the VF TX issue for Sageville.
Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
Acked-by: Cunming Liang <cunming.liang@intel.com>
The mbuf leak happens when the assigned number of rx descriptor is not
power of 2 in vector mode.
As it's presumed on vpmd rx (for rx_tail wrap), adding condition check
to prevent it.
The root cause reference code in *_recv_raw_pkts_vec* as below.
"rxq->rx_tail = (uint16_t)(rxq->rx_tail & (rxq->nb_rx_desc - 1));".
Reported-by: Stephen Hemminger <stephen@networkplumber.org>
Signed-off-by: Cunming Liang <cunming.liang@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
RSS offload types were defined separately for 1/10G and 40G NICs,
and have no relationship with flow types. The modifications are to
unify all RSS offload types for all PMDs. Unified RSS offload types
have new and common names which can be used for any PMD or
applications, and decouple from specific hardwares.
Signed-off-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
[Thomas: merge with fm10k]
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
For ixgbe, when queue start fails, for example, mbuf allocate
failure, the device will still start successfully, which could be
an issue.
Add return status check of queue start to avoid this issue.
Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
It needs config RSS and IXGBE_MRQC and IXGBE_VFPSRTYPE to enable VF RSS.
The psrtype will determine how many queues the received packets will distribute to,
and the value of psrtype should depends on both facet: max VF rxq number which
has been negotiated with PF, and the number of rxq specified in config on guest.
Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
Reviewed-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Put global register configuring out of loop for queue; also fix typo and indent.
Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
Reviewed-by: Vlad Zolotarov <vladz@cloudius-systems.com>
Add missing setup for X540 MAC type when setting up VF.
Additional check exists in Linux driver but not in DPDK.
Signed-off-by: Bill Hong <bhong@brocade.com>
Signed-off-by: Stephen Hemminger <shemming@brocade.com>
When using multiple processes, the TX function used in all processes
should be the same, otherwise the secondary processes cannot transmit
more than tx-ring-size - 1 packets.
To achieve this, we extract out the code to select the ixgbe TX function
to be used into a separate function inside the ixgbe driver, and call
that from a secondary process when it is attaching to an
already-configured NIC.
Testing with symmetric MP app shows that we are able to RX and TX from
both primary and secondary processes once this patch is applied.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Reshma Pattan <reshma.pattan@intel.com>
Commit 1224decaa4 ("support TCP segmentation offload")
changed the way the bitfields are assigned in ixgbe, example:
tx_offload_mask.l2_len = ~0;
This result in a compilation error with clang:
error: implicit truncation from 'int' to bitfield
changes value from -1 to 127 [-Werror,-Wbitfield-constant-conversion]
Replacing the '=' with a '|=' fixes the issue.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
CACHE_LINE_SIZE is a macro defined in machine/param.h in FreeBSD and
conflicts with DPDK macro version.
Adding RTE_ prefix to avoid conflicts.
CACHE_LINE_MASK and CACHE_LINE_ROUNDUP are also prefixed.
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
[Thomas: updated on HEAD, including PPC]
Implement TSO (TCP segmentation offload) in ixgbe driver. The driver is
now able to use PKT_TX_TCP_SEG mbuf flag and mbuf hardware offload infos
(l2_len, l3_len, l4_len, tso_segsz) to configure the hardware support of
TCP segmentation.
In ixgbe, when doing TSO, the IP length must not be included in the TCP
pseudo header checksum. A new function ixgbe_fix_tcp_phdr_cksum() is
used to fix the pseudo header checksum of the packet before giving it to
the hardware.
In the patch, the tx_desc_cksum_flags_to_olinfo() and
tx_desc_ol_flags_to_cmdtype() functions have been reworked to make them
clearer. This should not impact performance as gcc (version 4.8 in my
case) is smart enough to convert the tests into a code that does not
contain any branch instruction.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Some of the NICs supported by DPDK have a possibility to accelerate TCP
traffic by using segmentation offload. The application prepares a packet
with valid TCP header with size up to 64K and deleguates the
segmentation to the NIC.
Implement the generic part of TCP segmentation offload in rte_mbuf. It
introduces 2 new fields in rte_mbuf: l4_len (length of L4 header in bytes)
and tso_segsz (MSS of packets).
To delegate the TCP segmentation to the hardware, the user has to:
- set the PKT_TX_TCP_SEG flag in mbuf->ol_flags (this flag implies
PKT_TX_TCP_CKSUM)
- set the flag PKT_TX_IPV4 or PKT_TX_IPV6
- set PKT_TX_IP_CKSUM if it's IPv4, and set the IP checksum to 0 in
the packet
- fill the mbuf offload information: l2_len, l3_len, l4_len, tso_segsz
- calculate the pseudo header checksum without taking ip_len in account,
and set it in the TCP header, for instance by using
rte_ipv4_phdr_cksum(ip_hdr, ol_flags)
The API is inspired from ixgbe hardware (the next commit adds the
support for ixgbe), but it seems generic enough to be used for other
hw/drivers in the future.
This commit also reworks the way l2_len and l3_len are used in igb
and ixgbe drivers as the l2_l3_len is not available anymore in mbuf.
Signed-off-by: Mirek Walukiewicz <miroslaw.walukiewicz@intel.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
This definition is specific to Intel PMD drivers and its definition
"indicate what bits required for building TX context" shows that it
should not be in the generic rte_mbuf.h but in the PMD driver.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Since commit 4332beee9 "mbuf: expand ol_flags field to 64-bits", the
packet flags are now 64 bits wide. Some occurences were forgotten in
the ixgbe driver.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
According to Intel® 82599 10 GbE Controller Datasheet (Table 7-38), both
L2 and L3 lengths are needed to offload the IP checksum.
Note that the e1000 driver does not need to be patched as it already
contains the fix.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Config PFVML2FLT register in ixgbe PMD to enable it receive broadcast and multicast packets;
also factorize the common logic with ixgbe_set_pool_rx_mode.
Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
If the vector pmd option is turned on in the compile time config file,
then always call the vector rxq setup, since we can now use the vector
PMD for receiving jumbo frames that need chained mbufs, a.k.a scattered
packets. Up till now, this function was not being called when receiving
scattered packets, potentially leading to problems with mbufs not being
properly initialized.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Changchun Ouyang <Changchun.ouyang@intel.com>
Make a small improvement to slow path TX performance by adding in a
prefetch for the second mbuf cache line.
Also move assignment of l2/l3 length values only when needed.
What I've done with the prefetches is two-fold:
1) changed it from prefetching the mbuf (first cache line) to prefetching
the mbuf pool pointer (second cache line) so that when we go to access
the pool pointer to free transmitted mbufs we don't get a cache miss. When
clearing the ring and freeing mbufs, the pool pointer is the only mbuf
field used, so we don't need that first cache line.
2) changed the code to prefetch earlier - in effect to prefetch one mbuf
ahead. The original code prefetched the mbuf to be freed as soon as it
started processing the mbuf to replace it. Instead now, every time we
calculate what the next mbuf position is going to be we prefetch the mbuf
in that position (i.e. the mbuf pool pointer we are going to free the mbuf
to), even while we are still updating the previous mbuf slot on the ring.
This gives the prefetch much more time to resolve and get the data we need
in the cache before we need it.
In terms of performance difference, a quick sanity test using testpmd
on a Xeon (Sandy Bridge uarch) platform showed performance increases
between approx 8-18%, depending on the particular RX path used in
conjuntion with this TX path code.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The receive functions for packets do not modify the next pointer so
the next pointer should always be cleared on mbuf free, just in case.
The slow-path TX needs to clear it, and the standard mbuf free function
also needs to clear it. Fast path TX does not handle chained mbufs so
is unaffected
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Rename start_?x_per_q to ?x_deferred_start
and add comments.
Signed-off-by: Changchun Ouyang <changchun.ouyang@intel.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
When Flow Director was used together with bulk alloc, id and hash
was swapped when packet matches flow director filter due to improper
fdir field initialization.
Signed-off-by: Pawel Wodkowski <pawelx.wodkowski@intel.com>
Reviewed-by: Helin Zhang <helin.zhang@intel.com>
Reviewed-by: Jingjing Wu <jingjing.wu@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
[Thomas: merged with mbuf changes]
- remove leading \n in some messages,
- remove trailing \n in some messages,
- split multi lines messages.
Signed-off-by: David Marchand <david.marchand@6wind.com>
Reviewed-by: Jay Rolette <rolette@infiniteio.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Prepare for next commit, indent sections where log messages will be modified so
that next patch is only about \n.
Signed-off-by: David Marchand <david.marchand@6wind.com>
[Thomas: fix also some missing whitespaces]
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
- We should not use DEBUGOUT*/DEBUGFUNC macros in pmd code.
These macros come as compat wrappers for base driver.
- We should avoid calling RTE_LOG directly as pmd provides a wrapper for logs.
- Replace some PMD_INIT_LOG(DEBUG, "some_func") with PMD_INIT_FUNC_TRACE().
Signed-off-by: David Marchand <david.marchand@6wind.com>
Reviewed-by: Jay Rolette <rolette@infiniteio.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Provide a wrapper routine to enable receive of scattered packets with a
vector driver. This improves the performance of the slow-path RX.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Adjust the fast-path code to fix the regression caused by the pool
pointer moving to the second cache line. This change adjusts the
prefetching and also the way in which the mbufs are freed back to the
mempool.
Note: slow-path e.g. path supporting jumbo frames, is still slower, but
is dealt with by a later commit
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The vector PMD expects fields to be in a specific order so that it can
do vector operations on multiple fields at a time. Following mbuf
rework, adjust driver to take account of the new layout and re-enable it
in the config.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The offload flags field (ol_flags) was 16-bits and had no further room
for expansion. This patch increases the field size to 64-bits, using up
the remaining reserved space in the single-cache-line mbuf.
NOTE: none of the values for existing flags have been changed, i.e. no
new numbers have been explicitly reserved between existing flag
definitions.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
The mbuf structure already contains a pointer to the beginning of the
buffer (m->buf_addr). It is not needed to use 8 bytes again to store
another pointer to the beginning of the data.
Using a 16 bits unsigned integer is enough as we know that a mbuf is
never longer than 64KB. We gain 6 bytes in the structure thanks to
this modification.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
* Updated to apply to latest on mainline.
* Disabled vector PMD in config as it relies heavily on the mbuf layout
This will be re-enabled in a subsequent commit once vPMD has been
reworked to take account of mbuf changes.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The vlan_macip structure combined a vlan tag id with l2 and l3 headers
lengths for tracking offloads. However, this structure was only used as
a unit by the e1000 and ixgbe drivers, not generally.
This patch removes the structure from the mbuf header and places the
fields into the mbuf structure directly at the required point, without
any net effect on the structure layout. This allows us to treat the vlan
tags and header length fields as separate for future mbuf changes. The
drivers which were written to use the combined structure still do so,
using a driver-local definition of it.
Reduce perf regression caused by splitting vlan_macip field. This is
done by providing a single uint16_t value to allow writing/clearing
the l2 and l3 lengths together. There is still a small perf hit to the
slow path TX due to the reads from vlan_tci and l2/l3 lengths being
separated. (<5% in my tests with testpmd with no extra params).
Unfortunately, this cannot be eliminated, without restoring the vlan
tags and l2/l3 lengths as a combined 32-bit field. This would prevent
us from ever looking to move those fields about and is an artificial tie
that applies only for performance in igb and ixgbe drivers. Therefore,
this patch keeps the vlan_tci field separate from the lengths as the
best solution going forward.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
In some cases we may want to tag a packet for a particular destination
or output port, so rename the "in_port" field in the mbuf to just "port"
so that it can be re-used for this purpose if an application needs it.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The rte_pktmbuf structure was initially included in the rte_mbuf
structure. This was needed when there was 2 types of mbuf (ctrl and
packet). As the control mbuf has been removed, we can merge the
rte_pktmbuf into the rte_mbuf structure.
Advantages of doing this:
- the access to mbuf fields is easier (ex: m->data instead of m->pkt.data)
- make the structure more consistent: for instance, there was no reason
to have the ol_flags field in rte_mbuf
- it will allow a deeper reorganization of the rte_mbuf structure in the
next commits, allowing to gain several bytes in it
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
[Bruce: updated for latest code and new example apps]
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The initial role of rte_ctrlmbuf is to carry generic messages (data
pointer + data length) but it's not used by the DPDK or it applications.
Keeping it implies:
- loosing 1 byte in the rte_mbuf structure
- having some dead code rte_mbuf.[ch]
This patch removes this feature. Thanks to it, it is now possible to
simplify the rte_mbuf structure by merging the rte_pktmbuf structure
in it. This is done in next commit.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
* Updated patch to HEAD.
* Modified patch to retain the old function names for ctrl mbufs as
macros. This helps with app compatibility, and allows the concept
of a control mbuf to be reintroduced via a single-bit flag in
a future change.
* Updated the packet framework ip_pipeline example application to
work following this change.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The function rte_snprintf serves no useful purpose. It is the
same as snprintf() for all valid inputs. Deprecate it and
replace all uses in current code.
Leave the tests for the deprecated function in place.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
The vpmd RX don't accept burst size less than 32.
As vPMD is set =y by default, while default testpmd burst size is 16.
Which will cause RX nothing if not assign burst size correctly.
Signed-off-by: Cunming Liang <cunming.liang@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Yong Liu <yong.liu@intel.com>
Tested-by: Zhaochen Zhan <zhaochen.zhan@intel.com>
The support of jumbo frames in the ixgbevf Poll Mode Driver of 10GbE
82599 VF functions consists in the following enhancements:
- Implement the mtu_set function in the ixgbevf PMD, using the IXGBE_VF_SET_LPE
request of the version 1.0 of the VF/PF mailbox API for this purpose.
- Add a detailed explanation on the VF/PF rx max frame len negotiation.
Signed-off-by: Ivan Boule <ivan.boule@6wind.com>
Signed-off-by: David Marchand <david.marchand@6wind.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
