It seems that with gcc >5.x and -O2/-O3 optimization breaks packet
grouping algorithm.
When last packet pointer "lp" and "pnum->u64" buffer points the same
memory buffer, high optimization can cause unpredictable results.
It seems that assignment of precalculated group sizes may interfere
with initialization of new group size when lp points value inside
current group and didn't should be changed.
With gcc >5.x and optimization we cannot be sure which assignment will
be done first, so the group size can be counted incorrectly.
This patch eliminates intersection of assignment of initial group size
(lp[0] = 1) and precalculated group sizes when gptbl[v].idx < 4.
Fixes: 94c54b4158 ("examples/l3fwd: rework exact-match")
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Define and use ETH_LINK_UP and ETH_LINK_DOWN where appropriate.
Signed-off-by: Marc Sune <marcdevel@gmail.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Originally l3fwd used 16-bit value to store dest_port value.
To accommodate 24-bit nexthop dest_port was increased to 32-bit,
though some further packet processing code remained unchanged and
still expects dest_port to be 16-bit.
That is not correct and can cause l3fwd invalid behaviour or even
process crash/hang on some input packet patterns.
For the fix, I choose the simplest approach and restored dest_port
as 16-bit value, plus necessary conversions from 32 to 16 bit values
after lpm_lookupx4.
Fixes: dc81ebbaca ("lpm: extend IPv4 next hop field")
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Not all tx ports was included in tx_port_id array, used to periodically
drain only available ports. This caused that some packets remain in buffer
when application stops to receiving packets.
Fixes: 52c97adc1f ("examples/l3fwd: fix exact match performance")
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
As a example to use ptype info, l3fwd needs firstly to use
rte_eth_dev_get_supported_ptypes() API to check if device and/or
its PMD driver will parse and fill the needed packet type; if not,
use the newly added option, --parse-ptype, to analyze it in the
callback softly.
As the mode of EXACT_MATCH uses the 5 tuples to caculate hash, so
we narrow down its scope to:
a. ip packets with no extensions, and
b. L4 payload should be either tcp or udp.
Note: this patch does not completely solve the issue, "cannot run
l3fwd on virtio or other devices", because hw_ip_checksum may be
not supported by the devices. Currently we can:
a. remove this requirements, or
b. wait for virtio front end (pmd) to support it.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Added validation for queue id of config parameter tuple.
This validation enforces user to enter queue ids of a port
from 0 and in sequence.
This additional validation on queue ids avoids ixgbe crash caused
by null rxq pointer access inside ixgbe_dev_rx_init.
Reason for null rxq is, L3fwd application allocates memory only for
queues passed by user. But rte_eth_dev_start tries to initialize rx
queues in sequence from 0 to nb_rx_queues,
which is not true and coredump while accessing the unallocated queue .
Fixes: af75078fec ("first public release")
Signed-off-by: Reshma Pattan <reshma.pattan@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The flag ENABLE_MULTI_BUFFER_OPTIMIZE has been removed so the
related comments are now useless.
Fixes: 268888b5b0 ("examples/l3fwd: modularize")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
It seems that for the most use cases, previous hash_multi_lookup provides
better performance, and more, sequential lookup can cause significant
performance drop.
This patch sets previously optional hash_multi_lookup method as default.
It also provides some minor optimizations such as queue drain only on used
tx ports.
Fixes: 94c54b4158 ("examples/l3fwd: rework exact-match")
Fixes: dc81ebbaca ("lpm: extend IPv4 next hop field")
Fixes: 64d3955de1 ("examples/l3fwd: fix ARM build")
Reported-by: Qian Xu <qian.q.xu@intel.com>
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
l3fwd does not compile with HASH_MULTI_LOOKUP.
2 issues:
* in 64d395 mask0 changed type from xmm_t to rte_xmm_t
-> use x field from rte_xmm_t
* in dc81eb dst_port parameter changed to uint32_t
-> change uint16_t dst_port to uin32_t dsp_port
Fixes: dc81ebbaca ("lpm: extend IPv4 next hop field")
Fixes: 64d3955de1 ("examples/l3fwd: fix ARM build")
Signed-off-by: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
Enable NEON support in exact match mode.
l3fwd example did not compile on ARM due to SSE2 instrincics used
in generic part.
Some instrinsins were used to initialize data structures and those were
replaced by ordinary structure initalization.
All SSE2 intrinsics used in forwarding, i.e. masking the IP/TCP header
are moved to single inline function and made arch-specific.
Signed-off-by: Maciej Czekaj <maciej.czekaj@caviumnetworks.com>
A new rte_lpm_config structure is used so LPM library will allocate
exactly the amount of memory which is necessary to hold application’s
rules.
Signed-off-by: Michal Kobylinski <michalx.kobylinski@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
This patch extend next_hop field from 8-bits to 24-bits in LPM library
for IPv4.
Added versioning symbols to functions and updated
library and applications that have a dependency on LPM library.
Signed-off-by: Michal Kobylinski <michalx.kobylinski@intel.com>
Acked-by: David Hunt <david.hunt@intel.com>
Current implementation of Exact-Match uses different execution path than
for LPM. Unifying them allows to reuse big part of LPM code and sightly
increase performance of Exact-Match.
Main changes:
-------------
* Packet classification stage is separated from the rest of path for both
LPM and EM.
* Packet processing, modifying and transmit part is the same for LPM and EM
and mostly based on the current LPM implementation.
* Shared code is moved to the common file "l3fwd_sse.h".
* While sequential packet classification in EM path, seems to be faster
than using multi hash lookup, used before, it is used by default. Old
implementation is moved to the file l3fwd_em_hlm_sse.h and can be enabled
with HASH_LOOKUP_MULTI global define in compilation time.
Signed-off-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
The main problem with l3fwd is that it is too monolithic with everything
being in one file, and the various options all controlled by compile time
flags. This means that it's hard to read and understand, and when making
any changes, you need to go to a lot of work to try and ensure you cover
all the code paths, since a compile of the app will not touch large parts
of the l3fwd codebase.
Following changes were done to fix the issues mentioned above
- Split out the various lpm and hash specific functionality into separate
files, so that l3fwd code has one file for common code e.g. args
processing, mempool creation, and then individual files for the various
forwarding approaches.
Following are new file lists
main.c (Common code for args processing, memppol creation, etc)
l3fwd_em.c (Hash/Exact match aka 'EM' functionality)
l3fwd_em_sse.h (SSE4_1 buffer optimizated 'EM' code)
l3fwd_lpm.c (Longest Prefix Match aka 'LPM' functionality)
l3fwd_lpm_sse.h (SSE4_1 buffer optimizated 'LPM' code)
l3fwd.h (Common include for 'EM' and 'LPM')
- The choosing of the lpm/hash path should be done at runtime, not
compile time, via a command-line argument. This will ensure that
both code paths get compiled in a single go
Following examples show runtime options provided
Select 'LPM' or 'EM' based on run time selection f.e.
> l3fwd -c 0x1 -n 1 -- -p 0x1 -E ... (EM)
> l3fwd -c 0x1 -n 1 -- -p 0x1 -L ... (LPM)
Options "E" and "L" are mutualy-exclusive.
If none selected, "L" is default.
Signed-off-by: Ravi Kerur <rkerur@gmail.com>
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Tested-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Tomasz Kulasek <tomaszx.kulasek@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
clang reports this error:
examples/l3fwd/main.c:550:1: error: unused function 'send_packetsx4'
The function is used only when ENABLE_MULTI_BUFFER_OPTIMIZE is 1.
Fixes: 96ff445371 ("examples/l3fwd: reorganise and optimize LPM code path")
Fixes: 6f1c1e28d9 ("examples/l3fwd: fix build with exact-match enabled")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
fix the error reported by checkpatch:
"ERROR: return is not a function, parentheses are not required"
remove parentheses in return like:
"return (logical expressions)"
remove parentheses in return a function like:
"return (rte_mempool_lookup(...))"
Fixes: 6307b909b8 ("lib: remove extra parenthesis after return")
Signed-off-by: Huawei Xie <huawei.xie@intel.com>
Handle SIGINT and SIGTERM in l3fwd.
Signed-off-by: Zhihong Wang <zhihong.wang@intel.com>
Acked-by: Michael Qiu <michael.qiu@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
l3fwd app expects PMDs to return packets whose L2 header is
16-byte aligned due to usage of _mm_load_si128()/_mm_store_si128()
intrinsics in the app. However, most of the protocol stacks expects
packets such that its IP/L3 header be aligned on a 16-byte boundary.
Based on the recommendations received on dpdk-dev, we are changing
the l3fwd app to use _mm_loadu_si128()/_mm_loadu_si128() so that the
address need not be 16-byte aligned and thereby preventing crash.
We have tested that there is no performance impact due to this
change.
Signed-off-by: Harish Patil <harish.patil@qlogic.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Lookup burst size was changed for exact match
from 4 to 8, for both ipv4 and ipv6, but actually only
4 keys were being looked up for ipv6, instead of 8,
causing random segmentation faults.
Fixes: 80fcb4d4 ("examples/l3fwd: increase lookup burst size to 8")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Reshma Pattan <reshma.pattan@intel.com>
Fix minor, and non critical, copy and paste error in strncmp() of eth-dest
commandline argument.
Fixes: bd785f6f67 ("examples/l3fwd: make destination mac address configurable")
Signed-off-by: John McNamara <john.mcnamara@intel.com>
Acked-by: Andrey Chilikin <andrey.chilikin@intel.com>
The extended unified packet type is now part of the standard ABI.
As mbuf struct is changed, the mbuf library version is incremented.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
L3fwd was trying to use an inexistent function "simple_ipv6_fwd_4pkts",
instead it should be "simple_ipv6_fwd_8pkts", and "simple_ipv8_fwd_4pkts"
instead of "simple_ipv4_fwd_8pkts".
clang reports some unused functions, used only for LPM lookup:
examples/l3fwd/main.c:545:1: error: unused function 'send_packetsx4'
examples/l3fwd/main.c:1165:1: error: unused function 'rfc1812_process'
Fixes: 80fcb4d4 ("examples/l3fwd: increase lookup burst size to 8")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
[Thomas: more #if to fix clang warnings]
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
With the new hash implementation, the minimum lookup burst size
to get good performance is 8, since its internal pipeline
consists of 4 stages of 2 entries each, so to avoid
duplication, burst size should be 8 or more entries.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
To unify packet types among all PMDs, bit masks of packet type for
'ol_flags' are replaced by unified packet type.
To avoid breaking ABI compatibility, all the changes would be
enabled by RTE_NEXT_ABI.
Signed-off-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
The cuckoo hash has a fixed number of entries per bucket, so the
configuration parameter for this is unused. We change this field in the
parameters struct to "reserved" to indicate that there is now no such
parameter value, while at the same time keeping ABI consistency.
Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")
Suggested-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
This patch simply applies the transform previously committed in
scripts/cocci/mtod-offset.cocci. No other modifications have been
made here.
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Add a command-line parameter to l3fwd, to allow the user to specify the
destination mac address for each ethernet port used.
Signed-off-by: Andrey Chilikin <andrey.chilikin@intel.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Latest mbuf changes (priv_size addition and related fixes)
exposed small problem with testpmd and few other sample apps:
when mbuf size is exaclty 2KB or less, that causes
ixgbe PMD to select scattered RX even for configs with 'normal'
max packet length (max_rx_pkt_len == ETHER_MAX_LEN).
To overcome that problem and unify the code, new macro was created
to represent recommended minimal buffer length for mbuf.
When appropriate, samples are updated to use that macro.
Fixes: dfb03bbe2b ("app/testpmd: use standard functions to initialize
mbufs and mbuf pool")
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
When it's possible, use the new helper to create the mbuf pools.
Most of the patch is trivial, except for the following files that
have some specifics (indirect mbufs):
- ip_fragmentation
- ip_pipeline
- ipv4_multicast
- vhost
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
New data type to manipulate 256 bit AVX values.
Rename field in the rte_xmm to keep common naming across SSE/AVX fields.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
Fix the following error:
error: unused function 'l3fwd_simple_forward'
The l3fwd_simple_forward() is maybe unused, due to compilation options
(APP_LOOKUP_METHOD, ENABLE_MULTI_BUFFER_OPTIMIZE). As the combinatorial
is quite big, it looks simpler to add the __attribute__((unused)) on
this function, so that the compiler does not complain.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Signed-off-by: David Marchand <david.marchand@6wind.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
For apps that were using default rte_eth_rxconf and rte_eth_txconf
structures, these have been removed and now they are obtained by
calling rte_eth_dev_info_get, just before setting up RX/TX queues.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: David Marchand <david.marchand@6wind.com>
Since commit a155d43011 ("support link bonding device initialization"),
rte_eal_pci_probe() is called in rte_eal_init().
So it doesn't have to be called by application anymore.
It has been fixed for testpmd in commit 2950a76931,
and this patch remove it from other applications.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
The rte_pktmbuf structure was initially included in the rte_mbuf
structure. This was needed when there was 2 types of mbuf (ctrl and
packet). As the control mbuf has been removed, we can merge the
rte_pktmbuf into the rte_mbuf structure.
Advantages of doing this:
- the access to mbuf fields is easier (ex: m->data instead of m->pkt.data)
- make the structure more consistent: for instance, there was no reason
to have the ol_flags field in rte_mbuf
- it will allow a deeper reorganization of the rte_mbuf structure in the
next commits, allowing to gain several bytes in it
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
[Bruce: updated for latest code and new example apps]
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
No need for that 'x bit' on source files.
Signed-off-by: David Marchand <david.marchand@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Latest changes introduced a small degradation for the corner case
when each input packet is destined to the different port.
For the test-case when 1 core manages 4 ports and packet stream looks like:
IPV4_DSTPORT0, IPV4_DSTPORT1, IPV4_DSTPORT3, IPV4_DSTPORT4, IPV4_DSTPORT0, ...
non-optimised code path outperforms optimised one by 2-3%.
These changes supposed to close that gap.
From my testing: now for the case described above optimised code path
produces same numbers as non-optimised one.
For other test-cases numbers remain about the same.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
The function rte_snprintf serves no useful purpose. It is the
same as snprintf() for all valid inputs. Deprecate it and
replace all uses in current code.
Leave the tests for the deprecated function in place.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
- i40e RSS flags have been added (and enlarged to 64-bit)
- A new configuration of 'uint8_t rss_key_len' has been added in
'struct rte_eth_rss_conf' to support different length of RSS keys.
- In each PMD, only the supported flags are masked.
Signed-off-by: Helin Zhang <helin.zhang@intel.com>
Signed-off-by: Jing Chen <jing.d.chen@intel.com>
Acked-by: Cunming Liang <cunming.liang@intel.com>
Acked-by: Jijiang Liu <jijiang.liu@intel.com>
Acked-by: Jingjing Wu <jingjing.wu@intel.com>
Acked-by: Heqing Zhu <heqing.zhu@intel.com>
Tested-by: Waterman Cao <waterman.cao@intel.com>
With latest HW and optimised RX/TX path there is a huge gap between
tespmd iofwd and l3fwd performance results.
So there is an attempt to optimise l3fwd LPM code path and reduce the gap:
- Instead of processing each input packet up to completion -
divide packet processing into several stages and perform
stage by stage for the whole burst.
- Unroll things by the factor of 4 whenever possible.
- Use SSE instincts for some operations (bswap, replace MAC addresses, etc).
- Avoid TX packet buffering whenever possible.
- Move some checks from RX/TX into setup phase.
Note that new(optimized) code path can be switched on/off by setting
ENABLE_MULTI_BUFFER_OPTIMIZE macro to 1/0.
Some performance data:
SUT: dual-socket board IVB 2.8GHz, 2x1GB pages.
4 ports on 4 NICs (all at socket 0) connected to the traffic generator.
kernel: 3.11.3-201.fc19.x86_64, gcc: 4.8.2.
64B packets, using the packet flooding method.
All 4 ports are managed by one logical core:
Optimised scalar PMD RX/TX was used.
DIFF % (NEW-OLD)
IPV4-CONT-BURST: +23%
IPV6-CONT-BURST : +13%
IPV4/IPV6-CONT-BURST: +8%
IPV4-4STREAMSX8: +7%
IPV4-4STREAMSX1: -2%
Test cases description:
IPV4-CONT-BURST - IPV4 packets all packets from the one input port
are destined for the same output port.
IPV6-CONT-BURST - IPV6 packets all packets from the one input port
are destined for the same output port.
IPV4/IPV6-CONT-BURST - mix of the first 2 with interleave=1
(e.g: IPV4,IPV6,IPV4,IPV6, ...)
IPV4-4STREAMSX1 - 4 streams of IPV4 packets, where all packets
from same stream are destined for the same output port
(e.g: IPV4_DST_P0, IPV4_DST_P1, IPV4_DST_P2, IPV4_DST_P3, IPV4_DST_P0, ...)
IPV4-4STREAMSX8 - same as above but packets for each stream
are coming in groups of 8
(e.g: IPV4_DST_P0 X 8, IPV4_DST_P1 X 8, IPV4_DST_P2 X 8, IPV4_DST_P3 X 8,
IPV4_DST_P0 X 8, ...)
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Tested-by: Waterman Cao <waterman.cao@intel.com>
Acked-by: Pablo de Lara Guarch <pablo.de.lara.guarch@intel.com>
This commit removes trailing whitespace from lines in files. Almost all
files are affected, as the BSD license copyright header had trailing
whitespace on 4 lines in it [hence the number of files reporting 8 lines
changed in the diffstat].
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Neil Horman <nhorman@tuxdriver.com>
[Thomas: remove spaces before tabs in libs]
[Thomas: remove more trailing spaces in non-C files]
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
The "default" part in configuration filenames is misleading.
Rename this as "native", as this is the RTE_MACHINE that is set in these files.
This should make it clearer for people who build DPDK on a system then run it on
another one.
Signed-off-by: David Marchand <david.marchand@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Now that we've converted all the pmds in dpdk to use the driver registration
macro, rte_pmd_init_all has become empty. As theres no reason to keep it around
anymore, just remove it and fix up all the eample callers.
Signed-off-by: Neil Horman <nhorman@tuxdriver.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
A series of minor changes to example applications included in the
Intel DPDK 1.6 release.
* changes to NIC configuration flags, e.g. specifying RSS
* replacing local "DIM" macro with common "RTE_DIM" macro
* minor whitespace changes for alignment.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
