The function rte_hash_lookup_multi() was renamed rte_hash_lookup_bulk()
in DPDK 1.4 and was kept as an undocumented alias.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
To avoid a race condition while creating a new hash object, the
list has to be locked before the lookup, and released only once the
new object is added in the list.
As the lock is held by the rte_ring_create(), move its creation at the
beginning of the function and only take the lock after the ring is
created to avoid a deadlock.
Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Change rte_hash*_create() functions to return NULL and set rte_errno to
EEXIST when the object name already exists. This is the behavior
described in the API documentation in the header file.
These functions were returning a pointer to the existing object in that
case, but it is a problem as the caller did not know if the object had
to be freed or not.
Doing this change also makes the hash API more consistent with the other
APIs (mempool, rings, ...).
Fixes: 916e4f4f4e ("memory: fix for multi process support")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Hash library used a function pointer to choose a different
key compare function, depending on the key size.
As a result, multiple processes could not use the same hash table,
as the function addresses vary from one process to another.
Instead, a jump table is used, so each process has its own
function addresses, accessing this table with an index stored
in the hash table (note that using a custom key compare function
is not supported in multi-process mode).
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Instead of using RTE_ARCH_X86_64, RTE_ARCH_X86_32
and RTE_ARCH_I686, use directly RTTE_ARCH_X86
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
rte_hash_set_cmp_func() had an incorrect Doxygen comment
for one of its parameters.
Fixes: 95da2f8e9c61 ("hash: customize compare function")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Fix crc32c hash functions to return a valid crc32c value for
data lengths not multiple of 4 bytes.
ARM code is not tested.
Fixes: af75078fece3 ("first public release")
Signed-off-by: Didier Pallard <didier.pallard@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When built in a C++ application, the jhash include fails:
rte_jhash.h:123:22: error:
invalid conversion from ‘const void*’ to ‘const uint32_t*’ [-fpermissive]
const uint32_t *k = key;
^
The cast must be explicit for C++.
Fixes: 8718219a8737 ("hash: add new jhash functions")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
fix the error reported by checkpatch:
"ERROR: return is not a function, parentheses are not required"
remove parentheses in return like:
"return (logical expressions)"
remove parentheses in return a function like:
"return (rte_mempool_lookup(...))"
Fixes: 6307b909b8e0 ("lib: remove extra parenthesis after return")
Signed-off-by: Huawei Xie <huawei.xie@intel.com>
This patch fixes the compile errors caused by lacking of "size_t"
definition in rte_hash.h.
The compile error exists on IBM POWER and ARM.
The errors are like:
In file included from app/test/test_hash_scaling.c:35:0:
rte_hash.h:70:70: error: unknown type name ‘size_t’
Fixes: 95da2f8e9c61 ("hash: customize compare function")
Signed-off-by: Chao Zhu <chaozhu@linux.vnet.ibm.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Give user a chance to customize the hash key compare function.
The default rte_hash_cmp_eq function is set in the rte_hash_create
function, but these builtin ones may not good enough, so the user
may call this to override the default one.
Signed-off-by: Yu Nemo Wenbin <yuwb_bjy@ctbri.com.cn>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
rte_hash_create function was accidentally duplicated in
DPDK_2.1 in rte_hash_version.map.
Fixes: 473d1beb ("hash: allow to store data in hash table")
Reported-by: Ferruh Yigit <ferruh.yigit@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
select hash function for cuckoo, fbk as rte_hash_crc_4byte
if arm64-CRC extension available
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
armv8-a has optional CRC32 extension, march=armv8-a+crc enables code
generation for the ARMv8-A architecture together with
the optional CRC32 extensions.
added RTE_MACHINE_CPUFLAG_CRC32 to detect the availability of
CRC32 extension in compile time. At run-time, The RTE_CPUFLAG_CRC32
can be used to find the availability.
armv8-a+crc target support added in GCC 4.9,
Used inline assembly and emulated __ARM_FEATURE_CRC32 to work
with tool-chain < 4.9
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
The following measurements shows improvement over the default
libc memcmp function
Length(B) by X% over libc memcmp
16 149.57%
32 122.7%
48 104.96%
64 98.21%
80 93.75%
96 90.55%
112 110.48%
128 137.24%
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
If user has not added an all zero key in the hash table,
and tries to look it up, it results in an incorrect hit,
as dummy slot in the key table has all zero as well.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
If using multiple cores on a system with hardware transactional
memory support, thread scaling does not work, as there was a single
point in the hash library which is a bottleneck for all threads,
which is the "free_slots" ring, which stores all the indices of
the free slots in the table.
This patch fixes the problem, by creating a local cache per logical core,
which stores locally indices of free slots,
so most times, writer threads will not interfere each other.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Since freeing a ring is now possible, then when freeing
a hash table, its internal ring can be freed as well.
Therefore when a new table, with the same name as a previously
freed table, is created, there is no need to look up
the already allocated ring.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
When calculating the size for the table which allocates
the keys, size was calculated wrongly from multiplying
two 32-bit variables, resulting on a 32-bit number,
before casting to 64-bit, so maximum size was 4G.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
The function rte_jhash2() was renamed rte_jhash_32b and
macros RTE_HASH_KEY_LENGTH_MAX and RTE_HASH_BUCKET_ENTRIES_MAX
were tagged as deprecated, so they can be removed in 2.2.
RTE_HASH_KEY_LENGTH is replaced in unit tests by an internal macro
for the memory allocation of all keys used.
The library version number is incremented.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Make thash library arch-independent.
Leave unaligned union rte_thash_tuple if no support for SSE3.
Makes 32bit compiler happy by adding ULL suffix.
Signed-off-by: Vladimir Medvedkin <medvedkinv@gmail.com>
When adding with cuckoo hash a key which was already inserted
a new slot is dequeued and then enqueued back, but the enqueue
operation was not done properly.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
In order to keep the ABI consistent with the old hash library,
hash_func_init_val field has been moved, so it remains
at the same offset as previously, since hash_func and
hash_func_init_val are fields accessed by the public function
rte_hash_hash and must keep the same offset as older versions.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Hash library uses optimized compare functions that use
x86 intrinsics, therefore non-x86 systems could not build
the library. In that case, the compare function is set
to the generic memcmp.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Reported-by: Zhigang Lu <zlu@ezchip.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Zhigang Lu <zlu@ezchip.com>
_mm_test_all_zeros is not available for CPUs with no SSE4.1,
therefore, DPDK would not build.
This patch adds an alternative for this, using _mm_cmpeq_epi32 and
_mm_movemask_epi8.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
gcc 4.4 and 4.5 throws following error:
rte_cuckoo_hash.c:145: error: flexible array member in otherwise empty struct.
This is due to empty length in flexible array, which has been changed to use
size 0 in the declaration of the array.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Reported-by: Olga Shern <olgas@mellanox.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When encountering a loop while adding a new entry,
element out of bounds of array was being unnecessarily resetted.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Move malloc inside eal and create a new section in MAINTAINERS file for
Memory Allocation in EAL.
Create a dummy malloc library to avoid breaking applications that have
librte_malloc in their DT_NEEDED entries.
This is the first step towards using malloc to allocate memory directly
from memsegs. Thus, memzones would allocate memory through malloc,
allowing to free memzones.
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
The cuckoo hash has a fixed number of entries per bucket, so the
configuration parameter for this is unused. We change this field in the
parameters struct to "reserved" to indicate that there is now no such
parameter value, while at the same time keeping ABI consistency.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Suggested-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
This is necessary because the required CPU flags may not be defined on
other architectures.
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
Signed-off-by: Zhigang Lu <zlu@ezchip.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Since now rte_hash structure is private, a new function
has been added to let the user iterate through the hash table,
returning next key and data associated on each iteration,
plus the position where they were stored.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Usually hash tables not only store keys, but also data associated
to them. In order to maintain the existing API, the old functions
will still return the index where the key was stored.
The new functions will return the data associated to that key.
In the case of the lookup_bulk function, it will return also
the number of entries found and a bitmask of which entries
were found.
Unit tests have been updated to use these new functions.
As a final point, a flag has been added in rte_hash_parameters
to indicate if there are new parameters for future versions,
so there is no need to maintain multiple versions
of the existing functions in the future.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
[Thomas: fix || operator in a precondition check]
Added reset function to be able to empty the table,
without having to destroy and create it again.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
This patch replaces the existing hash library with another approach,
using the Cuckoo Hash method to resolve collisions (open addressing),
which pushes items from a full bucket when a new entry tries
to be added in it, storing the evicted entry in an alternative location,
using a secondary hash function.
This gives the user the ability to store more entries when a bucket
is full, in comparison with the previous implementation.
Therefore, the unit test has been updated, as some scenarios have changed
(such as the previous removed restriction).
Also note that the API has not been changed, although new fields
have been added in the rte_hash structure (structure is internal now).
The main change when creating a new table is that the number of entries
per bucket is fixed now, so its parameter is ignored now
(still there to maintain the same parameters structure).
The hash unit test has been updated to reflect these changes.
As a last note, the maximum burst size in lookup_burst function
hash been increased to 64, to improve performance.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
rte_hash structure should not be a public structure,
and therefore it should be moved to the C file and be declared
as internal. rte_hash_hash implementation is also moved
to the C file, as it uses the structure.
This patch also removes part of a unit test that was checking
a field of the structure.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
In the no-x86 branch, a parentheses is missing.
And need to include <limits.h> header to use CHAR_BIT.
Fixes: 49361c3f3cfa ("hash: remove duplicated code")
Signed-off-by: Bruce Liu <2103458176@qq.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Software implementation of the Toeplitz hash function used by RSS.
Can be used either for packet distribution on single queue NIC
or for simulating of RSS computation on specific NIC (for example
after GRE header decapsulating).
Signed-off-by: Vladimir Medvedkin <medvedkinv@gmail.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
rte_hash_hash is a public function but was not in
rte_hash_version.map
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Since sig_tbl_bucket_size and key_tbl_key_size are explicitly aligned
at initialization, offset dereferences in the hash table code cannot
possibly be unaligned. However, the compiler is unaware of this fact
and complains on -Wcast-align. This patch modifies the code to use
RTE_PTR_ADD(), thereby silencing the compiler by casting through (void
*).
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Changed name to something more meaningful,
and mark rte_jhash2 as deprecated.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
rte_jhash is basically like __rte_jhash_2hashes but
it returns only 1 hash, instead of 2.
In order to remove duplicated code, rte_jhash calls __rte_jhash_2hashes,
passing 0 as the second seed and returning just the first hash value.
(performance penalty is negligible)
The same is done with rte_jhash2. Also, rte_jhash2 is just an specific case
where keys are multiple of 32 bits, and where no key alignment check is required.
So,to avoid duplicated code, the function calls __rte_jhash_2hashes
with check_align = 0 (to use the optimal path)
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
With the jhash update, two new functions were introduced:
- rte_jhash_2hashes: Same as rte_jhash, but takes two seeds
and return two hashes (uint32_ts)
- rte_jhash2_2hashes: Same as rte_jhash2, but takes two seeds
and return two hashes (uint32_ts)
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
[Thomas: fix doxygen typos]
Jenkins hash function was developed originally in 1996,
and was integrated in first versions of DPDK.
The function has been improved in 2006,
achieving up to 35% better performance, compared to the original one.
This patch integrates that code into the rte_jhash library.
It also updates the precalculated hash values in the unit test,
as the code now returns different values (expected).
A final note has been added in release notes for stating
the changes made.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Fix rte_hash_crc() function by making use of uintptr_t variable
to hold a pointer to data being hashed. In this way, casting uint64_t
pointer to uint32_t avoided.
Fixes: 614289298daf ("hash: slice CRC data into 8-byte pieces")
Signed-off-by: Yerden Zhumabekov <e_zhumabekov@sts.kz>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When compile target i686 in platform x86_64, the stub function will
be called, and return zero.
Fixes: 12fa4a00788d ("hash: fix unsupported crc instruction in i686 platform")
Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Acked-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Error: unsupported instruction `crc32'
The root cause is that i686 platform does not support 'crc32q'
Need make it only available in x86_64 platform.
Signed-off-by: Michael Qiu <michael.qiu@intel.com>
Acked-by: Yerden Zhumabekov <yerden.zhumabekov@sts.kz>
