To avoid a race condition while creating a new hash object, the
list has to be locked before the lookup, and released only once the
new object is added in the list.
As the lock is held by the rte_ring_create(), move its creation at the
beginning of the function and only take the lock after the ring is
created to avoid a deadlock.
Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Change rte_hash*_create() functions to return NULL and set rte_errno to
EEXIST when the object name already exists. This is the behavior
described in the API documentation in the header file.
These functions were returning a pointer to the existing object in that
case, but it is a problem as the caller did not know if the object had
to be freed or not.
Doing this change also makes the hash API more consistent with the other
APIs (mempool, rings, ...).
Fixes: 916e4f4f4e ("memory: fix for multi process support")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Hash library used a function pointer to choose a different
key compare function, depending on the key size.
As a result, multiple processes could not use the same hash table,
as the function addresses vary from one process to another.
Instead, a jump table is used, so each process has its own
function addresses, accessing this table with an index stored
in the hash table (note that using a custom key compare function
is not supported in multi-process mode).
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Instead of using RTE_ARCH_X86_64, RTE_ARCH_X86_32
and RTE_ARCH_I686, use directly RTTE_ARCH_X86
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
fix the error reported by checkpatch:
"ERROR: return is not a function, parentheses are not required"
remove parentheses in return like:
"return (logical expressions)"
remove parentheses in return a function like:
"return (rte_mempool_lookup(...))"
Fixes: 6307b909b8e0 ("lib: remove extra parenthesis after return")
Signed-off-by: Huawei Xie <huawei.xie@intel.com>
Give user a chance to customize the hash key compare function.
The default rte_hash_cmp_eq function is set in the rte_hash_create
function, but these builtin ones may not good enough, so the user
may call this to override the default one.
Signed-off-by: Yu Nemo Wenbin <yuwb_bjy@ctbri.com.cn>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
select hash function for cuckoo, fbk as rte_hash_crc_4byte
if arm64-CRC extension available
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
The following measurements shows improvement over the default
libc memcmp function
Length(B) by X% over libc memcmp
16 149.57%
32 122.7%
48 104.96%
64 98.21%
80 93.75%
96 90.55%
112 110.48%
128 137.24%
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
If user has not added an all zero key in the hash table,
and tries to look it up, it results in an incorrect hit,
as dummy slot in the key table has all zero as well.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
If using multiple cores on a system with hardware transactional
memory support, thread scaling does not work, as there was a single
point in the hash library which is a bottleneck for all threads,
which is the "free_slots" ring, which stores all the indices of
the free slots in the table.
This patch fixes the problem, by creating a local cache per logical core,
which stores locally indices of free slots,
so most times, writer threads will not interfere each other.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Since freeing a ring is now possible, then when freeing
a hash table, its internal ring can be freed as well.
Therefore when a new table, with the same name as a previously
freed table, is created, there is no need to look up
the already allocated ring.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
When calculating the size for the table which allocates
the keys, size was calculated wrongly from multiplying
two 32-bit variables, resulting on a 32-bit number,
before casting to 64-bit, so maximum size was 4G.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When adding with cuckoo hash a key which was already inserted
a new slot is dequeued and then enqueued back, but the enqueue
operation was not done properly.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Xavier Simonart <xavier.simonart@intel.com>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
In order to keep the ABI consistent with the old hash library,
hash_func_init_val field has been moved, so it remains
at the same offset as previously, since hash_func and
hash_func_init_val are fields accessed by the public function
rte_hash_hash and must keep the same offset as older versions.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Hash library uses optimized compare functions that use
x86 intrinsics, therefore non-x86 systems could not build
the library. In that case, the compare function is set
to the generic memcmp.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Reported-by: Zhigang Lu <zlu@ezchip.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Zhigang Lu <zlu@ezchip.com>
_mm_test_all_zeros is not available for CPUs with no SSE4.1,
therefore, DPDK would not build.
This patch adds an alternative for this, using _mm_cmpeq_epi32 and
_mm_movemask_epi8.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
gcc 4.4 and 4.5 throws following error:
rte_cuckoo_hash.c:145: error: flexible array member in otherwise empty struct.
This is due to empty length in flexible array, which has been changed to use
size 0 in the declaration of the array.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Reported-by: Olga Shern <olgas@mellanox.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
When encountering a loop while adding a new entry,
element out of bounds of array was being unnecessarily resetted.
Fixes: 48a399119619 ("hash: replace with cuckoo hash implementation")
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Since now rte_hash structure is private, a new function
has been added to let the user iterate through the hash table,
returning next key and data associated on each iteration,
plus the position where they were stored.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Usually hash tables not only store keys, but also data associated
to them. In order to maintain the existing API, the old functions
will still return the index where the key was stored.
The new functions will return the data associated to that key.
In the case of the lookup_bulk function, it will return also
the number of entries found and a bitmask of which entries
were found.
Unit tests have been updated to use these new functions.
As a final point, a flag has been added in rte_hash_parameters
to indicate if there are new parameters for future versions,
so there is no need to maintain multiple versions
of the existing functions in the future.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
[Thomas: fix || operator in a precondition check]
Added reset function to be able to empty the table,
without having to destroy and create it again.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
This patch replaces the existing hash library with another approach,
using the Cuckoo Hash method to resolve collisions (open addressing),
which pushes items from a full bucket when a new entry tries
to be added in it, storing the evicted entry in an alternative location,
using a secondary hash function.
This gives the user the ability to store more entries when a bucket
is full, in comparison with the previous implementation.
Therefore, the unit test has been updated, as some scenarios have changed
(such as the previous removed restriction).
Also note that the API has not been changed, although new fields
have been added in the rte_hash structure (structure is internal now).
The main change when creating a new table is that the number of entries
per bucket is fixed now, so its parameter is ignored now
(still there to maintain the same parameters structure).
The hash unit test has been updated to reflect these changes.
As a last note, the maximum burst size in lookup_burst function
hash been increased to 64, to improve performance.
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
