Commit Graph

125 Commits

Author SHA1 Message Date
Shreyansh Jain
0da959d484 hash: fix comment for lookup
rte_hash_lookup_with_hash() has wrong comment for its 'sig' param.

Fixes: 1a9f648be2 ("hash: fix for multi-process apps")
Cc: stable@dpdk.org

Signed-off-by: Shreyansh Jain <shreyansh.jain@nxp.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2018-04-15 15:07:11 +02:00
Pavan Nikhilesh
e166e55c1a hash: fix missing spinlock unlock in add key
Fix missing spinlock unlock during add key when key is already present.

Fixes: be856325cb ("hash: add scalable multi-writer insertion with Intel TSX")
Cc: stable@dpdk.org

Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2018-04-10 23:35:40 +02:00
Bruce Richardson
579182f2b0 build: set compat lib as universal dependency
By making "compat" lib (which consists of a header only) a dependency of
the EAL, we make the header file available to all other libs, drivers and
apps, and thereby make it less work to do ABI versioning.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
2018-01-30 21:59:00 +01:00
Bruce Richardson
6c9457c279 build: replace license text with SPDX tag
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Luca Boccassi <bluca@debian.org>
2018-01-30 21:58:59 +01:00
Bruce Richardson
5b9656b157 lib: build with meson
Add non-EAL libraries to DPDK build. The compat lib is a special case,
along with the previously-added EAL, but all other libs can be build using
the same set of commands, where the individual meson.build files only need
to specify their dependencies, source files, header files and ABI versions.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Luca Boccassi <luca.boccassi@gmail.com>
2018-01-30 17:49:16 +01:00
Elza Mathew
c43cb3b184 hash: select fbk function at run-time
Compile-time function selection can potentially lead to
lower performance on generic builds done by distros.
Replaced compile time flag checks with run-time function
selection.

Signed-off-by: Elza Mathew <elza.mathew@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2018-01-20 15:35:16 +01:00
Elza Mathew
3a47be9abb hash: select cuckoo function at run-time
Compile-time function selection can potentially lead to
lower performance on generic builds done by distros.
Replaced compile time flag checks with run-time function
selection.

Signed-off-by: Elza Mathew <elza.mathew@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2018-01-20 15:34:50 +01:00
Adrien Mazarguil
0d440d081c lib: fix missing includes in exported headers
Many exported headers rely on definitions found in rte_config.h without
including it, as shown by the following command:

 grep -L '^#include <rte_config.h>' -- \
  $(grep -Rl \
    $(sed -n '/^#define \([^ ]\+\).*$/{s//\1/;H;};${x;s/\n//;s/\n/\\|/g;p;}' \
      build/include/rte_config.h) \
    -- build/include/)

We cannot assume external applications will include rte_config.h on their
own, neither directly nor through a -include parameter like DPDK does
internally.

This not only causes obvious compilation failures that can be reproduced
with check-includes.sh such as:

 [...]/rte_memory.h:88:43: error: ‘RTE_CACHE_LINE_SIZE’ was not declared in
     this scope
  #define __rte_cache_aligned __rte_aligned(RTE_CACHE_LINE_SIZE)
                                            ^

It also results in less visible issues, for instance rte_hash_crc.h relying
on RTE_ARCH_X86_64's presence to provide dedicated inline functions.

This patch partially reverts the commit below and adds missing include
lines to the remaining files.

Fixes: f1a7a5c5f4 ("remove include of generated config header")
Cc: stable@dpdk.org

Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
2018-01-17 00:31:05 +01:00
Zhiyong Yang
134975073a lib: remove unnecessary pointer cast
void * pointer can be assigned to any data type pointer.
Unnecessary cast can be removed in order to keep code clearer.

Signed-off-by: Zhiyong Yang <zhiyong.yang@intel.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
2018-01-16 01:53:35 +01:00
Jerin Jacob
baf3d63c15 lib: use SPDX tag for Cavium copyright files
Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
2018-01-09 16:19:01 +01:00
Bruce Richardson
369991d997 lib: use SPDX tag for Intel copyright files
Replace the BSD license header with the SPDX tag for files
with only an Intel copyright on them.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2018-01-04 22:41:39 +01:00
Thomas Monjalon
4c00cfdc0e remove useless memzone includes
The memzone header is often included without good reason.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2017-11-06 22:12:08 +01:00
Thomas Monjalon
17715a5339 use macro to declare constructor functions
It is easier to find all constructor functions when they use
the same macros RTE_INIT or RTE_INIT_PRIO.

The macro definitions are moved from rte_eal.h to rte_common.h.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2017-11-06 21:56:07 +01:00
Olivier Matz
cbc12b0a96 mk: do not generate LDLIBS from directory dependencies
The list of libraries in LDLIBS was generated from the DEPDIRS-xyz
variable. This is valid when the subdirectory name match the library
name, but it's not always the case, especially for PMDs.

The patches removes this feature and explicitly adds the proper
libraries in LDLIBS.

Some DEPDIRS-xyz variables become useless, remove them.

Reported-by: Gage Eads <gage.eads@intel.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Reviewed-by: Gage Eads <gage.eads@intel.com>
2017-10-24 02:14:57 +02:00
Yangchao Zhou
3fb1ea032b hash: optimize Toeplitz RSS computation
Use rte_bsf32 and fast bit unset operation to optimize the
softrss computation.
The following measurements shows improvement over the default
softrss computation function.

tuple lens old(cycles) new(cycles)
    3        1225         337
    9        3743         992

Signed-off-by: Yangchao Zhou <zhouyates@gmail.com>
Reviewed-by: Vladimir Medvedkin <medvedkinv@gmail.com>
2017-10-07 13:50:43 +02:00
Pablo de Lara
98b8ec7060 hash: fix eviction counter
When adding a new entry in a hash table, there is
a maximum number of evictions that can be
performed. When the counter of these evictions reaches
this maximum, the entry cannot be added, as it is considered
that the algorithm has encountered an infinite loop.

The problem with the current implementation, is that this
counter was declared as a static variable.
If there are multiple threads adding entries in the same table
or in different tables, they should access different counters,
one per core and per table.

Therefore, the variable has been modified to be non-static.

Fixes: 243e93a504 ("hash: fix unlimited cuckoo path")
Cc: stable@dpdk.org

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2017-10-07 13:50:43 +02:00
Chas Williams
0c57f40e66 hash: document jhash boundary behavior
Due to the uint32_t accesses in the hash computation, keys that aren't
aligned to a uint32_t boundary or multiples of uint32_t in length, may
see accesses beyond the end of the key.  This may cross a page boundary.

Signed-off-by: Chas Williams <chas3@att.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: John McNamara <john.mcnamara@intel.com>
2017-10-05 23:37:52 +02:00
Stephen Hemminger
d24b29d167 lib: remove duplicate includes
Include files only need to be refrenced once per file.

Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
2017-07-16 17:30:06 +02:00
Mike Stolarchuk
61d04efc38 hash: fix lock release on add
When adding items to a hash table with multiple threads,
there is an spinlock used to prevent data corruption
(unless Transactional Memory is supported).

If there is a failure, the spinlock should be released,
but there were cases where that was not happening.

Fixes: be856325cb ("hash: add scalable multi-writer insertion with Intel TSX")
Cc: stable@dpdk.org

Signed-off-by: Mike Stolarchuk <mike.stolarchuk@bigswitch.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2017-07-08 19:00:22 +02:00
Jerin Jacob
3abcd29f2d update Cavium Inc copyright headers
Replace the incorrect reference to "Cavium Networks", "Cavium Ltd"
company name with correct the "Cavium, Inc" company name in
copyright headers.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
2017-07-08 17:43:49 +02:00
Bruce Richardson
4f4cd8717e hash: remove checks for SSE
Since SSE4 is now part of the minimum requirements for DPDK, we don't need
a fallback case to handle selection of algorithm when SSE4 is unavailable.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2017-07-04 14:35:41 +02:00
Jerin Jacob
577329e66b eal: switch to architecture specific pause function
Remove rte_pause() definition from rte_common.h and
switchover to architecture specific rte_pause.h

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
2017-07-03 23:58:51 +02:00
Ashwin Sekhar T K
fa50d3b27a hash: compile ARMv8 CRC32 support conditionally
Compile the armv8a CRC32 support only if the machine
has the CRC extensions i.e if RTE_MACHINE_CPUFLAG_CRC32
is defined.

Removed the .arch assembly directives as these are no
more necessary.

Signed-off-by: Ashwin Sekhar T K <ashwin.sekhar@caviumnetworks.com>
Reviewed-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
2017-07-03 22:27:42 +02:00
Ashwin Sekhar T K
6cba794e48 hash: support NEON for Toeplitz algo
Verified the changes with thash_autotest unit test case

Signed-off-by: Ashwin Sekhar T K <ashwin.sekhar@caviumnetworks.com>
Acked-by: Jianbo Liu <jianbo.liu@linaro.org>
2017-07-03 14:17:37 +02:00
Jerin Jacob
98a7ea332b fix typos using codespell utility
Fixing typos across dpdk source code using codespell utility.
Skipped the ethdev driver's base code fixes to keep the base
code intact.

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: John McNamara <john.mcnamara@intel.com>
2017-06-14 23:54:13 +02:00
Ferruh Yigit
d2e0260445 hash: fix icc build
build error with icc version 17.0.4 (gcc version 7.0.0 compatibility):

In file included from .../dpdk/lib/librte_hash/rte_fbk_hash.h(59),
                 from .../dpdk/lib/librte_hash/rte_fbk_hash.c(54):
.../dpdk/x86_64-native-linuxapp-icc/include/rte_hash_crc.h(480):
 error #1292: unknown attribute "fallthrough"
                __attribute__ ((fallthrough));
                                ^

In file included from .../dpdk/lib/librte_hash/rte_fbk_hash.h(59),
                 from .../dpdk/lib/librte_hash/rte_fbk_hash.c(54):
.../dpdk/x86_64-native-linuxapp-icc/include/rte_hash_crc.h(486):
 error #1292: unknown attribute "fallthrough"
                __attribute__ ((fallthrough));
                                ^
This code patch hit when gcc > 7 installed and ICC doesn't recognize
fallthrough attribute.

Fixed by disabling code when compiled with ICC.

Fixes: 3dfb9facb0 ("lib: add switch fall-through comments")

Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
2017-06-14 22:54:54 +02:00
Jerin Jacob
70361cb13d hash: add switch fall-through comments for arm64
This fixes compiler warnings with GCC 7 for arm64 build.
Fixes: da8dcc27f6 ("hash: use armv8-a CRC32 instructions")

Signed-off-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
2017-05-10 16:29:26 +02:00
Bruce Richardson
3dfb9facb0 lib: add switch fall-through comments
With GCC 7 we need to explicitly document when we are falling through from
one switch case to another.

Fixes: af75078fec ("first public release")
Fixes: 8bae1da2af ("hash: fallback to software CRC32 implementation")
Fixes: 9ec201f5d6 ("mbuf: provide bulk allocation")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
2017-05-05 18:36:55 +02:00
Bruce Richardson
ecaed092b6 ring: return remaining entry count when dequeuing
Add an extra parameter to the ring dequeue burst/bulk functions so that
those functions can optionally return the amount of remaining objs in the
ring. This information can be used by applications in a number of ways,
for instance, with single-consumer queues, it provides a max
dequeue size which is guaranteed to work.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
2017-03-29 22:32:20 +02:00
Bruce Richardson
14fbffb0aa ring: return free space when enqueuing
Add an extra parameter to the ring enqueue burst/bulk functions so that
those functions can optionally return the amount of free space in the
ring. This information can be used by applications in a number of ways,
for instance, with single-producer queues, it provides a max
enqueue size which is guaranteed to work. It can also be used to
implement watermark functionality in apps, replacing the older
functionality with a more flexible version, which enables apps to
implement multiple watermark thresholds, rather than just one.

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
2017-03-29 22:32:04 +02:00
Olivier Matz
feb9f680cd mk: optimize directory dependencies
Before this patch, the management of dependencies between directories
had several issues:

- the generation of .depdirs, done at configuration is slow: it can take
  more than one minute on some slow targets (usually ~10s on a standard
  PC without -j).

- for instance, it is possible to express a dependency like:
  - app/foo depends on lib/librte_foo
  - and lib/librte_foo depends on app/bar
  But this won't work because the directories are traversed with a
  depth-first algorithm, so we have to choose between doing 'app' before
  or after 'lib'.

- the script depdirs-rule.sh is too complex.

- we cannot use "make -d" for debug, because the output of make is used for
  the generation of .depdirs.

This patch moves the DEPDIRS-* variables in the upper Makefile, making
the dependencies much easier to calculate. A DEPDIRS variable is still
used to process library dependencies in LDLIBS.

After this commit, "make config" is almost immediate.

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Tested-by: Robin Jarry <robin.jarry@6wind.com>
Tested-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
2017-03-27 23:28:43 +02:00
Pablo de Lara
94eaebad74 hash: fix bucket size usage
Multiwriter insert function was using a fixed value for
the bucket size, instead of using the
RTE_HASH_BUCKET_ENTRIES macro, which value was changed
recently (making it inconsistent in this case).

Fixes: be856325cb ("hash: add scalable multi-writer insertion with Intel TSX")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-10-12 18:40:52 +02:00
Pablo de Lara
243e93a504 hash: fix unlimited cuckoo path
When trying to insert a new entry, if its target bucket is full,
the alternative location (bucket) of one of the entries is checked,
to try to find an empty slot, with make_space_bucket.
This function is called every time a new bucket is checked, recursively.
To avoid having a very long insert operation (and to avoid filling up
the stack), a limit in the number of pushes is introduced.

Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-10-12 18:40:51 +02:00
Byron Marohn
ff15d9c0ba hash: modify lookup bulk pipeline
This patch replaces the pipelined rte_hash lookup mechanism with a
loop-and-jump model, which performs significantly better,
especially for smaller table sizes and smaller table occupancies.

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Sameh Gobriel <sameh.gobriel@intel.com>
2016-10-05 12:10:49 +02:00
Byron Marohn
58017c98ed hash: add vectorized comparison
In lookup bulk function, the signatures of all entries
are compared against the signature of the key that is being looked up.
Now that all the signatures are together, they can be compared
with vector instructions (SSE, AVX2), achieving higher lookup performance.

Also, entries per bucket are increased to 8 when using processors
with AVX2, as 256 bits can be compared at once, which is the size of
8x32-bit signatures.

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Sameh Gobriel <sameh.gobriel@intel.com>
2016-10-05 12:09:50 +02:00
Byron Marohn
8a9f542f32 hash: reorganize bucket structure
Move current signatures of all entries together in the bucket
and same with all alternative signatures, instead of having
current and alternative signatures together per entry in the bucket.
This will be benefitial in the next commits, where a vectorized
comparison will be performed, achieving better performance.

The alternative signatures have been moved away from
the current signatures, to make the key indices be consecutive
to the current signatures, as these two fields are used by lookup,
so they are in the same cache line.

Signed-off-by: Byron Marohn <byron.marohn@intel.com>
Signed-off-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Sameh Gobriel <sameh.gobriel@intel.com>
2016-10-05 12:08:56 +02:00
Pablo de Lara
02a08eb355 hash: reorder hash structure
In order to optimize lookup performance, hash structure
is reordered, so all fields used for lookup will be
in the first cache line.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Sameh Gobriel <sameh.gobriel@intel.com>
2016-10-05 12:08:04 +02:00
Pablo de Lara
5230bc4c77 hash: fix free slot check
In function rte_hash_cuckoo_insert_mw_tm, while looking for
an empty slot, only the first entry in the bucket was being checked,
as key_idx array was not being iterated.

Fixes: 5fc74c2e14 ("hash: check if slot is empty with key index")

Reported-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2016-10-04 11:40:40 +02:00
Pablo de Lara
5fc74c2e14 hash: check if slot is empty with key index
Instead of checking if the current and alternative signatures are 0,
it is faster to check if the key index associated to an entry
is 0, meaning that the slot is empty.

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2016-09-29 21:51:27 +02:00
Pablo de Lara
24c20a7221 hash: fix false zero signature key hit lookup
This commit fixes a corner case scenario. When a key is deleted,
its signature in the hash table gets clear, which should prevent
a lookup of that same key, unless the signature of the key is all zeroes.

In that case, there will be a match, and key would be compared against
the key that is in the table (which does not get cleared,
as the performance penalty would be high), resulting in a wrong hit.

To prevent this from happening, the key index associated to that entry
should be set to zero when deleting it, so in case that same key
is looked up just after a deletion, it will point to the dummy key slot,
which guarantees a miss.

Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2016-09-29 21:50:32 +02:00
Pablo de Lara
1621f69abb hash: fix ring size
Ring stores the free slots available to be used in the key table.
The ring size was being increased by 1, because of the dummy slot,
used for key misses, but this is not actually stored in the ring,
so there is no need to increase it.

Fixes: 5915699153 ("hash: fix scaling by reducing contention")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Saikrishna Edupuganti <saikrishna.edupuganti@intel.com>
2016-09-29 21:48:25 +02:00
Masoud Hasanifard
8331c64847 hash: fix custom compare
Set cmp_jump_table_idx to KEY_CUSTOM in rte_hash_cmp_eq so that the custom
function we are setting in rte_hash_set_cmp_func properly works. The custom
function is only called by rte_hash_cmp_eq if cmp_jump_table_idx is set to
KEY_CUSTOM.

Fixes: 95da2f8e9c ("hash: customize compare function")

Signed-off-by: Masoud Hasanifard <masoudhasanifard@gmail.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-09-22 17:38:17 +02:00
Adrien Mazarguil
79d6f5fc58 lib: work around unnamed structs/unions
Exported header files used by applications should allow the strictest
compiler flags. Language extensions used in many places must be explicitly
marked to avoid warnings and compilation failures.

Unnamed structs/unions are allowed since C11, however many compiler
versions do not use this mode by default.

This commit prevents the following errors:

 error: ISO C99 doesn't support unnamed structs/unions
 error: struct has no named members

Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
2016-09-13 15:35:28 +02:00
Adrien Mazarguil
347a1e037f lib: use C99 syntax for zero-size arrays
Exported header files used by applications should allow the strictest
compiler flags. Language extensions used in many places must be explicitly
marked or removed to avoid warnings and compilation failures.

The extension keyword is used whenever the C99 syntax cannot do it.

This commit prevents the following errors:

 error: ISO C forbids zero-size array `[...]'

Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
2016-09-13 15:35:28 +02:00
Yari Adan Petralanda
6dc34e0afe hash: retrieve a key given its position
The function rte_hash_get_key_with_position is added in this patch.
As the position returned when adding a key is frequently used as an
offset into an array of user data, this function performs the operation
of retrieving a key given this offset.

A possible use case would be to delete a key from the hash table when
its entry in the array of data has certain value. For instance, the key
could be a flow 5-tuple, and the value stored in the array a time
stamp.

Signed-off-by: Juan Antonio Montesinos <juan.antonio.montesinos.delgado@ericsson.com>
Signed-off-by: Yari Adan Petralanda <yari.adan.petralanda@ericsson.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-07-10 14:56:45 +02:00
Wei Shen
be856325cb hash: add scalable multi-writer insertion with Intel TSX
This patch introduced scalable multi-writer Cuckoo Hash insertion
based on a split Cuckoo Search and Move operation using Intel
TSX. It can do scalable hash insertion with 22 cores with little
performance loss and negligible TSX abortion rate.

* Added an extra rte_hash flag definition to switch default single writer
  Cuckoo Hash behavior to multiwriter.
    - If HTM is available, it would use hardware feature for concurrency.
    - If HTM is not available, it would fall back to spinlock.

* Created a rte_cuckoo_hash_x86.h file to hold all x86-arch related
  cuckoo_hash functions. And rte_cuckoo_hash.c uses compile time flag to
  select x86 file or other platform-specific implementations. While HTM check
  is still done at runtime (same idea with
  RTE_HASH_EXTRA_FLAGS_TRANS_MEM_SUPPORT)

* Moved rte_hash private struct definitions to rte_cuckoo_hash.h, to allow
  rte_cuckoo_hash_x86.h or future platform dependent functions to include.

* Following new functions are created for consistent names when new platform
  TM support are added.
    - rte_hash_cuckoo_move_insert_mw_tm: do insertion with bucket movement.
    - rte_hash_cuckoo_insert_mw_tm: do insertion without bucket movement.

* One extra multi-writer test case is added.

Signed-off-by: Wei Shen <wei1.shen@intel.com>
Signed-off-by: Sameh Gobriel <sameh.gobriel@intel.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-06-24 16:25:07 +02:00
Thomas Monjalon
656ecbe93c hash: remove aliases for bulk lookup
The function rte_hash_lookup_multi() was renamed rte_hash_lookup_bulk()
in DPDK 1.4 and was kept as an undocumented alias.

Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
2016-04-20 13:49:36 +02:00
Olivier Matz
5d7bfb7337 hash: fix race condition at creation
To avoid a race condition while creating a new hash object, the
list has to be locked before the lookup, and released only once the
new object is added in the list.

As the lock is held by the rte_ring_create(), move its creation at the
beginning of the function and only take the lock after the ring is
created to avoid a deadlock.

Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-04-06 17:30:51 +02:00
Olivier Matz
1aadacb5b0 hash: fix allocation of an existing object
Change rte_hash*_create() functions to return NULL and set rte_errno to
EEXIST when the object name already exists. This is the behavior
described in the API documentation in the header file.

These functions were returning a pointer to the existing object in that
case, but it is a problem as the caller did not know if the object had
to be freed or not.

Doing this change also makes the hash API more consistent with the other
APIs (mempool, rings, ...).

Fixes: 916e4f4f4e ("memory: fix for multi process support")

Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
2016-04-06 17:30:48 +02:00
Pablo de Lara
f9bd334211 hash: fix multi-process support
Hash library used a function pointer to choose a different
key compare function, depending on the key size.
As a result, multiple processes could not use the same hash table,
as the function addresses vary from one process to another.

Instead, a jump table is used, so each process has its own
function addresses, accessing this table with an index stored
in the hash table (note that using a custom key compare function
is not supported in multi-process mode).

Fixes: 48a3991196 ("hash: replace with cuckoo hash implementation")

Signed-off-by: Pablo de Lara <pablo.de.lara.guarch@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
2016-04-01 18:56:27 +02:00