The mempool local cache was not initialized properly leading to
undefined behavior in cases where the allocated memory was used
previously and left with data.
Fixes: 213af31e09 ("mempool: reduce structure size if no cache needed")
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The log history uses rte_mempool. In order to remove the mempool
dependency in EAL (and improve the build), this feature is deprecated.
The ABI is kept but the behaviour is now voided because it seems this
function was not used. The history can be read from syslog.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
In function call rte_mempool_xmem_size, division by expression total_size
which may be zero has undefined behavior.
Coverity issue: 13243
Fixes: 148f963fb5 ("xen: core library changes")
Signed-off-by: Slawomir Mrozowicz <slawomirx.mrozowicz@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Add a new flag to remove the constraint of having physically contiguous
objects inside a mempool.
Add this flag to the log history mempool to start, but we could add
it in most cases where objects are not mbufs.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Check the return value of snprintf to ensure that the name of
the object is not truncated.
By the way, update the test to avoid to trigger an error in
that case.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Now that we can populate a mempool with any virtual memory,
it is easier to introduce a function to populate a mempool
with memory coming from an anonymous mapping, as it's done
in test-pmd.
The next commit will replace test-pmd anonymous mapping by
this function.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Instead of creating the internal ring at mempool creation, do
it when populating the mempool with the first memory chunk. The
objective here is to simplify the change of external handler
when it will be introduced.
For instance, this will be possible:
mp = rte_mempool_create_empty(...)
rte_mempool_set_ext_handler(mp, my_handler)
rte_mempool_populate_default()
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Avoid to have a specific file for that, and remove #ifdefs.
Now that we have introduced a function to populate a mempool
with a virtual area, the support of xen dom0 is much easier.
The only thing we need to do is to convert the guest physical
address into the machine physical address using rte_mem_phy2mch().
This function does nothing when not running xen.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a new function rte_mempool_create_empty()
that allocates a mempool that is not populated.
The functions rte_mempool_create() and rte_mempool_xmem_create()
now make use of it, making their code much easier to read.
Currently, they are the only users of rte_mempool_create_empty()
but the function will be made public in next commits.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce rte_mempool_free() that:
- unlink the mempool from the global list if it is found
- free all the memory chunks using their free callbacks
- free the internal ring
- free the memzone containing the mempool
Currently this function is only used in error cases when
creating a new mempool, but it will be made public later
in the patch series.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Storing the pointer to the memzone instead of the physical address
provides more information than just the physical address: for instance,
the memzone flags.
Moreover, keeping the memzone pointer will allow us to free the mempool
(this is done later in the series).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a new function rte_mempool_populate_virt() that is now called
by default when hugepages are not supported. This function populate the
mempool with several physically contiguous chunks whose minimum size is
the page size of the system.
Thanks to this, rte_mempool_create() will work properly in without
hugepages (if the object size is smaller than a page size), and 2
specific workarouds can be removed:
- trailer_size was artificially extended to a page size
- rte_mempool_virt2phy() did not rely on object physical address
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce rte_mempool_populate_default() which allocates
mempool objects in several memzones.
The mempool header is now always allocated in a specific memzone
(not with its objects). Thanks to this modification, we can remove
many specific behavior that was required when hugepages are not
enabled in case we are using rte_mempool_xmem_create().
This change requires to update how kni and mellanox drivers lookup for
mbuf memory. For now, this will only work if there is only one memory
chunk (like today), but we could make use of rte_mempool_mem_iter() to
support more memory chunks.
We can also remove RTE_MEMPOOL_OBJ_NAME that is not required anymore for
the lookup, as memory chunks are referenced by the mempool.
Note that rte_mempool_create() is still broken (it was the case before)
when there is no hugepages support (rte_mempool_create_xmem() has to be
used). This is fixed in next commit.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Update rte_mempool_xmem_size() so that when the page_shift argument is
set to 0, assume that memory is physically contiguous, allowing to
ignore page boundaries. This will be used in the next commits.
By the way, rename the variable 'n' as 'obj_per_page' and avoid the
affectation inside the if().
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a free callback that is passed to the populate* functions,
which is used when freeing a mempool. This is unused now, but as next
commits will populate the mempool with several chunks of memory, we
need a way to free them properly on error.
Later in the series, we will also introduce a public rte_mempool_free()
and the ability for the user to populate a mempool with its own memory.
For that, we also need a free callback.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit simplifies rte_mempool_xmem_usage().
Since previous commit, the function rte_mempool_xmem_usage() is
now the last user of rte_mempool_obj_mem_iter(). This complex
code can now be moved inside the function. We can get rid of the
callback and do some simplification to make the code more readable.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
In the same model than rte_mempool_obj_iter(), introduce
rte_mempool_mem_iter() to iterate the memory chunks attached
to the mempool.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Do not use paddr table to store the mempool memory chunks.
This will allow to have several chunks with different virtual addresses.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes MEMPOOL_IS_CONTIG().
The next commits will change the behavior of the mempool library so that
the objects will never be allocated in the same memzone than the mempool
header. Therefore, there is no reason to keep this macro that would
always return 0.
This macro was only used in app/test.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Store the physical address of the object in its header. It simplifies
rte_mempool_virt2phy() and prepares the removing of the paddr[] table
in the mempool header.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This makes the code of rte_mempool_create() clearer, and it will make
the introduction of external mempool handler easier (in another patch
series). Indeed, this function contains the specific part when a ring is
used, but it could be replaced by something else in the future.
This commit also adds a socket_id field in the mempool structure that
is used by this new function.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Before this patch, the mempool elements were initialized at the time
they were added to the mempool. This patch changes this to do the
initialization of all objects once the mempool is populated, using
rte_mempool_obj_iter() introduced in previous commits.
Thanks to this modification, we are getting closer to a new API
that would allow us to do:
mempool_init()
mempool_populate(mem1)
mempool_populate(mem2)
mempool_populate(mem3)
mempool_init_obj()
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Use the new rte_mempool_obj_iter() instead the old rte_mempool_obj_iter()
to iterate among objects to audit them (check for cookies).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Now that the mempool objects are chained into a list, we can use it to
browse them. This implies a rework of rte_mempool_obj_iter() API, that
does not need to take as many arguments as before. The previous function
is kept as a private function, and renamed in this commit. It will be
removed in a next commit of the patch series.
The only internal users of this function are the mellanox drivers. The
code is updated accordingly.
Introducing an API compatibility for this function has been considered,
but it is not easy to do without keeping the old code, as the previous
function could also be used to browse elements that were not added in a
mempool. Moreover, the API is already be broken by other patches in this
version.
The library version was already updated in
commit 213af31e09 ("mempool: reduce structure size if no cache needed")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
In next commits, we will use an iterator to walk through the objects in
mempool in rte_mempool_audit(). This iterator takes a "struct
rte_mempool *" as a parameter because it is assumed that the callback
function can modify the mempool.
The previous approach was to introduce a RTE_DECONST() macro, but
after discussion it seems that removing the const qualifier is better
to avoid fooling the compiler, and also because these functions are
not used in datapath (possible compiler optimizations due to const
are not critical).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes the const qualifier for the mempool in
rte_mempool_walk() callback prototype.
Indeed, most functions that can be done on a mempool require a non-const
mempool pointer, except the dump and the audit. Therefore, the
mempool_walk() is more useful if the mempool pointer is not const.
This is required by next commit where the mellanox drivers use
rte_mempool_walk() to iterate the mempools, then rte_mempool_obj_iter()
to iterate the objects in each mempool.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a list entry in object header so they can be listed and
browsed. The objective is to introduce a more simple way to browse the
elements of a mempool.
The next commits will update rte_mempool_obj_iter() to use this list,
and remove the previous complex implementation.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit renames mempool_obj_ctor_t as mempool_obj_cb_t.
In next commits, we will add the ability to populate the
mempool and iterate through objects using the same function.
We will use the same callback type for that. As the callback is
not a constructor anymore, rename it into rte_mempool_obj_cb_t.
The rte_mempool_obj_iter_t that was used to iterate over objects
will be removed in next commits.
No functional change.
In this commit, the API is preserved through a compat typedef.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Since commits d2e0ca22f and 97e7e685b the headers and trailers
of the mempool are defined as a structure. We can get their
size using a sizeof instead of doing a calculation that will
become wrong at the first structure update.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
There's no reason to keep this function inlined. Move it to
rte_mempool.c. We need to export the function for when compiling
with shared libraries + debug. We also need to keep the macro,
because we don't want to call an empty function when debug is
disabled.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit replaces elt_size by total_elt_size when appropriate.
In some mempool functions, we use the size of the elements as arguments
or variables. There is a confusion between the size including or not
including the header and trailer.
To avoid this confusion:
- update the API documentation
- rename the variables and argument names as "elt_size" when the size
does not include the header and trailer, or else as "total_elt_size".
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked by: Keith Wiles <keith.wiles@intel.com>
No functional change, just fix some comments and styling issues.
Also avoid to duplicate comments between rte_mempool_create()
and rte_mempool_xmem_create().
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked by: Keith Wiles <keith.wiles@intel.com>
The rte_mempool structure is changed, which will cause an ABI change
for this structure. Providing backward compat is not reasonable
here as this structure is used in multiple defines/inlines.
Allow mempool cache support to be dynamic depending on if the
mempool being created needs cache support. Saves about 1.5M of
memory used by the rte_mempool structure.
Allocating small mempools which do not require cache can consume
larges amounts of memory if you have a number of these mempools.
Signed-off-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The macro RTE_VERIFY always checks a condition.
It is optimized with "unlikely" hint.
While this macro is well suited for test applications, it is preferred
in libraries and examples to enable such check in debug mode.
That's why the macro RTE_ASSERT is introduced to call RTE_VERIFY only
if built with debug logs enabled.
A lot of assert macros were duplicated and enabled with a specific flag.
Removing these #ifdef allows to test these code branches more easily
and avoid dead code pitfalls.
The ENA_ASSERT is kept (in debug mode only) because it has more
parameters to log.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Since commits ff909fe21f and 4e32101f9b, it is now possible to free
memzones and rings.
The rte_mempool_create() should be modified to take advantage of this
and not leak memory when an allocation fails.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
fix the error reported by checkpatch:
"ERROR: return is not a function, parentheses are not required"
remove parentheses in return like:
"return (logical expressions)"
remove parentheses in return a function like:
"return (rte_mempool_lookup(...))"
Fixes: 6307b909b8 ("lib: remove extra parenthesis after return")
Signed-off-by: Huawei Xie <huawei.xie@intel.com>
The function rte_mempool_obj_iter used in mlx drivers
was not exported. So the driver loading was failing:
EAL: open shared lib librte_pmd_mlx4.so
EAL: x86_64-native-linuxapp-gcc/lib/librte_pmd_mlx4.so:
undefined symbol: rte_mempool_obj_iter
Fixes: 9d41beed24 ("lib: provide initial versioning")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
There is a new function in the EAL API for internal use.
It has neither a proper prefix nor a .map export:
libethdev.so: undefined reference to `is_xen_dom0_supported'
Fixes: 719dbebceb ("xen: allow determining DOM0 at runtime")
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Optimize for quad-channel by default, this should work well for
all the cases, better than the previous value of one anyway.
Suggested-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Panu Matilainen <pmatilai@redhat.com>
Acked-by: David Marchand <david.marchand@6wind.com>
When DPDK is being compiled in C++ project using g++ then
'invalid conversion from' error appears. Added explicit
typecast on function return to get rid of the error.
Fixes: 6cf14ce4ce ("mempool: silence warning on pointer arithmetic")
Signed-off-by: Sergey Balabanov <balabanovsv@ecotelecom.ru>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
__mempool_get_trailer() calculated header's address.
The address of trailer should set after element area.
This patch fixes this calculating.
Fixes: 97e7e685bf ("mempool: add structure for object trailers")
Signed-off-by: Yuichi Nakai <xoxyuxu@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Move malloc inside eal and create a new section in MAINTAINERS file for
Memory Allocation in EAL.
Create a dummy malloc library to avoid breaking applications that have
librte_malloc in their DT_NEEDED entries.
This is the first step towards using malloc to allocate memory directly
from memsegs. Thus, memzones would allocate memory through malloc,
allowing to free memzones.
Signed-off-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
On TILE-Gx and TILE-Mx platforms, the buffers fed into the hardware
buffer manager require a 128-byte alignment. With this change, we
allow configuration based override of the element alignment, and
default to RTE_CACHE_LINE_SIZE if left unspecified.
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
Signed-off-by: Zhigang Lu <zlu@ezchip.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Translating from a mempool object to the mempool pointer does not break
alignment constraints. However, the compiler is unaware of this fact and
complains on -Wcast-align. This patch modifies the code to use RTE_PTR_SUB(),
thereby silencing the compiler by casting through (void *).
Signed-off-by: Cyril Chemparathy <cchemparathy@ezchip.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Added Doxygen @param for missing API parameter in
rte_mempool_obj_iter(), to fix Doxygen warning. Also added
minor grammar fixes to that function documentation.
Signed-off-by: John McNamara <john.mcnamara@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Each object stored in mempools are suffixed by a trailer, storing
a cookie in debug mode which help to detect memory corruptions.
Like for headers, introduce a structure that materializes the content of
this trailer.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
