Although the physical address won't be correct in memory segment,
this allows at least to retrieve the physical address using
rte_mem_virt2phy(). Indeed, if the page is not locked, the page
may not be present in physical memory.
With next commit, it allows a mempool to have properly filled physical
addresses when using --no-huge option.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce rte_mempool_populate_default() which allocates
mempool objects in several memzones.
The mempool header is now always allocated in a specific memzone
(not with its objects). Thanks to this modification, we can remove
many specific behavior that was required when hugepages are not
enabled in case we are using rte_mempool_xmem_create().
This change requires to update how kni and mellanox drivers lookup for
mbuf memory. For now, this will only work if there is only one memory
chunk (like today), but we could make use of rte_mempool_mem_iter() to
support more memory chunks.
We can also remove RTE_MEMPOOL_OBJ_NAME that is not required anymore for
the lookup, as memory chunks are referenced by the mempool.
Note that rte_mempool_create() is still broken (it was the case before)
when there is no hugepages support (rte_mempool_create_xmem() has to be
used). This is fixed in next commit.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Update rte_mempool_xmem_size() so that when the page_shift argument is
set to 0, assume that memory is physically contiguous, allowing to
ignore page boundaries. This will be used in the next commits.
By the way, rename the variable 'n' as 'obj_per_page' and avoid the
affectation inside the if().
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a free callback that is passed to the populate* functions,
which is used when freeing a mempool. This is unused now, but as next
commits will populate the mempool with several chunks of memory, we
need a way to free them properly on error.
Later in the series, we will also introduce a public rte_mempool_free()
and the ability for the user to populate a mempool with its own memory.
For that, we also need a free callback.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit simplifies rte_mempool_xmem_usage().
Since previous commit, the function rte_mempool_xmem_usage() is
now the last user of rte_mempool_obj_mem_iter(). This complex
code can now be moved inside the function. We can get rid of the
callback and do some simplification to make the code more readable.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
In the same model than rte_mempool_obj_iter(), introduce
rte_mempool_mem_iter() to iterate the memory chunks attached
to the mempool.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Do not use paddr table to store the mempool memory chunks.
This will allow to have several chunks with different virtual addresses.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes MEMPOOL_IS_CONTIG().
The next commits will change the behavior of the mempool library so that
the objects will never be allocated in the same memzone than the mempool
header. Therefore, there is no reason to keep this macro that would
always return 0.
This macro was only used in app/test.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Store the physical address of the object in its header. It simplifies
rte_mempool_virt2phy() and prepares the removing of the paddr[] table
in the mempool header.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This makes the code of rte_mempool_create() clearer, and it will make
the introduction of external mempool handler easier (in another patch
series). Indeed, this function contains the specific part when a ring is
used, but it could be replaced by something else in the future.
This commit also adds a socket_id field in the mempool structure that
is used by this new function.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Before this patch, the mempool elements were initialized at the time
they were added to the mempool. This patch changes this to do the
initialization of all objects once the mempool is populated, using
rte_mempool_obj_iter() introduced in previous commits.
Thanks to this modification, we are getting closer to a new API
that would allow us to do:
mempool_init()
mempool_populate(mem1)
mempool_populate(mem2)
mempool_populate(mem3)
mempool_init_obj()
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Use the new rte_mempool_obj_iter() instead the old rte_mempool_obj_iter()
to iterate among objects to audit them (check for cookies).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Now that the mempool objects are chained into a list, we can use it to
browse them. This implies a rework of rte_mempool_obj_iter() API, that
does not need to take as many arguments as before. The previous function
is kept as a private function, and renamed in this commit. It will be
removed in a next commit of the patch series.
The only internal users of this function are the mellanox drivers. The
code is updated accordingly.
Introducing an API compatibility for this function has been considered,
but it is not easy to do without keeping the old code, as the previous
function could also be used to browse elements that were not added in a
mempool. Moreover, the API is already be broken by other patches in this
version.
The library version was already updated in
commit 213af31e0960 ("mempool: reduce structure size if no cache needed")
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
In next commits, we will use an iterator to walk through the objects in
mempool in rte_mempool_audit(). This iterator takes a "struct
rte_mempool *" as a parameter because it is assumed that the callback
function can modify the mempool.
The previous approach was to introduce a RTE_DECONST() macro, but
after discussion it seems that removing the const qualifier is better
to avoid fooling the compiler, and also because these functions are
not used in datapath (possible compiler optimizations due to const
are not critical).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit removes the const qualifier for the mempool in
rte_mempool_walk() callback prototype.
Indeed, most functions that can be done on a mempool require a non-const
mempool pointer, except the dump and the audit. Therefore, the
mempool_walk() is more useful if the mempool pointer is not const.
This is required by next commit where the mellanox drivers use
rte_mempool_walk() to iterate the mempools, then rte_mempool_obj_iter()
to iterate the objects in each mempool.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Introduce a list entry in object header so they can be listed and
browsed. The objective is to introduce a more simple way to browse the
elements of a mempool.
The next commits will update rte_mempool_obj_iter() to use this list,
and remove the previous complex implementation.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit renames mempool_obj_ctor_t as mempool_obj_cb_t.
In next commits, we will add the ability to populate the
mempool and iterate through objects using the same function.
We will use the same callback type for that. As the callback is
not a constructor anymore, rename it into rte_mempool_obj_cb_t.
The rte_mempool_obj_iter_t that was used to iterate over objects
will be removed in next commits.
No functional change.
In this commit, the API is preserved through a compat typedef.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Since commits d2e0ca22f and 97e7e685b the headers and trailers
of the mempool are defined as a structure. We can get their
size using a sizeof instead of doing a calculation that will
become wrong at the first structure update.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
There's no reason to keep this function inlined. Move it to
rte_mempool.c. We need to export the function for when compiling
with shared libraries + debug. We also need to keep the macro,
because we don't want to call an empty function when debug is
disabled.
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
This commit replaces elt_size by total_elt_size when appropriate.
In some mempool functions, we use the size of the elements as arguments
or variables. There is a confusion between the size including or not
including the header and trailer.
To avoid this confusion:
- update the API documentation
- rename the variables and argument names as "elt_size" when the size
does not include the header and trailer, or else as "total_elt_size".
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked by: Keith Wiles <keith.wiles@intel.com>
No functional change, just fix some comments and styling issues.
Also avoid to duplicate comments between rte_mempool_create()
and rte_mempool_xmem_create().
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked by: Keith Wiles <keith.wiles@intel.com>
The rte_pktmbuf_detach() function should decrease refcnt on a direct
buffer as stated in doc/guides/prog_guide/mbuf_lib.rst:
"whenever the indirect buffer is detached, the reference counter on the
direct buffer is decremented."
Signed-off-by: Hiroyuki Mikita <h.mikita89@gmail.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The value returned by rdline_init() was not checked in cmdline_new().
On error, free the allocated memory and return NULL.
This condition should not happen today, but it's safer to do the check
in case rdline_init() is updated.
Coverity issue: 13204
Fixes: af75078fece3 ("first public release")
Signed-off-by: Marcin Kerlin <marcinx.kerlin@intel.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
The functions rte_log_cur_msg_loglevel() and rte_log_cur_msg_logtype()
return the current log level/type for the message being processed. They
are used when implementing a user-defined logging stream.
The current log levels and types were stored in a table indexed by the
lcore_id, only returning a valid value for dataplane threads. Setting
and getting these values in a non dataplane thread was ignored, using
the global value instead.
To fix this issue, a per-thread variable could be used (with
RTE_DEFINE_PER_LCORE), allowing any pthread to set and retrieve its
current log level or type.
Signed-off-by: Maxime Leroy <maxime.leroy@6wind.com>
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
The rte_mempool structure is changed, which will cause an ABI change
for this structure. Providing backward compat is not reasonable
here as this structure is used in multiple defines/inlines.
Allow mempool cache support to be dynamic depending on if the
mempool being created needs cache support. Saves about 1.5M of
memory used by the rte_mempool structure.
Allocating small mempools which do not require cache can consume
larges amounts of memory if you have a number of these mempools.
Signed-off-by: Keith Wiles <keith.wiles@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Many drivers provide their own implementation of rte_mbuf_raw_alloc(),
duplicating the code. Introduce a new public function in rte_mbuf to
allocate a raw mbuf (uninitialized).
Signed-off-by: Olivier Matz <olivier.matz@6wind.com>
Function strerror(errno) has built strings only for non-negative errno values.
for negative values of errno it describe error as "Unknown error -errno"
to be more descriptive i put string "channel not found" taken from header.
The negative argument will be interpreted as a very large unsigned value.
Coverity issue: 13266
Coverity issue: 13269
Fixes: 445c6528b55f ("power: common interface for guest and host")
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Fix issue reported by Coverity.
Coverity ID 124556
If the buffer is treated as a null terminated string in later operations,
a buffer overflow or over-read may occur.
In vhost_set_ifname: The string buffer may not have a null terminator if
the source string's length is equal to the buffer size
Fixes: 54292e9520e0 ("vhost: support ifname for vhost-user")
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Acked-by: Yuanhan Liu <yuanhan.liu@linux.intel.com>
The header file rte_debug.h makes use of the "unlikely" macro which
means it should include the rte_branch_prediction.h header file.
Fixes: 50705e8e3cdd ("eal: add assert macro for debug")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
The macro RTE_VERIFY always checks a condition.
It is optimized with "unlikely" hint.
While this macro is well suited for test applications, it is preferred
in libraries and examples to enable such check in debug mode.
That's why the macro RTE_ASSERT is introduced to call RTE_VERIFY only
if built with debug logs enabled.
A lot of assert macros were duplicated and enabled with a specific flag.
Removing these #ifdef allows to test these code branches more easily
and avoid dead code pitfalls.
The ENA_ASSERT is kept (in debug mode only) because it has more
parameters to log.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
The default was to compile every logs (including debug) and set
the default level to debug.
As some debug logs may hurt performance, a notice is added and the
default level is now info.
In order to enable debug logs, they must be compiled with
RTE_LOG_LEVEL=RTE_LOG_DEBUG and enabled at runtime with --log-level=8.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Some relevant EAL logs are raised from debug to info level in order to
show the available number of cores and the detected devices.
The driver ids and name are logged only if the device is not blacklisted.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
While parsing token string there may be several modes:
- fixed single string
- multi-choice single string
- any single string
This patch add one more mode - any multi string.
Signed-off-by: Piotr Azarewicz <piotrx.t.azarewicz@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Fix issue reported by clang scan-build
Value of pointer tbl_next was uninitialized. When function lookup_step()
take else branch it may provide garbage into tbl = tbl_next;
Fixes: 5c510e13a9cb ("lpm: add IPv6 support")
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Back then when we fixed the missing free lpm I was to quickly to say yes
if it applies not only to the lpm6 but also to all of the lpm code.
It turned out to not apply to all of them. In rte_lpm_create_v20 there
is an unexpected fused allocation:
mem_size = sizeof(*lpm) + (sizeof(lpm->rules_tbl[0]) * max_rules);
[...]
lpm = (struct rte_lpm_v20 *)rte_zmalloc_socket(mem_name,mem_size,
RTE_CACHE_LINE_SIZE, socket_id);
That causes lpm->rules_tbl not to have an own struct malloc_elem that
can be derived via RTE_PTR_SUB(data, MALLOC_ELEM_HEADER_LEN) in
malloc_elem_from_data.
Due to that the rte_lpm_free_v20 accidentially misderives the elem and
assumes it is ELEM_FREE triggering in malloc_elem_free
if (!malloc_elem_cookies_ok(elem) || elem->state !=
return -1;
While it seems counter-intuitive the way to properly remove rules_tbl in
the old fused allocation style of rte_lpm_free_v20 is to not remove it.
The newer rte_lpm_free_v1604 is safe because in rte_lpm_create_v1604
rules_tbl is a separate allocation.
Fixes: d4c18f0a1d5d ("lpm: fix missing free")
Signed-off-by: Christian Ehrhardt <christian.ehrhardt@canonical.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
If any fragment hole is found in ipv4_frag_reassemble() and
ipv6_frag_reassemble(), whole ip_frag_pkt mbufs are moved to death-row.
Any mbufs already chained to another mbuf are freed multiple times as
there are still in ip_frag_pkt array.
Signed-off-by: Chaeyong Chong <cychong@gmail.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Although previous implementation returned an error when trying to release
a memzone assigned to an ivshmem device, it stills freed it.
Fixes: cd10c42eb5bc ("mem: fix ivshmem freeing")
Signed-off-by: Mauricio Vasquez B <mauricio.vasquezbernal@studenti.polito.it>
Acked-by: Sergio Gonzalez Monroy <sergio.gonzalez.monroy@intel.com>
rx_q fifo may have chained mbufs, merge them into single skb before
handing to the network stack.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Currently every time a KNI interface goes up, its ethernet address
is reassigned.
After this patch ethernet address is assigned only once,
at initialization time.
Suggested-by: Sergey Balabanov <balabanovsv@ecotelecom.ru>
Signed-off-by: Igor Ryzhov <iryzhov@nfware.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Fix issue reported by Coverity.
Coverity ID 13194
The function returns a value that indicates an error condition. If this
is not checked, the error condition may not be handled correctly.
Fixes: 2f4adfad0a69 ("vfio: add multiprocess support")
Signed-off-by: Daniel Mrzyglod <danielx.t.mrzyglod@intel.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
The function rte_memcpy_func() is used in ARM and PPC implementations
of rte_memcpy().
There are some useless copies in Tile and some ARM branches.
It was also declared without doxygen comment in the generic header.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
After having removed the deprecated stuff, we can start pushing
new fixes and features in the version 16.07.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Some flags were poisoned after having been removed from EAL and mbuf
in releases 1.8 (b10eef348d, 62814bc2e9) and 2.0 (4769bc5a27cc).
After several releases, they have probably disappeared from all
applications going to upgrade to DPDK 16.07.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: David Marchand <david.marchand@6wind.com>
The function rte_hash_lookup_multi() was renamed rte_hash_lookup_bulk()
in DPDK 1.4 and was kept as an undocumented alias.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Some statistics were deprecated since release 2.1 (49f386542af4).
The last deprecated counter to be used was imcasts.
The VF loopback statistics are also removed as they are used only
in igb and duplicated in extended statistics.
The new counters should be added to extended statistics.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Remy Horton <remy.horton@intel.com>
The driver i40e was using a specific PCI config before the release 16.04.
Since 16.04, it is always enabled in i40e (commit 56465cfaf).
The API has been deprecated in the commit 68f77593823cab.
The igb_uio implementation has been deprecated in commit b7cf8e155.
The config helper - through igb_uio sysfs entries - is now removed.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Helin Zhang <helin.zhang@intel.com>
Acked-by: David Marchand <david.marchand@6wind.com>
Support of PCAP file has been added to rte_port in release 16.04
as NEXT_ABI. It is in the standard ABI of the release 16.07.
Signed-off-by: Thomas Monjalon <thomas.monjalon@6wind.com>
Acked-by: Cristian Dumitrescu <cristian.dumitrescu@intel.com>
