A warning is issued when using an argument to likely() or unlikely()
builtins which is evaluated to a pointer value, as __builtin_expect()
expects a 'long int' type for its first argument. With this fix
a pointer value is converted to an integer with the value of 0 or 1.
Signed-off-by: Aleksey Baulin <aleksey.baulin@gmail.com>
This patch provides an option to do rte_memcpy() using 'restrict'
qualifier, which can induce GCC to do optimizations by using more
efficient instructions, providing some performance gain over memcpy()
on some ARM64 platforms/enviroments.
The memory copy performance differs between different ARM64
platforms. And a more recent glibc (e.g. 2.23 or later)
can provide a better memcpy() performance compared to old glibc
versions. It's always suggested to use a more recent glibc if
possible, from which the entire system can get benefit. If for some
reason an old glibc has to be used, this patch is provided for an
alternative.
This implementation can improve memory copy on some ARM64
platforms, when an old glibc (e.g. 2.19, 2.17...) is being used.
It is disabled by default and needs "RTE_ARCH_ARM64_MEMCPY"
defined to activate. It's not always proving better performance
than memcpy() so users need to run DPDK unit test
"memcpy_perf_autotest" and customize parameters in "customization
section" in rte_memcpy_64.h for best performance.
Compiler version will also impact the rte_memcpy() performance.
It's observed on some platforms and with the same code, GCC 7.2.0
compiled binary can provide better performance than GCC 4.8.5. It's
suggested to use GCC 5.4.0 or later.
Signed-off-by: Herbert Guan <herbert.guan@arm.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Kernels v4.4 and earlier does have vfio, but not
the noiommu mode, so the file does not exist.
Check and report errors on open/read in noiommu check.
Signed-off-by: Jonas Pfefferle <jpf@zurich.ibm.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Adding common test assertion macros for unit testing.
Replaced common macros in test/test.h with new RTE_TEST_ASSERT_* macros.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
This patch fixes the following compilation errors in bsdapp
lib/librte_eal/bsdapp/eal/eal.c:782:5:
error: no previous prototype for function 'rte_vfio_clear_group'
int rte_vfio_clear_group(int vfio_group_fd)
^
lib/librte_eal/bsdapp/eal/eal.c:782:30:
error: unused parameter 'vfio_group_fd'
int rte_vfio_clear_group(int vfio_group_fd)
^
Fixes: c564a2a20093 ("vfio: expose clear group function for internal usages")
Signed-off-by: Moti Haimovsky <motih@mellanox.com>
Use global default loglevel to DEBUG(8) and dynamic default loglevel
to INFO(7).
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Reviewed-by: Olivier Matz <olivier.matz@6wind.com>
Make max vfio groups compile-time configurable so that platforms can
choose vfio group limit.
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
other vfio based module e.g. fslmc will also need to use
the clear_group call.
So, exposing it and renaming it to *rte_vfio_clear_group*
Signed-off-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
In some case, one device are accessed by different processes via
different BARs, so one uio device may be opened by more than one
process, for this case we just need to enable interrupt once, and
pci_clear_master only when the last process closed.
Fixes: 5f6ff30dc507 ("igb_uio: fix interrupt enablement after FLR in VM")
Cc: stable@dpdk.org
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Many exported headers rely on definitions found in rte_config.h without
including it, as shown by the following command:
grep -L '^#include <rte_config.h>' -- \
$(grep -Rl \
$(sed -n '/^#define \([^ ]\+\).*$/{s//\1/;H;};${x;s/\n//;s/\n/\\|/g;p;}' \
build/include/rte_config.h) \
-- build/include/)
We cannot assume external applications will include rte_config.h on their
own, neither directly nor through a -include parameter like DPDK does
internally.
This not only causes obvious compilation failures that can be reproduced
with check-includes.sh such as:
[...]/rte_memory.h:88:43: error: ‘RTE_CACHE_LINE_SIZE’ was not declared in
this scope
#define __rte_cache_aligned __rte_aligned(RTE_CACHE_LINE_SIZE)
^
It also results in less visible issues, for instance rte_hash_crc.h relying
on RTE_ARCH_X86_64's presence to provide dedicated inline functions.
This patch partially reverts the commit below and adds missing include
lines to the remaining files.
Fixes: f1a7a5c5f404 ("remove include of generated config header")
Cc: stable@dpdk.org
Signed-off-by: Adrien Mazarguil <adrien.mazarguil@6wind.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
The 'register' keyword does nothing, and has been removed in C++17.
Remove it for compatibility, like following commit:
Fixes: 0d5f2ed12f9e ("eal: remove use of register keyword")
Signed-off-by: Avi Kivity <avi@scylladb.com>
rte_eal_check_module() might return -1, which would have been a
"not false" condition for mod_available. Fix that to only report
vfio being enabled if rte_eal_check_module() returns 1.
Fixes: 221f7c220d6b ("vfio: move global config out of PCI files")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
In cases when alignment is bigger than boundary, we may incorrectly
calculate end of a bounded malloc element.
Consider this: suppose we are allocating a bounded malloc element
that should be of 128 bytes in size, bounded to 128 bytes and
aligned on a 256-byte boundary. Suppose our malloc element ends
at 0x140 - that is, 256 plus one cacheline.
So, right at the start, we are aligning our new_data_start to
include the required element size, and to be aligned on a specified
boundary - so new_data_start becomes 0. This fails the following
bounds check, because our element cannot go above 128 bytes from
the start, and we are at 320. So, we enter the bounds handling
branch.
While we're in there, we are aligning end_pt to our boundedness
requirement of 128 byte, and end up with 0x100 (since 256 is
128-byte aligned). We recalculate new_data_size and it stays at
0, however our end is at 0x100, which is beyond the 128 byte
boundary, and we report inability to reserve a bounded element
when we could have.
This patch adds an end_pt recalculation after new_data_start
adjustment - we already know that size <= bound, so we can do it
safely - and we then correctly report that we can, in fact, try
using this element for bounded malloc allocation.
Fixes: fafcc11985a2 ("mem: rework memzone to be allocated by malloc")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
When we're gathering statistics, we are traversing the freelist,
which may change under our feet in multithreaded scenario. This
is verified by occasional segfaults when running malloc autotest
on a machine with big amount of cores.
This patch protects malloc heap stats call with a lock. It changes
its definition in the process due to locking invalidating the
const-ness, but this isn't a public API, so that's OK.
Fixes: 2a5c356e177d ("memory: stats for malloc")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
We check if there's space in config after we allocated the memzone,
but if there isn't, we never free it back. This patch adds memzone
free if there's no room in memzone config.
Fixes: ff909fe21f0a ("mem: introduce memzone freeing")
Cc: stable@dpdk.org
Signed-off-by: Anatoly Burakov <anatoly.burakov@intel.com>
This commit adds a new attribute to the service cores attributes
API, which allows the application to retrieve the number of times
that a service-core called the service to perform its action.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit introduces a new API, allowing the application to
reset attributes of a service like the cycle count. Given this
functionality is now exposed to the user, remove the resetting
of stats during a dump() call.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
This commit adds a new function to the service API to allow
the application to retrieve items about each individual service
in the system. A unit test checks the return values of a variety
of invalid and valid calls.
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
The CPUID instruction is caught by hypervisor which can return
a flag indicating one is running, and its name.
Suggested-by: Stephen Hemminger <sthemmin@microsoft.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Jerin Jacob <jerin.jacob@caviumnetworks.com>
Print a warning if the --base-virtaddr hint is not respected
since this might lead to problems when mapping memory in
the secondary process.
Signed-off-by: Jonas Pfefferle <jpf@zurich.ibm.com>
Acked-by: Anatoly Burakov <anatoly.burakov@intel.com>
This patch fixes a potential bug, which was not consistently
showing up in the unit tests. The issue was that the service-
lcore being started was not in a "WAIT" state, and hence EAL
would return -EBUSY instead of launching the lcore.
In order to ensure a core is in a launch-ready state, the application
must call rte_eal_wait_lcore, to ensure that the core has completed
its previous task, and that EAL is ready to re-launch it.
The call to rte_eal_wait_lcore() is explicitly not in the
service core function, to make it visible to the application.
Requiring an explicit function call ensures the developer sees
that a lcore could block in the rte_eal_wait_lcore() function
if the core hasn't returned from its previous function.
From a usability perspective, hiding the wait_lcore() inside
service cores would cause confusion.
This patch adds rte_eal_wait_lcore() calls to the unit tests,
to ensure that the lcores for testing functionality are ready
to run the test.
Fixes: 21698354c832 ("service: introduce service cores concept")
Cc: stable@dpdk.org
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
This patch fixes the reset of the service core,
that when rte_service_lcore_del() is called, the
lcore_role is restored to RTE.
This issue was reported as when running the unit tests, an
error was thrown that "failed to allocate lcore". Investigating
revealed that the state of the service-cores after del() was
not allowing a core to be re-used at a later point in time.
Fixes: 21698354c832 ("service: introduce service cores concept")
Cc: stable@dpdk.org
Reported-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
When adding service the number of mapped cores should only be incremented
when the core is not already a service core or vice versa.
Fixes: 21698354c832 ("service: introduce service cores concept")
Cc: stable@dpdk.org
Signed-off-by: Pavan Nikhilesh <pbhagavatula@caviumnetworks.com>
Acked-by: Harry van Haaren <harry.van.haaren@intel.com>
Repeated occurrences of 'the'.
The change was obtained using the following command:
sed -i "s;the the ;the ;" `git grep -l "the "`
Signed-off-by: Thierry Herbelot <thierry.herbelot@6wind.com>
mmap(2) returns MAP_FAILED, not NULL, on failure.
Signed-off-by: Michael McConville <mmcco@mykolab.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
include <sys/vmmeter.h> to fix build
otherwise the build fails with FreeBSD 12, like
In file included from contigmem.c:57:
/usr/srcs/head/src/sys/vm/vm_phys.h:122:10: error:
use of undeclared identifier 'vm_cnt'
return (vm_cnt.v_free_count += adj);
^
Signed-off-by: Kefu Chai <tchaikov@gmail.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Replace the BSD license header with the SPDX tag for files
with only an Intel copyright on them.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Update RTE_VERIFY macro to make it possible to use complex expressions
in RTE_ASSERT.
Now it’s possible to have % char inside the expression, for example:
RTE_ASSERT((sizeof(some_struct) % 64) == 0)
Before the patch, “%" sign acts like a conversion specification
beginning character.
Fixes: 148f963fb532 ("xen: core library changes")
Signed-off-by: Ilya V. Matveychikov <matvejchikov@gmail.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
build error:
.../dpdk/build/build/lib/librte_eal/linuxapp/kni/igb_main.c:2809:2:
error: implicit declaration of function ‘setup_timer’;
did you mean ‘sk_stop_timer’? [-Werror=implicit-function-declaration]
setup_timer(&adapter->watchdog_timer, &igb_watchdog,
^~~~~~~~~~~
sk_stop_timer
cc1: all warnings being treated as errors
error observed whed CONFIG_RTE_KNI_KMOD_ETHTOOL config option enabled.
Because Linux removed setup_timer macros for kernel version >= 4.15
Linux: 513ae785c63c ("timer: Remove setup_*timer() interface")
Replaced setup_timer with timer_setup for new kernel versions.
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
More error reported for device reset in release() [1],
when device pass-through to the guest, host kernel crash on guest exit.
Removing the reset completely.
This is close to reverting commit b58eedfc7dd5 [2], taking into account
previous fix to remove reset in open as well [3], but not exactly same.
With latest code, interrupts are enabled in uio open() callback and
disabled in uio release() callback, so when a DPDK application exit
device interrupts are disabled. Previously interrupts were only enabled
once in igb_uio module insert and disabled in module removal.
Also with latest code device set as bus master in open() and master
cleared in release(), clearing bus master should prevent further DMA
which was one of the target of the initial patch.
The initial intention was also to reset the device to be sure it has
been left in proper state, but currently that part is missing because of
reported problem(s).
Still igb_uio should be safer comparing to the pre b58eedfc7dd5 state.
[1]
http://dpdk.org/ml/archives/dev/2017-November/081459.html
[2]
b58eedfc7dd5 ("igb_uio: issue FLR during open and release of device file")
[3]
f73b38e9245d ("igb_uio: remove device reset in open")
Fixes: e3a64deae2d5 ("igb_uio: prevent reset for bnx2x devices")
Fixes: b58eedfc7dd5 ("igb_uio: issue FLR during open and release of device file")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>
Move the vdev bus from lib/librte_eal to drivers/bus.
As the crypto vdev helper function refers to data structure
in rte_vdev.h, so we move those helper function into drivers/bus
too.
Signed-off-by: Jianfeng Tan <jianfeng.tan@intel.com>
This commit fixes a possible race condition if an application
uses the service-cores infrastructure and the function to run
a service on an application lcore at the same time.
The fix is to change the num_mapped_cores variable to be an
atomic variable. This causes concurrent accesses by multiple
threads to a service using rte_service_run_iter_on_app_lcore()
to detect if another core is currently mapped to the service,
and refuses to run if it is not multi-thread safe.
The run iteration on app lcore function has two arguments, the
service id to run, and if atomics should be used to serialize access
to multi-thread unsafe services. This allows applications to choose
if they wish to use use the service-cores feature, or if they
take responsibility themselves for serializing invoking a service.
See doxygen documentation for more details.
Two unit tests were added to verify the behaviour of the
function to run a service on an application core, testing both
a multi-thread safe service, and a multi-thread unsafe service.
The doxygen API documentation for the function has been updated
to reflect the current and correct behaviour.
Fixes: e9139a32f6e8 ("service: add function to run on app lcore")
Signed-off-by: Harry van Haaren <harry.van.haaren@intel.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The check for the existence of the default plugin directory calls stat
using an incorrect variable, which will cause a NULL pointer dereference
error.
Coverity issue: 198440
Fixes: d6a4399cdfc9 ("eal: avoid error for non-existent default PMD path")
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Aaron Conole <aconole@redhat.com>
For virtual device, the rte_intr_handle struct is
initialized by the virtual device driver, including
the event fd assignment. If the event fd need to be
read for clean, an argument is required for the proper
event fd read.
This patch adds efd_counter_size in rte_intr_handle
struct to tell the rx interrupt process the read size.
Signed-off-by: Jingjing Wu <jingjing.wu@intel.com>
Reviewed-by: Jianfeng Tan <jianfeng.tan@intel.com>
Revert the patchset run-time Linking support including the following
3 commits:
Fixes: 84cc318424d4 ("eal/x86: select optimized memcpy at run-time")
Fixes: c7fbc80fe60f ("test: select memcpy alignment unit at run-time")
Fixes: 5f180ae32962 ("efd: move AVX2 lookup in its own compilation unit")
The patchset would cause perf drop in vhost/virtio loopback performance
test. Because the run-time dispatch must cost at least a function call
comparing to the compile-time dispatch. And the reference cpu cycles value
is small. And in the test, when using 128-256 bytes packet, it would cause
16%-20% perf drop with mergeble path. When using 256 bytes packet, it would
cause 13% perf drop with vector path.
Signed-off-by: Xiaoyun Li <xiaoyun.li@intel.com>
Some devices are having problem on device reset that happens during DPDK
application exit [1].
Create a static list of devices and exclude them from device reset.
[1]
http://dpdk.org/ml/archives/dev/2017-November/080927.html
Fixes: b58eedfc7dd5 ("igb_uio: issue FLR during open and release of device file")
Cc: stable@dpdk.org
Signed-off-by: Ferruh Yigit <ferruh.yigit@intel.com>