The value of *tail should be the prod->tail not prod->head. After
modification, it can record 'tail' so head/tail can be updated
accordingly.
Fixes: 664ff4b172 ("ring: introduce peek style API")
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Remove the unwanted call to "_rte_ring_do_enqueue_elem" to allow for
correct handling of RTS/HTS modes.
Fixes: e6ba4731c0 ("ring: introduce RTS ring mode")
Signed-off-by: Feifei Wang <feifei.wang2@arm.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Acked-by: Honnappa Nagarahalli <honnappa.nagarahalli@arm.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
The vhost library provide an infrastructure in order to help the DPDK
users to manage vhost devices.
One of the infrastructure parts is the features enablement APIs.
Some features bits may be defined only in the internal file vhost.h in
case the kernel version doesn't include them.
Hence, user running on old kernel may not be able to manage thus
features.
Move all the feature bits definitions to the API file rte_vhost.h.
Fixes: db69be54b6 ("vhost: hide internal code")
Fixes: 8d286dbeb8 ("vhost: fix multiple queue not enabled for old kernels")
Fixes: 3d3c6590b5 ("vhost: enable virtio MTU feature")
Fixes: 704098fc47 ("vhost: fix build with old kernels")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When virtq call or kick file descriptors are changed in the device
configuration when the queue is ready, the application and the vDPA
driver should be notified to be aligned to the new file descriptors.
Notify the state to be disabled before the file descriptor update and
return it back to be enabled after the update.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Some vDPA drivers' basic configurations should be updated when the
guest memory is hotplugged.
Close vDPA device before hotplug operation and recreate it after the
hotplug operation is done.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Some guest drivers may not configure disabled virtio queues.
In this case, the vhost management never notifies the application and
the vDPA device readiness because it waits to the device to be ready.
The current ready state means that all the virtio queues should be
configured regardless the enablement status.
In order to support this case, this patch changes the ready state:
The device is ready when at least 1 queue pair is configured and
enabled.
So, now, the application and vDPA driver are notifies when the first
queue pair is configured and enabled.
Also the queue notifications will be triggered according to the new
ready definition.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
No need to take access lock in the vhost-user message handler when
vDPA driver controls all the data-path of the vhost device.
It allows the vDPA set_vring_state operation callback to configure
guest notifications.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
As an arrangement to per queue operations in the vDPA device it is
needed to change the next experimental API:
The API ``rte_vhost_host_notifier_ctrl`` was changed to be per queue
instead of per device.
A `qid` parameter was added to the API arguments list.
Setting the parameter to the value RTE_VHOST_QUEUE_ALL configures the
host notifier to all the device queues as done before this patch.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch split the vDPA header file in two, making
rte_vdpa_device structure opaque to the application.
Applications should only include rte_vdpa.h, while drivers
should include both rte_vdpa.h and rte_vdpa_dev.h.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This API is no more useful, this patch removes it.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This patch is preliminary work to make the vDPA device
structure opaque to the user application. Some callbacks
of the vDPA devices are used to query capabilities before
attaching to a Vhost port. This patch introduces wrappers
for these ops.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
There is no more notion of device ID outside of vdpa.c.
We can now move from array to linked-list model for keeping
track of the vDPA devices.
There is no point in using array here, as all vDPA API are
used from the control path, so no performance concerns.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
vDPA is no more used outside of the vDPA internals,
so remove rte_vdpa_get_device() API that is now useless.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This patch replaces the use of vDPA device ID with
vDPA device pointer. The goals is to remove the vDPA
device ID to avoid confusion with the Vhost ID.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This removes the notion of device ID in Vhost library
as a preliminary step to get rid of the vDPA device ID.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This patch is a preliminary step to get rid of the
vDPA device ID. It makes vDPA callbacks to use the
vDPA device struct as a reference instead of the ID.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This patch makes the vDPA framework to no more
support only PCI devices, but any devices by relying
on the generic device name as identifier.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
This patch introduces vDPA device class. It will enable
application to iterate over the vDPA devices.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Adrián Moreno <amorenoz@redhat.com>
vcopyq_laneq_u32 should be implemented for aarch32 which doesn't have
the intrinsic.
This fixes build of examples/l3fwd for armv7.
Fixes: 3c4b4024c2 ("arch/arm: add vcopyq_laneq_u32 for old gcc")
Cc: stable@dpdk.org
Signed-off-by: Ruifeng Wang <ruifeng.wang@arm.com>
Reviewed-by: Ferruh Yigit <ferruh.yigit@intel.com>
The vDPA device offloads all the datapath of the vhost
device to the HW device.
In order to expose to the user traffic information this
patch introduces new 3 APIs to get traffic statistics, the
device statistics name and to reset the statistics per
virtio queue.
The statistics are taken directly from the vDPA driver
managing the HW device and can be different for each vendor
driver.
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
As announced during v20.05 release cycle, this
patch makes reply-ack protocol feature to be enabled
unconditionally.
This protocol feature makes the communication between the
master and the slave more robust, avoiding for example
possible undefined behaviour with VHOST_USER_SET_MEM_TABLE.
Also, reply-ack support will be required for upcoming
VHOST_USER_SET_STATUS request.
Note that this protocol feature was disabled by default
because Qemu version 2.7.0 to 2.9.0 had a bug causing a
deadlock when reply-ack was negotiated and multiqueue
enabled. These Qemu version are now very old and no more
maintained, so we can reasonably consider we no more
support them.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Casting thread ID to handle is not accurate way to get thread handle.
Need to use OpenThread function to get thread handle from thread ID.
pthread_setaffinity_np and pthread_getaffinity_np functions
for Windows are affected because of it.
Signed-off-by: Tasnim Bashar <tbashar@mellanox.com>
Uses SetupAPI.h functions to scan PCI tree.
Uses DEVPKEY_Device_Numa_Node to get the PCI NUMA node.
Uses SPDRP_BUSNUMBER and SPDRP_BUSNUMBER to get the BDF.
scanning currently supports types RTE_KDRV_NONE.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
the struct rte_pci_addr defines domain as uint32_t variable however
the PCI_PRI_FMT macro used for logging the struct sets the format
of domain to uint16_t.
The mismatch causes the following warning messages
in Windows clang build:
format specifies type 'unsigned short' but the argument
has type 'uint32_t' (aka 'unsigned int') [-Wformat]
Fixes: af75078fec ("first public release")
Cc: stable@dpdk.org
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Added <sys/types.h> in rte_pci header file
to include off_t type since it is missing for Windows.
Define the implementation of the Linux function rte_pci_get_sysfs_path
in pci_common.c for Linux OS only as it is unneeded for other OSs
and to avoid the warning on deprecated call to getenv() on Windows:
"warning: 'getenv' is deprecated: This function or variable may be unsafe.
Consider using _dupenv_s instead."
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Changing all of PCIs Unix memory mapping to the
new memory allocation API wrapper.
Change all of PCI mapping function usage in
bus/pci to support the new API.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Move common functions between Unix and Windows to eal_common_options.c.
Those functions are getter functions for rte_application_usage_hook.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Move common functions between Unix and Windows to eal_common_config.c.
Those functions are getter functions for IOVA,
configuration, Multi-process.
Move rte_config, internal_config, early_mem_config and runtime_dir
to be defined in the common file with getter functions.
Refactor the users of the config variables above to use
the getter functions.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
The MinGW build for Windows has special cases where exported
function contain additional prefix:
__emutls_v.per_lcore__*
To avoid adding those prefixed functions to the version.map file
the map_to_def.py script was modified to create a map file for MinGW
with the needed changed.
The file name was changed to map_to_win.py and lib/meson.build map output
was unified with drivers/meson.build output
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
For each mbuf byte, free_space[i] == 0 means the space is occupied,
free_space[i] != 0 means space is free.
Fixes: 4958ca3a44 ("mbuf: support dynamic fields and flags")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
The value free_space[i] is used to save the size of biggest aligned
element that can fit in the zone, current implementation has one flaw,
for example, if user registers dynfield1 (size = 4, align = 4, req = 124)
first, the free_space would be as below after registration:
0070: 08 08 08 08 08 08 08 08
0078: 08 08 08 08 00 00 00 00
Then if user continues to register dynfield2 (size = 4, align = 4),
free_space would become:
0070: 00 00 00 00 04 04 04 04
0078: 04 04 04 04 00 00 00 00
Further request dynfield3 (size = 8, align = 8) would fail to register
due to alignment requirement can't be satisfied, though there is enough
space remained in mbuf.
This patch fixes above issue by saving alignment only in aligned zone,
after the fix, above registrations order can be satisfied, free_space
would be like:
After dynfield1 registration:
0070: 08 08 08 08 08 08 08 08
0078: 04 04 04 04 00 00 00 00
After dynfield2 registration:
0070: 08 08 08 08 08 08 08 08
0078: 00 00 00 00 00 00 00 00
After dynfield3 registration:
0070: 00 00 00 00 00 00 00 00
0078: 00 00 00 00 00 00 00 00
This patch also reduces iterations in process_score() by jumping align
steps in each loop.
Fixes: 4958ca3a44 ("mbuf: support dynamic fields and flags")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
Set rte_errno as ENOMEM when allocation failure.
Fixes: 4958ca3a44 ("mbuf: support dynamic fields and flags")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
We should make sure off + size < sizeof(struct rte_mbuf) to avoid
possible out-of-bounds access of free_space array, there is no issue
currently due to the low bits of free_flags (which is adjacent to
free_space) are always set to 0. But we shouldn't rely on it since it's
fragile and layout of struct mbuf_dyn_shm may be changed in the future.
This patch adds boundary check explicitly to avoid potential risk of
out-of-bounds access.
Fixes: 4958ca3a44 ("mbuf: support dynamic fields and flags")
Cc: stable@dpdk.org
Signed-off-by: Xiaolong Ye <xiaolong.ye@intel.com>
Acked-by: Olivier Matz <olivier.matz@6wind.com>
64-bit support was missing from the functions pipe_profile_check
and rte_sched_subport_config_pipe_profile_table.
Fixes: 68c1f26d42 ("sched: support 64-bit values")
Cc: stable@dpdk.org
Signed-off-by: Archit Pandey <architpandeynitk@gmail.com>
Acked-by: Jasvinder Singh <jasvinder.singh@intel.com>
In function rte_sched_subport_free, there is code to free all allocated
stuff related to scheduler subport.
First there are some checks, and in the end, rte_bitmap_free is called.
Now, rte_bitmap_free is a dummy function, and it just checks if
provided pointer to bitmap is valid or not. So, actual memory for
subport is not freed.
This patch fixes this by removing call to rte_bitmap_free, and
instead calling rte_free.
Fixes: d9213b829a ("sched: remove pipe params config from port level")
Cc: stable@dpdk.org
Signed-off-by: Hrvoje Habjanic <hrvoje.habjanic@zg.ht.hr>
Acked-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Jasvinder Singh <jasvinder.singh@intel.com>
Make x86 JIT to generate native code for
(BPF_ABS | <size> | BPF_LD) and (BPF_IND | <size> | BPF_LD)
instructions.
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Acked-by: Morten Brørup <mb@smartsharesystems.com>
To fill the gap with linux kernel eBPF implementation,
add support for two non-generic instructions:
(BPF_ABS | <size> | BPF_LD) and (BPF_IND | <size> | BPF_LD)
which are used to access packet data.
These instructions can only be used when BPF context is a pointer
to 'struct rte_mbuf' (i.e: RTE_BPF_ARG_PTR_MBUF type).
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
eval_add()/eval_sub() not always correctly estimate
minimum and maximum possible values of add/sub operations.
Fixes: 8021917293 ("bpf: add extra validation for input BPF program")
Cc: stable@dpdk.org
Signed-off-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
Support the debug functions in eal_common_debug.c for Windows.
Implementation of rte_dump_stack to get a backtrace similarly to Unix
and of rte_eal_cleanup in eal.c.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
Tested-by: Dmitry Kozlyuk <dmitry.kozliuk@gmail.com>
Move common functions between Unix and Windows to eal_common_debug.c.
Those functions are rte_exit, __rte_panic and rte_dump_registers
which has the same implementation on Unix and Windows.
Signed-off-by: Tal Shnaiderman <talshn@mellanox.com>
An issue has been observed where epoll file descriptor
list rebuilds every time an interrupt/alarm event is
received.
eal_intr_process_interrupts() should notify pipe fd only
if any source is removed from the source list i.e (rv > 0)
Fixes: 0c7ce182a7 ("eal: add pending interrupt callback unregister")
Cc: stable@dpdk.org
Signed-off-by: Harman Kalra <hkalra@marvell.com>
The code didn't compile when using exported meter functions under Windows.
error LNK2001: unresolved external symbol
rte_meter_srtcm_color_aware_check
error LNK2001: unresolved external symbol
rte_meter_srtcm_color_blind_check
error LNK2001: unresolved external symbol
rte_meter_trtcm_color_aware_check
error LNK2001: unresolved external symbol
rte_meter_trtcm_color_blind_check
error LNK2001: unresolved external symbol
rte_meter_trtcm_rfc4115_color_aware_check
error LNK2001: unresolved external symbol
rte_meter_trtcm_rfc4115_color_blind_check
The cause was that there were some inline functions that were included in
the export list.
To solve this the functions were removed from rte_meter_version.map export
list which are implemented in the header and shouldn't be exported.
Fixes: 655796d2b5 ("meter: support RFC4115 trTCM")
Fixes: 9d41beed24 ("lib: provide initial versioning")
Cc: stable@dpdk.org
Signed-off-by: Fady Bader <fady@mellanox.com>
EAL common timer doesn't compile under Windows.
Compilation log:
error LNK2019:
unresolved external symbol nanosleep referenced in function
rte_delay_us_sleep
error LNK2019:
unresolved external symbol get_tsc_freq referenced in function set_tsc_freq
error LNK2019:
unresolved external symbol sleep referenced in function set_tsc_freq
The reason was that some functions called POSIX functions.
The solution was to move POSIX dependent functions from common to Unix.
Signed-off-by: Fady Bader <fady@mellanox.com>
Reviewed-by: Tal Shnaiderman <talshn@mellanox.com>
Acked-by: Ranjit Menon <ranjit.menon@intel.com>
All API's should check that they support the flag values
passed. If an application passes an invalid flag it could
cause problems in later ABI.
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Konstantin Ananyev <konstantin.ananyev@intel.com>
