When Qemu suspends a VM, HW notifier is un-mmapped while vCPU thread may
still be active and write notifier through kick socket.
PMD kick handler thread tries to install HW notifier through client
socket. In such case, it will timeout and slow down device close.
This patch skips HW notifier install if VQ or device in middle of
shutdown.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
In Ctrl+C handling, sometimes kick handling thread gets endless EGAIN
error and fall into dead lock.
Kick happens frequently in real system due to busy traffic or retry
mechanism. This patch simplifies kick firmware anyway and skip setting
hardware notifier due to potential device error, notifier could be set
in next successful kick request.
Fixes: 62c813706e ("vdpa/mlx5: map doorbell")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Improve the devargs handling in two aspects:
- Parse the devargs string only once.
- Return error and report for unknown keys.
The common driver parses once the devargs string into a dictionary, then
provides it to all the drivers' probe. Each driver updates within it
which keys it has used, then common driver receives the updated
dictionary and reports about unknown devargs.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Buffer for MCDI channel is allocated using rte_memzone_reserve_aligned
with zone name 'mcdi'. Since multiple MCDI channels are needed to
support multiple VF(s) and rte_memzone_reserve_aligned expects unique
zone names, append PCI address to zone name to make it unique.
Signed-off-by: Abhimanyu Saini <asaini@xilinx.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When sva is null, sfc_vdpa_info(sva, ...) will cause a null
dereference. Use SFC_VDPA_GENERIC_LOG() to avoid that.
See macros sfc_vdpa_info and SFC_VDPA_GENERIC_LOG
defined in drivers/vdpa/sfc/sfc_vdpa_log.h for detail.
Fixes: 5e7596ba7c ("vdpa/sfc: introduce Xilinx vDPA driver")
Cc: stable@dpdk.org
Signed-off-by: Weiguo Li <liwg06@foxmail.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Fixes: b11961363b ("vdpa/sfc: support device configure and close")
Cc: stable@dpdk.org
Signed-off-by: Weiguo Li <liwg06@foxmail.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Functions like free, rte_free, and rte_mempool_free
already handle NULL pointer so the checks here are not necessary.
Remove redundant NULL pointer checks before free functions
found by nullfree.cocci
Signed-off-by: Stephen Hemminger <stephen@networkplumber.org>
When the event thread polls traffic and a virtq is stopping, the FW loses
synchronization in the virtq indexes.
It causes LM failure on synchronization between the HOST indexes to
the GUEST indexes.
Unset the event thread before the queue stop in the LM process.
Fixes: 31b9c29c86 ("vdpa/mlx5: support close and config operations")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Some drivers currently have their own checks and give some non
consistent reasons when an internal dependency is unavailable.
drivers/meson.build also checks for internal dependencies via 'deps'.
Let's rely on it for consistency, and smaller code.
Signed-off-by: David Marchand <david.marchand@redhat.com>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Long Li <longli@microsoft.com>
The return value of "mlx5_os_wrapped_mkey_create" is checked in the
caller. A zero means success without any error.
The typo in the if-condition should be fixed in case there is a
misjudgment.
Fixes: 398ea8450c ("vdpa/mlx5: workaround dirty bitmap MR creation")
Cc: stable@dpdk.org
Signed-off-by: Bing Zhao <bingz@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Due to kernel issue in direct MKEY creation using the DevX API, this
patch replaces the virtio MR creation to use Verbs API.
Fixes: cc07a42da2 ("vdpa/mlx5: prepare memory regions")
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Signed-off-by: Matan Azrad <matan@nvidia.com>
Due to kernel driver/FW issues in direct MKEY creation using the DevX
API, this patch replaces the dirty bitmap MR creation to use wrapped
mkey instead.
Fixes: 9d39e57f21 ("vdpa/mlx5: support live migration")
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Signed-off-by: Matan Azrad <matan@nvidia.com>
The number of WQEBBs was provided to QP create, and QP size was calculated
by multiplying the number of WQEBBs by 64, which is the send WQE size.
When creating RQ in the QP (i.e., vdpa driver), the queue size was bigger
because the receive WQE size is 16.
Provide queue size to QP create instead of the number of WQEBBs.
Fixes: f9213ab12c ("common/mlx5: share DevX queue pair operations")
Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
The DevX interface for QP creation expects the number of WQEBBs.
Wrongly, the number of descriptors was provided to the QP creation.
In addition, the QP size must be a power of 2 what was not guaranteed.
Provide the number of WQEBBs to the QP creation API.
Round up the SQ size to a power of 2.
Rename (sq/rq)_size to num_of_(send/receive)_wqes.
Fixes: 6152534e21 ("crypto/mlx5: support queue pairs operations")
Cc: stable@dpdk.org
Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Tal Shnaiderman <talshn@nvidia.com>
The rdma-core library can map doorbell register in two ways, depending
on the environment variable "MLX5_SHUT_UP_BF":
- as regular cached memory, the variable is either missing or set to
zero. This type of mapping may cause the significant doorbell
register writing latency and requires an explicit memory write
barrier to mitigate this issue and prevent write combining.
- as non-cached memory, the variable is present and set to not "0"
value. This type of mapping may cause performance impact under
heavy loading conditions but the explicit write memory barrier is
not required and it may improve core performance.
The UAR creation function maps a doorbell in one of the above ways
according to the system. In run time, it always adds an explicit memory
barrier after writing to.
In cases where the doorbell was mapped as non-cached memory, the
explicit memory barrier is unnecessary and may impair performance.
The commit [1] solved this problem for a Tx queue. In run time, it
checks the mapping type and provides the memory barrier after writing to
a Tx doorbell register if it is needed. The mapping type is extracted
directly from the uar_mmap_offset field in the queue properties.
This patch shares this code between the drivers and extends the above
solution for each of them.
[1] commit 8409a28573
("net/mlx5: control transmit doorbell register mapping")
Fixes: f8c97babc9 ("compress/mlx5: add data-path functions")
Fixes: 8e196c08ab ("crypto/mlx5: support enqueue/dequeue operations")
Fixes: 4d4e245ad6 ("regex/mlx5: support enqueue")
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
UAR mapping type can be affected by the devarg tx_db_nc, which can cause
setting the environment variable MLX5_SHUT_UP_BF.
So, the MLX5_SHUT_UP_BF value and the UAR mapping parameter affect the
UAR cache mode.
Wrongly, the devarg was considered for the MLX5_SHUT_UP_BF but not for
the UAR mapping parameter in all the drivers except the net.
Take the tx_db_nc devarg into account for all the drivers.
Fixes: ca1418ce39 ("common/mlx5: share device context object")
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Add support for unicast and broadcast MAC filter configuration.
Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Implement the vDPA ops get_notify_area to get the notify
area info of the queue.
Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Implement vDPA ops get_queue_num to get the maximum number
of queues supported by the device.
Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Add new vDPA PMD to support vDPA operations of Xilinx devices.
This patch implements probe and remove functions.
Signed-off-by: Vijay Kumar Srivastava <vsrivast@xilinx.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch marks the vDPA driver APIs as internal and
rename the corresponding header file to vdpa_driver.h.
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Removing direct access to interrupt handle structure fields,
rather use respective get set APIs for the same.
Making changes to all the drivers access the interrupt handle fields.
Signed-off-by: Harman Kalra <hkalra@marvell.com>
Acked-by: Hyong Youb Kim <hyonkim@cisco.com>
Signed-off-by: David Marchand <david.marchand@redhat.com>
Tested-by: Raslan Darawsheh <rasland@nvidia.com>
VAR is the device memory space for the virtio queues doorbells,
Qemu could mmap it to directly to speed up doorbell push.
On a busy system, Qemu takes time to release VAR resources during driver
shutdown. If vdpa restarted quickly, the VAR allocation failed with
error 28 since the VAR is singleton resource per device.
This patch adds retry mechanism for VAR allocation.
Fixes: 4cae722c1b ("vdpa/mlx5: move virtual doorbell alloc to probe")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
After a vDPA application restart, Qemu restores VQ with used and
available index, new incoming packet triggers virtio driver to
handle buffers. Under heavy traffic, no available buffer for
firmware to receive new packets, no Rx interrupts generated,
driver is stuck on endless interrupt waiting.
As a firmware workaround, this patch sends a notification after
VQ setup to ask driver handling buffers and filling new buffers.
Fixes: bff7350110 ("vdpa/mlx5: prepare virtio queues")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Add HCA attributes structure as a field of device config structure.
It query in common probing, and updates the timestamp format fields.
Each driver use HCA attributes from common device config structure,
instead of query it for itself.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Create shared Protection Domain in common area and add it and its PDN as
fields of common device structure.
Use this Protection Domain in all drivers and remove the PD and PDN
fields from their private structure.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Add option to get IB device after disabling RoCE. It is relevant if
there is vDPA class in device arguments list.
Use common device context in vDPA driver and remove the ctx field from
its private structure.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Create common probing structure that includes, for now, basic probing
information detected by the common driver and share it with all the
internal drivers.
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Currently drivers using QP (vDPA, crypto and compress, regex soon)
manage their memory, creation, modification and destruction of the QP,
in almost identical code.
Move QP memory management, creation and destruction to common.
Add common function to change QP state to RTS.
Add user_index attribute to QP creation.
It's for better code maintenance and reuse.
Signed-off-by: Raja Zidane <rzidane@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Use bool type for function's switch parameter,
this could avoid passing "1" or "0" which is not reader friendly.
Signed-off-by: Jilei Chen <chenjilei@cmss.chinamobile.com>
Acked-by: Xiao Wang <xiao.w.wang@intel.com>
When VM size is larger than 4G (u32) and memory region is larger than 4G,
the 32-bit GCD function overflowed and returned wrong value
that resulted in memory registration failure.
This patch calls 64-bit GCD function to avoid overflow.
Fixes: cc07a42da2 ("vdpa/mlx5: prepare memory regions")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Matan Azrad <matan@nvidia.com>
Error occurs when configuring meson with --buildtype=minsize
with GCC 11.1.0:
drivers/vdpa/mlx5/mlx5_vdpa_mem.c: In function ‘mlx5_vdpa_mem_register’:
drivers/vdpa/mlx5/mlx5_vdpa_mem.c:183:24: error:
initialization of ‘uint64_t’ {aka ‘long unsigned int’} from ‘void *’
makes integer from pointer without a cast [-Werror=int-conversion]
| uint64_t gcd = NULL;
| ^~~~
drivers/vdpa/mlx5/mlx5_vdpa_mem.c:244:75: error:
‘mode’ may be used uninitialized in this function [-Werror=maybe-uninitialized]
| klm_size = mode == MLX5_MKC_ACCESS_MODE_KLM ?
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
| KLM_SIZE_MAX_ALIGN(empty_region_sz) : gcd;
| ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Start a new release cycle with empty release notes.
The ABI version becomes 22.0.
The map files are updated to the new ABI major number (22).
The ABI exceptions are dropped and CI ABI checks are disabled because
compatibility is not preserved.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ferruh Yigit <ferruh.yigit@intel.com>
Acked-by: David Marchand <david.marchand@redhat.com>
The mlx5_vdpa_event_qp_create function makes shifting to the numeric
constant 1, then multiplies it by another constant and finally assigns
it into a uint64_t variable.
The numeric constant type is an int with a 32-bit sign. if after
shifting , its MSB (bit of sign) will change, the uint64 variable will
get into it a different value than what the function intended it to get.
Set the numeric constant 1 to be uint64_t in the first place.
Fixes: 8395927cdf ("vdpa/mlx5: prepare HW queues")
Cc: stable@dpdk.org
Signed-off-by: Michael Baum <michaelba@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
RoCE disabling requirement is based on PCI address.
In order to support Sub-Function, a conversion is needed
in the case of an auxiliary device.
SF device can be probed with such devargs string:
auxiliary:mlx5_core.sf.<id>,class=vdpa
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
This adds matching on the reserved field of VXLAN
header (the last 8-bits). The capability from rdma-core
is detected by creating a dummy matcher using misc5
when the device is probed.
For non-zero groups and FDB domain, the capability is
detected from rdma-core, meanwhile for NIC domain group
zero it's relying on the HCA_CAP from FW.
Signed-off-by: Rongwei Liu <rongweil@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Raslan Darawsheh <rasland@nvidia.com>
Packet was corrupted when TSO requested without CSUM update.
Enables CSUM automatically if only TSO requested.
Fixes: 2aa8444b00 ("vdpa/mlx5: support stateless offloads")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Add common devargs key definition for "bus", "class" and "driver".
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
The vDPA PCI device unplug process should release all the private
device resources and also to unregister the device.
The device unregistration was missed what remained the device data
invalid in the rte_vhost library.
Unregister the device in unplug process via the remove operation.
Fixes: 95276abaaf ("vdpa/mlx5: introduce Mellanox vDPA driver")
Cc: stable@dpdk.org
Reported-by: Eli Britstein <elibr@nvidia.com>
Signed-off-by: Matan Azrad <matan@nvidia.com>
Tested-by: Eli Britstein <elibr@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
The crypto driver added new fields to the mkey attributes struct:
crypto_en and set_remote_rw.
The entire mkey struct was not initialized, only specific fields in it,
which caused the new added fields not to be initialized resulting in a
mkey creation error.
This is fixed by initializing the entire mkey attributes struct to 0
which will prevent this issue from reoccurring if any fields are added
to the mkey struct in the future.
Fixes: 0111a74e13 ("common/mlx5: adjust DevX mkey fields for crypto")
Signed-off-by: Shiri Kuzin <shirik@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Let's try to enforce the convention where most drivers use a pmd. logtype
with their class reflected in it, and libraries use a lib. logtype.
Introduce two new macros:
- RTE_LOG_REGISTER_DEFAULT can be used when a single logtype is
used in a component. It is associated to the default name provided
by the build system,
- RTE_LOG_REGISTER_SUFFIX can be used when multiple logtypes are used,
and then the passed name is appended to the default name,
RTE_LOG_REGISTER is left untouched for existing external users
and for components that do not comply with the convention.
There is a new Meson variable log_prefix to adapt the default name
for baseband (pmd.bb.), bus (no pmd.) and mempool (no pmd.) classes.
Note: achieved with below commands + reverted change on net/bonding +
edits on crypto/virtio, compress/mlx5, regex/mlx5
$ git grep -l RTE_LOG_REGISTER drivers/ |
while read file; do
pattern=${file##drivers/};
class=${pattern%%/*};
pattern=${pattern#$class/};
drv=${pattern%%/*};
case "$class" in
baseband) pattern=pmd.bb.$drv;;
bus) pattern=bus.$drv;;
mempool) pattern=mempool.$drv;;
*) pattern=pmd.$class.$drv;;
esac
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
done
$ git grep -l RTE_LOG_REGISTER lib/ |
while read file; do
pattern=${file##lib/};
pattern=lib.${pattern%%/*};
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern',/RTE_LOG_REGISTER_DEFAULT(\1,/' $file;
sed -i -e 's/RTE_LOG_REGISTER(\(.*\), '$pattern'\.\(.*\),/RTE_LOG_REGISTER_SUFFIX(\1, \2,/' $file;
done
Signed-off-by: David Marchand <david.marchand@redhat.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
The driver should notify the guest for each traffic burst detected by CQ
polling.
The CQ polling trigger is defined by `event_mode` device argument,
either by busy polling on all the CQs or by blocked call to HW
completion event using DevX channel.
Also, the polling event modes can move to blocked call when the
traffic rate is low.
The current blocked call uses the EAL interrupt API suffering a lot
of overhead in the API management and serve all the drivers and
libraries using only single thread.
Use blocking FD of the DevX channel in order to do blocked call
directly by the DevX channel FD mechanism.
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The function pthread_setname_np is non-portable,
so it may be unavailable in old glibc or other systems.
The function rte_thread_setname is workarounding portability issues.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Chengwen Feng <fengchengwen@huawei.com>
The get_ib_device_match function iterates over the list of ib devices
returned by the get_device_list glue function and returns the ib device
matching the provided address.
Since this function is in use by several drivers, in this patch we
share the function in common part.
Signed-off-by: Shiri Kuzin <shirik@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
This patch supports set notify and vring relay thread name which is
helpful for debugging.
Signed-off-by: Chengwen Feng <fengchengwen@huawei.com>
Signed-off-by: Min Hu (Connor) <humin29@huawei.com>
Ensure all lists of drivers are standardized:
* one driver per line
* lists double-indented with spaces (as they are line continuations)
* elements in alphabetical order
* opening and closing list brackets "[" & "]" on own lines
* last element has trailing comma
Any code snippets in the list files is adjusted to single-indent using
whitespace to correspond to the new style also.
The lists of standard library dependencies per class, and other short
lists are not formatted one-per-line as these lists are not expected to
grow beyond 2 or 3 entries.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
The log levels are configured by using the name of the logs.
Some drivers are aligned to follow a common log name standard:
pmd.class.driver[.sub]
Some "common" drivers skip the "class" part:
pmd.driver.sub
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Xiao Wang <xiao.w.wang@intel.com>
Acked-by: Hemant Agrawal <hemant.agrawal@nxp.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
Acked-by: Min Hu (Connor) <humin29@huawei.com>
The HW virtq object can be destroyed either when the device is closed or
when the state of the virtq becomes disabled.
Some parameters of the virtq should continue to be managed when the
virtq state is changed but all of them must be initialized when the
device is closed.
Wrongly, the enable parameter stayed on when the device is closed what
might cause creation of invalid virtq in the next time a device is
assigned to the driver.
Clean all the virtqs memory when the device is closed.
Fixes: c47d6e8333 ("vdpa/mlx5: support queue update")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The return value of rte_pci_read_config should be checked.
Coverity issue: 302860
Fixes: a3f8150eac ("net/ifcvf: add ifcvf vDPA driver")
Cc: stable@dpdk.org
Signed-off-by: Xiao Wang <xiao.w.wang@intel.com>
Reviewed-by: Chenbo Xia <chenbo.xia@intel.com>
1/ The function pthread_yield() does not exist in musl libc,
and can be replaced with sched_yield() after including sched.h.
2/ The function pthread_attr_setaffinity_np() does not exist in musl libc,
and can be replaced with pthread_setaffinity_np() after pthread_create().
Fixes: b7fa0bf4d5 ("vdpa/mlx5: fix polling threads scheduling")
Fixes: 5cf3fd3af4 ("vdpa/mlx5: add CPU core parameter to bind polling thread")
Cc: stable@dpdk.org
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: David Marchand <david.marchand@redhat.com>
The page size is often retrieved from the macro PAGE_SIZE.
If PAGE_SIZE is not defined, it is either using hard coded default,
or getting the system value from the UNIX-only function sysconf().
Such definitions are replaced with the generic function
rte_mem_page_size() defined for each supported OS.
Removing PAGE_SIZE definitions will fix dlb drivers for musl libc,
because #ifdef checks were missing, causing redefinition errors.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Andrew Boyer <aboyer@pensando.io>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: David Marchand <david.marchand@redhat.com>
Acked-by: Timothy McDaniel <timothy.mcdaniel@intel.com>
This patch adds support for the timestamp format settings for
the receive and send queues. If the firmware version x.30.1000
or above is installed and the NIC timestamps are configured
with the real-time format, the default zero values for newly
added fields cause the queue creation to fail.
The patch queries the timestamp formats supported by the hardware
and sets the configuration values in queue context accordingly.
Fixes: 95276abaaf ("vdpa/mlx5: introduce Mellanox vDPA driver")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Acked-by: Ori Kam <orika@nvidia.com>
The macro DRV_LOG already includes a terminating line feed character
defined in PMD_DRV_LOG_.
The extra line feeds added in some messages are removed.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Matan Azrad <matan@nvidia.com>
When the event mode is with 0 fixed delay, the polling-thread will never
give-up CPU.
So, when multi-polling-threads are active, the context-switch between
them will be managed by the system which may affect latency according to
the time-out decided by the system.
In order to fix multi-devices polling thread scheduling, this patch
forces rescheduling for each CQ poll iteration.
Move the polling thread to SCHED_RR mode with maximum priority to
complete the fairness.
Fixes: 6956a48cab ("vdpa/mlx5: set polling mode default delay to zero")
Signed-off-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
When the vDPA device is closed, the driver polling thread is canceled.
The polling thread locks the configuration mutex while it polls the CQs.
When the cancellation happens, it may terminate the thread inside the
critical section what remains the configuration mutex locked.
After device close, the driver may be configured again, in this case,
for example, when the first queue state is updated, the driver tries to
lock the mutex again and deadlock appears.
Initialize the mutex after the polling thread cancellation.
Fixes: 99abbd62c2 ("vdpa/mlx5: fix queue update synchronization")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Maxime Coquelin <maxime.coquelin@redhat.com>
As announced in the deprecation note, remove all compatibility build
defines from previous make/meson versions and use only the standardized
ones - RTE_LIB_<name> for libraries, and RTE_<CLASS>_<NAME> for drivers.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
The next parameters control the HW queue moderation feature.
This feature helps to control the traffic performance and latency
trade-off.
Each packet completion report from HW to SW requires CQ processing by SW
and triggers interrupt for the guest driver. Interrupt report and
handling cost CPU cycles and time and the amount of this affects
directly on packet performance and latency.
hw_latency_mode parameters [int]
0, HW default.
1, Latency is counted from the first packet completion report.
2, Latency is counted from the last packet completion.
hw_max_latency_us parameters [int]
0 - 4095, The maximum time in microseconds that packet completion
report can be delayed.
hw_max_pending_comp parameter [int]
0 - 65535, The maximum number of pending packets completions in an HW
queue.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
For better performance and latency, this patch sets default event
handling mode to polling mode which uses dedicate thread per device to
poll and process event.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This patch adds new device argument to specify cpu core affinity to
event polling thread for better latency and throughput. The thread
could be also located by name "vDPA-mlx5-<id>".
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
To improve performance and latency, this patch sets Rx polling mode
default delay time to zero.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
To improve throughput and latency, this patch allows Rx polling timer
delay to 0us.
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Wrap the API to create/destroy event channel and to subscribe an event
with OS calls. In Linux those calls are implemented by glue functions
while in Windows they are not supported.
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Starting ConnectX-6 Dx, the VF device ID is generic
and not per chip.
https://pci-ids.ucw.cz/v2.2/pci.ids
101e ConnectX Family mlx5Gen Virtual Function
This means that all will have the same VF device ID.
Fixes: 5fc66630be ("net/mlx5: add ConnectX6-DX device ID")
Cc: stable@dpdk.org
Signed-off-by: Raslan Darawsheh <rasland@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
This patch provides the UAR allocation workaround for the
hosts where UAR allocation with Write-Combining memory
mapping type fails.
Fixes: 8395927cdf ("vdpa/mlx5: prepare HW queues")
Cc: stable@dpdk.org
Signed-off-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
The current DevX implementation of the relaxed ordering feature is
enabling relaxed ordering usage only if both relaxed ordering read AND
write are supported. In that case both relaxed ordering read and write
are activated.
This commit will optimize the usage of relaxed ordering by enabling it
when the read OR write features are supported. Each relaxed ordering
type will be activated according to its own capability bit.
This will align the DevX flow with the verbs implementation of
ibv_reg_mr when using the flag IBV_ACCESS_RELAXED_ORDERING
Fixes: 53ac93f71a ("net/mlx5: create relaxed ordering memory regions")
Cc: stable@dpdk.org
Signed-off-by: Tal Shnaiderman <talshn@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
If set TIS lag port affinity to auto, firmware assign port affinity on
each creation with Round Robin. In case of 2 PFs, if create virtq,
destroy and create again, then each virtq will get same port affinity.
To resolve this fw limitation, this patch sets create TIS with specified
affinity for each PF.
Fixes: bff7350110 ("vdpa/mlx5: prepare virtio queues")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When hardware error happens, vdpa didn't get such information and leave
driver in silent: working state but no response.
This patch subscribes firmware virtq error event and try to recover max
3 times in 3 seconds, stop virtq if max retry number reached.
When error happens, PMD log in warning level. If failed to recover,
outputs error log. Query virtq statistics to get error counters report.
Acked-by: Matan Azrad <matan@nvidia.com>
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
This adds the ConnectX-7 and Bluefield-3 device ids to the list of
supported Mellanox devices that run the MLX5 PMDs.
The devices is still in development stage.
Signed-off-by: Raslan Darawsheh <rasland@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
Use the newer macros defined by meson in all DPDK source code, to ensure
there are no errors when the old non-standard macros are removed.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
As discussed on the dpdk-dev mailing list[1], we can make some easy
improvements in standardizing the naming of the various components in DPDK,
and their associated feature-enabled macros.
Following this patch, each library will have the name in format,
'librte_<name>.so', and the macro indicating that library is enabled in the
build will have the form 'RTE_LIB_<NAME>'.
Similarly, for libraries, the equivalent name formats and macros are:
'librte_<class>_<name>.so' and 'RTE_<CLASS>_<NAME>', where class is the
device type taken from the relevant driver subdirectory name, i.e. 'net',
'crypto' etc.
To avoid too many changes at once for end applications, the old macro names
will still be provided in the build in this release, but will be removed
subsequently.
[1] http://inbox.dpdk.org/dev/ef7c1a87-79ab-e405-4202-39b7ad6b0c71@solarflare.com/t/#u
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Since each version map file is contained in the subdirectory of the library
it refers to, there is no need to include the library name in the filename.
This makes things simpler in case of library renaming.
Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Acked-by: Luca Boccassi <bluca@debian.org>
Acked-by: Andrew Rybchenko <andrew.rybchenko@oktetlabs.ru>
Acked-by: Rosen Xu <rosen.xu@intel.com>
VIRTIO_F_IOMMU_PLATFORM is now defined in recent kernel
headers, causing build issue.
Let's define it in the IFC vDPA driver only if it wasn't already.
Fixes: a3f8150eac ("net/ifcvf: add ifcvf vDPA driver")
Cc: stable@dpdk.org
Reported-by: Brandon Lo <blo@iol.unh.edu>
Signed-off-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Acked-by: David Marchand <david.marchand@redhat.com>
The CQ polling is done in order to notify the guest about new traffic
bursts and to release FW resources for the next bursts management.
When HW is faster than SW, it may be that all the FW resources are busy
in SW due to late polling.
In this case, due to wrong WQE counter masking, the fullness
calculation of the completions number is 0 while the queue is full.
Change the WQE counter masking to 16-bit wideness instead of the CQ
size mask as defined by the CQE format.
Fixes: c5f714e50b ("vdpa/mlx5: optimize completion queue poll")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The CQ configuration enables the collapse feature in HW what cause HW to
write all the completions in the first CQE.
When this feature is enabled the HW doesn't switch the owner bit when it
starts a new cycle of the CQ, not like working without the collapse
feature.
The current SW CQ polling wrongly added an assertion to validate the
owner bit switch what causes a panic in debug mode.
Remove the aforementioned assertion.
Fixes: c5f714e50b ("vdpa/mlx5: optimize completion queue poll")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
During vDPA device setup, if some error happens, event channel
release stucks at polling event channel.
Event channel fd is set to non-blocking in cqe setup, so if any
error happens before this function and after event channel created,
the pooling before releasing resources will stuck.
This patch moves event channel to non-blocking mode right after
creation.
Fixes: 8395927cdf ("vdpa/mlx5: prepare HW queues")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
A decision was made [1] to no longer support Make in DPDK, this patch
removes all Makefiles that do not make use of pkg-config, along with
the mk directory previously used by make.
[1] https://mails.dpdk.org/archives/dev/2020-April/162839.html
Signed-off-by: Ciara Power <ciara.power@intel.com>
Reviewed-by: Ruifeng Wang <ruifeng.wang@arm.com>
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Start a new release cycle with empty release notes.
The ABI version becomes 21.0.
The ABI major is back to normal, having only one number (21 vs 20.0).
The map files are updated to the new ABI major number (21).
The ABI exceptions are dropped.
Travis ABI check is disabled because compatibility is not preserved.
Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Ray Kinsella <mdr@ashroe.eu>
When a virtq is destroyed, the SW should be able to continue the virtq
processing from where the HW stopped.
The current destroy behavior in the driver saves the virtq state (used
and available indexes) only when LM is requested.
So, when LM is not requested the queue state is not saved and the SW
indexes stay invalid.
Save the virtq state in the virtq destroy process.
Fixes: bff7350110 ("vdpa/mlx5: prepare virtio queues")
Cc: stable@dpdk.org
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Xueming Li <xuemingl@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The driver CQ event management is done by non vhost library thread,
either the dpdk host thread or the internal vDPA driver thread.
When a queue is updated the CQ may be destroyed and created by the vhost
library thread via the queue state operation.
When the queue update feature was added, it didn't synchronize the CQ
management to the queue update what may cause invalid memory access.
Add the aforementioned synchronization by a new per device configuration
mutex.
Fixes: c47d6e8333 ("vdpa/mlx5: support queue update")
Signed-off-by: Xueming Li <xuemingl@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The device ready state in vhost lib is now defined as the state
that first queue pair is ready. And kick/callfd may be updated
by QEMU when ifc device is configured.
Although now ifc driver only supports one queue pair, it still
has to update callfd when working with QEMU. This patch fixes
this vring update problem by implementing the set_vring_state
callback.
Suggested-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Signed-off-by: Chenbo Xia <chenbo.xia@intel.com>
Acked-by: Xiao Wang <xiao.w.wang@intel.com>
Acked-by: Rosen Xu <rosen.xu@intel.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Vdpa example failed on vq setup, the api to get event queue of specified
core failed.
Internal api devx_query_eqn expects index of event queue vectors, no
need to use cpu id. As the doorbell handling thread is per device, it's
sufficient to use default event queue.
This patch uses the default id(0) as event queue index.
Fixes: 8395927cdf ("vdpa/mlx5: prepare HW queues")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
Vdpa device failed to initialize 2nd VQ during setup. From FW syndrome,
unsupported CQE size was specified in CQ initialization attributes.
The unsupported CQE size comes from uninitialized stack struct data, and
the struct has new fields defined recently which are not initialized in
vdpa code.
This patch initializes cq creation attributes with zero to avoid such
random data.
Fixes: 79a7e409a2 ("common/mlx5: prepare support of packet pacing")
Signed-off-by: Xueming Li <xuemingl@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
The issue is relevant only for the timer event modes: 0 and 1.
When the HW finishes to consume a burst of the guest Rx descriptors,
it creates a CQE in the CQ.
When traffic stops, the mlx5 driver arms the CQ to get a notification
when a specific CQE index is created - the index to be armed is the
next CQE index which should be polled by the driver.
The mlx5 driver configured the kernel driver to send notification to
the guest callfd in the same time of the armed CQE event.
It means that the guest was notified only for each first CQE in a
poll cycle, so if the driver polled CQEs of all the virtio queue
available descriptors, the guest was not notified again for the rest
because there was no any new CQE to trigger the guest notification.
Hence, the Rx queues might be stuck when the guest didn't work with
poll mode.
Remove prior kernel notification, and do manual notification after CQ
polling.
Fixes: a9dd7275a1 ("vdpa/mlx5: optimize notification events")
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Xueming Li <xuemingl@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When a virtq is destroyed by the driver, it must be removed from the
steering RQT which holds its reference.
The driver didn't remove the virtq from RQT before destroying it what
caused HW syndrome in virtq unset.
Remove the virtq from RQT before destroying it.
Fixes: 9f09b1ca15 ("vdpa/mlx5: recreate a virtq becoming enabled")
Cc: stable@dpdk.org
Signed-off-by: Xueming Li <xuemingl@mellanox.com>
Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
There are a lot of per virtq operations in the live migration
handling.
Before the driver support for queue update, when a virtq was not valid,
all the LM handling was terminated.
But now, when the driver supports queue update, the virtq can be invalid
as legal stage.
Skip invalid virtq in LM handling.
Fixes: c47d6e8333 ("vdpa/mlx5: support queue update")
Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Xueming Li <xuemingl@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
When dynamic flex parser feature is introduced, the support for misc
parameters 4 of flow table entry (FTE) match set is needed. The
structure of "mlx5_ifc_fte_match_param_bits" is extended with
"mlx5_ifc_fte_match_set_misc4_bits" at the end of it. The total size
of the FTE match set will be changed into 384 bytes from 320 bytes.
Low level user space driver (rdma-core) will have the validation of
the length of FTE match set. In the old release that no MISC4
supported in the rdma-core, and this will break the backward
compatibility, even if the MISC4 is not used in most cases, like
in vDPA driver.
In order not to break the compatibility old rdma-core, the length
adjustment needs to be done. In mlx5 vDPA driver, the lengths of
the matcher and value are both set to 320 without MISC4. There is
no need to change the structure definition, all bytes of the MISC4
will be discarded if it is not needed. Since the MISC4 parameter
is aligned with a 64B boundary and so does the whole FTE match set
parameter, there is no need to take any padding and alignment into
consideration when calculating the size.
Fixes: daa38a8924 ("net/mlx5: add flow translation of eCPRI header")
Signed-off-by: Bing Zhao <bingz@mellanox.com>
Acked-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
