Commit Graph

16 Commits

Author SHA1 Message Date
Xueming Li
476048d546 vdpa/mlx5: make statistics counter persistent
In order to speed-up the device suspend and resume, make the statistics
counters persistent in reconfiguration until the device gets removed.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2022-05-09 21:39:58 +02:00
Michael Baum
a3ade5e34d doc: add shared guide for mlx5 drivers
Adds new documentation for MLX5 common driver that contains:
 - Its features list (doesn't exist for now).
 - Its devargs description.
 - Device configuration information and tutorial.
 - Quick Start Guide for Mellanox OFED/EN.

Move into this doc all shared information from other MLX5 PMD docs and
add them reference to new common doc.

Signed-off-by: Michael Baum <michaelba@nvidia.com>
Reviewed-by: Raslan Darawsheh <rasland@nvidia.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2022-02-23 15:57:42 +01:00
Thomas Monjalon
cf8a8a8f48 vdpa/mlx5: support Sub-Function
RoCE disabling requirement is based on PCI address.
In order to support Sub-Function, a conversion is needed
in the case of an auxiliary device.

SF device can be probed with such devargs string:
  auxiliary:mlx5_core.sf.<id>,class=vdpa

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
Acked-by: Viacheslav Ovsiienko <viacheslavo@nvidia.com>
2021-07-22 00:11:14 +02:00
Matan Azrad
99f9d799ce vdpa/mlx5: improve interrupt management
The driver should notify the guest for each traffic burst detected by CQ
polling.

The CQ polling trigger is defined by `event_mode` device argument,
either by busy polling on all the CQs or by blocked call to HW
completion event using DevX channel.

Also, the polling event modes can move to blocked call when the
traffic rate is low.

The current blocked call uses the EAL interrupt API suffering a lot
of overhead in the API management and serve all the drivers and
libraries using only single thread.

Use blocking FD of the DevX channel in order to do blocked call
directly by the DevX channel FD mechanism.

Signed-off-by: Matan Azrad <matan@nvidia.com>
Acked-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2021-05-04 10:22:17 +02:00
Xueming Li
1f93bee4e7 vdpa/mlx5: add hardware queue moderation
The next parameters control the HW queue moderation feature.
This feature helps to control the traffic performance and latency
trade-off.

Each packet completion report from HW to SW requires CQ processing by SW
and triggers interrupt for the guest driver. Interrupt report and
handling cost CPU cycles and time and the amount of this affects
directly on packet performance and latency.

hw_latency_mode parameters [int]
  0, HW default.
  1, Latency is counted from the first packet completion report.
  2, Latency is counted from the last packet completion.
hw_max_latency_us parameters [int]
  0 - 4095, The maximum time in microseconds that packet completion
  report can be delayed.
hw_max_pending_comp parameter [int]
  0 - 65535, The maximum number of pending packets completions in an HW
queue.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2021-01-08 18:07:56 +01:00
Xueming Li
05421ec938 vdpa/mlx5: set default event mode to polling
For better performance and latency, this patch sets default event
handling mode to polling mode which uses dedicate thread per device to
poll and process event.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2021-01-08 18:07:55 +01:00
Xueming Li
5cf3fd3af4 vdpa/mlx5: add CPU core parameter to bind polling thread
This patch adds new device argument to specify cpu core affinity to
event polling thread for better latency and throughput. The thread
could be also located by name "vDPA-mlx5-<id>".

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2021-01-08 18:07:55 +01:00
Xueming Li
6956a48cab vdpa/mlx5: set polling mode default delay to zero
To improve throughput and latency, this patch allows Rx polling timer
delay to 0us.

Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Acked-by: Matan Azrad <matan@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2021-01-08 18:07:55 +01:00
Xueming Li
0474419bae vdpa/mlx5: handle hardware error
When hardware error happens, vdpa didn't get such information and leave
driver in silent: working state but no response.

This patch subscribes firmware virtq error event and try to recover max
3 times in 3 seconds, stop virtq if max retry number reached.

When error happens, PMD log in warning level. If failed to recover,
outputs error log. Query virtq statistics to get error counters report.

Acked-by: Matan Azrad <matan@nvidia.com>
Signed-off-by: Xueming Li <xuemingl@nvidia.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-11-03 23:35:05 +01:00
Bruce Richardson
8809f78c7d doc: fix driver names
Since the built driver filenames have changed in DPDK 20.11, we need to
update the driver doc to match.

Most drivers start their section with the driver filename highlighted in
bold, while a number were missing the highlight. When updating the names,
add the markers for bold text to any missing it, so as to have things more
consistent.

Fixes: a20b2c01a7 ("build: standardize component names and defines")

Signed-off-by: Bruce Richardson <bruce.richardson@intel.com>
Reviewed-by: David Marchand <david.marchand@redhat.com>
Reviewed-by: Rosen Xu <rosen.xu@intel.com>
Acked-by: Ajit Khaparde <ajit.khaparde@broadcom.com>
2020-11-03 16:23:03 +01:00
Ciara Power
dde524d3ff doc: remove references to make from vdpadevs guide
Make is no longer supported for compiling DPDK, references are now
removed in the documentation.

Signed-off-by: Ciara Power <ciara.power@intel.com>
Acked-by: Matan Azrad <matan@nvidia.com>
2020-10-22 22:54:05 +02:00
Sergey Madaminov
251691f107 doc: add a supported NIC in mlx5 vDPA guide
Update the docs, adding MCX621102AN-ADAT to the list of NICs supported
by MLX5 vDPA driver.

Suggested-by: William Tu <u9012063@gmail.com>
Signed-off-by: Sergey Madaminov <sergey.madaminov@gmail.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-07-30 00:41:23 +02:00
Matan Azrad
edc6391e45 vdpa/mlx5: control completion queue event mode
The CQ polling is necessary in order to manage guest notifications when
the guest doesn't work with poll mode (callfd != -1).

The CQ polling scheduling method can affect the host CPU utilization and
the traffic bandwidth.

Define 3 modes to control the CQ polling scheduling:

1. A timer thread which automatically adjusts its delays to the coming
   traffic rate.
2. A timer thread with fixed delay time.
3. Interrupts: Each CQE burst arms the CQ in order to get an interrupt
   event in the next traffic burst.

When traffic becomes off, mode 3 is taken automatically.

The interrupt management takes a lot of CPU cycles but forward traffic
event to the guest very fast.

Timer thread save the interrupt overhead but may add delay for the guest
notification.

Add device arguments to control on the mode.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-06-30 14:52:30 +02:00
Thomas Monjalon
ee76bddc76 doc: fix naming of Mellanox devices
The devices of the family ConnectX may have two letters as suffix.
Such suffix is preceded with a space and the second x is lowercase:
    - ConnectX-4 Lx
    - ConnectX-5 Ex
    - ConnectX-6 Dx

Uppercase of the device family name BlueField is also fixed.

The lists of supported devices are fixed.

Signed-off-by: Thomas Monjalon <thomas@monjalon.net>
2020-02-25 15:55:54 +01:00
Matan Azrad
9f71bada9d doc: update mlx5 vDPA requirements
The first Mellanox OFED version to support mlx5 vDPA driver is 5.0.

Signed-off-by: Matan Azrad <matan@mellanox.com>
2020-02-25 10:48:04 +01:00
Matan Azrad
95276abaaf vdpa/mlx5: introduce Mellanox vDPA driver
Add a new driver to support vDPA operations by Mellanox devices.

The first Mellanox devices which support vDPA operations are
ConnectX-6 Dx and Bluefield1 HCA for their PF ports and VF ports.

This driver is depending on rdma-core like the mlx5 PMD, also it is
going to use mlx5 DevX to create HW objects directly by the FW.
Hence, the common/mlx5 library is linked to the mlx5_vdpa driver.

This driver will not be compiled by default due to the above
dependencies.

Register a new log type for this driver.

Signed-off-by: Matan Azrad <matan@mellanox.com>
Acked-by: Viacheslav Ovsiienko <viacheslavo@mellanox.com>
Reviewed-by: Maxime Coquelin <maxime.coquelin@redhat.com>
2020-02-05 09:51:21 +01:00