198 Commits

Author SHA1 Message Date
Emmanuel Vadot
105a37cac7 linuxkpi: Add more pci functions needed by DRM
-pci_get_class : This function search for a matching pci device based on
   the class/subclass and returns a newly created pci_dev.
 - pci_{save,restore}_state : This is analogous to ours with the same name
 - pci_is_root_bus : Return true if this is the root bus
 - pci_get_domain_bus_and_slot : This function search for a matching pci
   device based on domain, bus and slot/function concat into a single
   unsigned int (devfn) and returns a newly created pci_dev
 - pci_bus_{read,write}_config* : Read/Write to the config space.

While here add some helper function to alloc and fill the pci_dev struct.

Reviewed by:   hselasky, bz (older version)
Differential Revision:	   https://reviews.freebsd.org/D27550
2021-01-12 12:31:00 +01:00
Hans Petter Selasky
f8f5b459d2 Update user access region, UAR, APIs in the core in mlx5core.
This change include several changes as listed below all related to UAR.
UAR is a special PCI memory area where the so-called doorbell register and
blue flame register live. Blue flame is a feature for sending small packets
more efficiently via a PCI memory page, instead of using PCI DMA.

- All structures and functions named xxx_uuars were renamed into xxx_bfreg.
- Remove partially implemented Blueflame support from mlx5en(4) and mlx5ib.
- Implement blue flame register allocator.
- Use blue flame register allocator in mlx5ib.
- A common UAR page is now allocated by the core to support doorbell register
  writes for all of mlx5en and mlx5ib, instead of allocating one UAR per
  sendqueue.
- Add support for DEVX query UAR.
- Add support for 4K UAR for libmlx5.

Linux commits:
7c043e908a74ae0a935037cdd984d0cb89b2b970
2f5ff26478adaff5ed9b7ad4079d6a710b5f27e7
0b80c14f009758cefeed0edff4f9141957964211
30aa60b3bd12bd79b5324b7b595bd3446ab24b52
5fe9dec0d045437e48f112b8fa705197bd7bc3c0
0118717583cda6f4f36092853ad0345e8150b286
a6d51b68611e98f05042ada662aed5dbe3279c1e

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 13:33:46 +01:00
Hans Petter Selasky
9a47ae044b Bump driver versions for mlx5en(4) and mlx4en(4).
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:55 +01:00
Hans Petter Selasky
480570dbb3 Fixes for SRIOV in mlx5core.
- call pci_iov_detach() on detaching from PCI device to take care of hang
  on destroying VFs after PF is down.

- disable eswitch SRIOV support right after pci_iov_detach(),
  else the eswitch cleanup sometimes occur while the SRIOV flow table
  is still present.

Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:55 +01:00
Hans Petter Selasky
98140747ca Update the PCI ID list in mlx5core.
- Add descriptions for new devices.
- Add support for Bluefield.

MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:54 +01:00
Hans Petter Selasky
82c7abe778 The "unsigned" type is the same like "unsigned int".
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:54 +01:00
Hans Petter Selasky
87b3c8cc99 Fix spelling in mlx5core.
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:54 +01:00
Hans Petter Selasky
daa150aaa3 Properly handle case where firmware dump returns more registers on second pass
in mlx5core.

Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:54 +01:00
Hans Petter Selasky
50a9f8bbc1 Downgrade error about missing VSC to warning and make messages consistent
in mlx5core.

Submitted by: kib@
MFC after: 1 week
Sponsored by: Mellanox Technologies // NVIDIA Networking
2021-01-08 12:35:53 +01:00
Hans Petter Selasky
7eefcb5eea Make mlx5_cmd_exec_cb() a safe API in mlx5core.
APIs that have deferred callbacks should have some kind of cleanup
function that callers can use to fence the callbacks. Otherwise things
like module unloading can lead to dangling function pointers, or worse.

The IB MR code is the only place that calls this function and had a
really poor attempt at creating this fence. Provide a good version in
the core code as future patches will add more places that need this
fence.

Linux commit:
e355477ed9e4f401e3931043df97325d38552d54

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:15:03 +00:00
Hans Petter Selasky
f34f0a65b2 Report EQE data upon CQ completion in mlx5core.
Report EQE data upon CQ completion to let upper layers use this data.

Linux commit:
4e0e2ea1886afe8c001971ff767f6670312a9b04

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:10:53 +00:00
Hans Petter Selasky
ffdb195f31 Enhance the mlx5_core_create_cq() function in mlx5core.
Enhance mlx5_core_create_cq() to get the command out buffer from the
callers to let them use the output.

Linux commit:
38164b771947be9baf06e78ffdfb650f8f3e908e

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:06:10 +00:00
Hans Petter Selasky
4a64b690f1 Use mlx5core to create/destroy all Dynamically Connected Targets, DCTs.
To prevent a hardware memory leak when a DEVX DCT object is destroyed
without calling drain DCT before, (e.g. under cleanup flow), need to
manage its creation and destruction via mlx5 core.

Linux commit:
c5ae1954c47d3fd8815bd5a592aba18702c93f33

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:03:18 +00:00
Hans Petter Selasky
194ddc011a Properly cleanup driver during remove_one() in mlx5core.
Cleanup all host resources, SYSCTLs, MSIX vectors and memory used
by the host and only leave the device allocated memory behind, if any,
because it may still be in use, when the PCI remove function is called.
Else future probe calls may fail due to SYSCTLs already existing.

MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-07 17:46:49 +00:00
Konstantin Belousov
596b98ba16 mlx5 sriov: Add controls for VFs to set port/node GUIDs.
Setting GUIDs make RoCE offloads functional on VFs.

Reported and tested by:	chuck
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:32:17 +00:00
Konstantin Belousov
cca1f7a12f mlx5 sriov: add error message for failed MAC programming on VF.
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:30:52 +00:00
Konstantin Belousov
2ea114b34e mlx5en: Implement SIOCGIFDOWNREASON.
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:27:03 +00:00
Konstantin Belousov
62daa4b6e8 mlx5_core: add mlx5_query_pddr().
And use it in mlx5_query_pddr_range_info() instead of direct register
access.

Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:25:55 +00:00
Konstantin Belousov
e088db5eae mlx5_core: Import PDDR register definitions
PDDR (Port Diagnostics Database Register) is used to read the physical
layer debug database, which contains helpful troubleshooting information
regarding the state of the link.

PDDR register can only be queried when PCAM register reports it as
supported in its register mask. A new helper macro was added to
the MLX5_CAP_* infrastructure in order to access this mask.

Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:23:51 +00:00
Hans Petter Selasky
1866c98e64 Infiniband clients must be attached and detached in a specific order in ibcore.
Currently the linking order of the infiniband, IB, modules decide in which
order the clients are attached and detached. For example one IB client may
use resources from another IB client. This can lead to a potential deadlock
at shutdown. For example if the ipoib is unregistered after the ib_multicast
client is detached, then if ipoib is using multicast addresses a deadlock may
happen, because ib_multicast will wait for all its resources to be freed before
returning from the remove method.

Fix this by using module_xxx_order() instead of module_xxx().

Differential Revision:	https://reviews.freebsd.org/D23973
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-07-06 08:50:11 +00:00
Konstantin Belousov
92d8df2f37 mlx5_core: remove unneccessary LFENCE instruction.
Use fence instead of barrier, which is optimized to take advantage of
the x86 TSO memory model.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2020-07-02 10:44:45 +00:00
Hans Petter Selasky
bf43f9812c Sync with Linux packet pacing enhancements in mlx5en(4).
Linux commit:
05d3ac978ed25b753bfe34fe76c50c31ee506a82

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-26 07:41:46 +00:00
Hans Petter Selasky
9550e3403e Update the TLS capability bit after recent PRM changes in mlx5en(4).
A CX6-DX firmware version equal to or newer than 12.27.0372 is
now required.

Sponsored by:	Mellanox Technologies
2020-05-25 12:31:48 +00:00
Konstantin Belousov
d0a4068359 mlx5_core: add more port module event types to decode.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	3 days
2020-05-20 11:20:45 +00:00
Konstantin Belousov
6418350cf4 mlx5_core: add "PMD type not enabled" port module event type.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	3 days
2020-05-20 11:10:10 +00:00
Konstantin Belousov
4ad58ea85b mlx5_core: lower the severity of message noting that no SR-IOV cap is present.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:47:14 +00:00
Konstantin Belousov
bbcb656af2 mlx5: Route NIC_VPORT_CHANGE events to eswitch code.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:44:48 +00:00
Konstantin Belousov
90959e7e37 mlx5: Read number of VF ports from the SR-IOV cap.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:43:39 +00:00
Konstantin Belousov
18a70fa574 mlx5: Use eswitch interface to configure VFs steering.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:40:26 +00:00
Konstantin Belousov
f6ca0b216a mlx5: Integrate eswitch and mpfs management code.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:33:39 +00:00
Konstantin Belousov
91ad1bd953 mlx5: Restore eswitch management code from attic.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:30:56 +00:00
Konstantin Belousov
9dfa078252 mlx5: Basic PCIe side of SR-IOV support.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:17:01 +00:00
Konstantin Belousov
e19a968f15 mlx5_core: add sysctls to report device capabilities.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 21:54:32 +00:00
Pawel Biernacki
7029da5c36 Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE.  All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by:	kib (mentor, blanket)
Commented by:	kib, gallatin, melifaro
Differential Revision:	https://reviews.freebsd.org/D23718
2020-02-26 14:26:36 +00:00
Hans Petter Selasky
f14d849862 Add support for disabling and polling MSIX interrupts in mlx5core.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-02-12 09:58:19 +00:00
Hans Petter Selasky
e48813009c Widen EPOCH(9) usage in mlx5en(4).
Make completion event path mostly lockless using EPOCH(9).

Implement a mechanism using EPOCH(9) which allows us to make
the callback path for completion events mostly lockless.

Simplify draining callback events using epoch_wait().

While at it make sure all receive completion callbacks are
covered by the network EPOCH(9), because this is required
when calling if_input() and ether_input() after r357012.

Sponsored by:	Mellanox Technologies
2020-01-30 12:35:13 +00:00
Konstantin Belousov
0cf6ff0a77 mlx5: Do not poke hardware for statistic after teardown is started.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-05 15:21:13 +00:00
Hans Petter Selasky
04f1690bf0 Add basic support for TCP/IP based hardware TLS offload to mlx5core.
The hardware offload is primarily targeted for TLS v1.2 and v1.3,
using AES 128/256 bit pre-shared keys. This patch adds all the needed
hardware structures, capabilites and firmware commands.

Sponsored by:	Mellanox Technologies
2019-12-05 15:16:19 +00:00
Konstantin Belousov
c47b170b02 mlx5: Do not try to enable fwdumps if scan space did not responded.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-02 14:22:55 +00:00
Konstantin Belousov
9a4af029b0 mlx5: Downgrade assert about misbehaving hardware to error message.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-02 14:21:40 +00:00
Hans Petter Selasky
67a254a074 Fix build failure for 32-bit platforms after r352991, due to
incorrect printf() formatter string.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 12:02:14 +00:00
Hans Petter Selasky
d3d47408fc Bump driver version for mlx5core, mlx5en(4) and mlx5ib(4).
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:15:35 +00:00
Hans Petter Selasky
fedc7bd218 Print numeric error_type and module_status in mlx5core
in case the strings are not available.

Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:06:01 +00:00
Hans Petter Selasky
63dd1be931 Improve mlx5_fwdump_prep logging in mlx5core.
Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:01:05 +00:00
Hans Petter Selasky
7bf46a63ce Randomize the delay when waiting for VSC flag in mlx5core.
The PRM suggests random 0 - 10ms to prevent multiple waiters on the same
interval in order to avoid starvation.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:59:44 +00:00
Hans Petter Selasky
59efbf791d Wait for FW readiness before initializing command interface in mlx5core.
Before attempting to initialize the command interface we must wait till
the fw_initializing bit is clear.

If we fail to meet this condition the hardware will drop our
configuration, specifically the descriptors page address.  This scenario
can happen when the firmware is still executing an FLR flow and did not
finish yet so the driver needs to wait for that to finish.

Linux commits:
6c780a0267b8
b8a92577f4be.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:53:28 +00:00
Hans Petter Selasky
3e40712eb0 Return an error from ioctl(MLX5_FW_RESET) if reset was rejected in mlx5core.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:24:13 +00:00
Hans Petter Selasky
048ddb58bc Move EEPROM information query from a sysctl in mlx5en(4) to an ioctl
in mlx5core. The EEPROM information is not only a property of the
mlx5en(4) driver.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:14:55 +00:00
Hans Petter Selasky
c28ef24918 Import Linux code to query/set buffer state in mlx5en(4).
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:05:34 +00:00
Hans Petter Selasky
a2f4f59ca8 Unify prints in mlx5core.
All prints in mlx5core should use on of the macros:
mlx5_core_err/dbg/warn

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:48:01 +00:00