189 Commits

Author SHA1 Message Date
Hans Petter Selasky
7eefcb5eea Make mlx5_cmd_exec_cb() a safe API in mlx5core.
APIs that have deferred callbacks should have some kind of cleanup
function that callers can use to fence the callbacks. Otherwise things
like module unloading can lead to dangling function pointers, or worse.

The IB MR code is the only place that calls this function and had a
really poor attempt at creating this fence. Provide a good version in
the core code as future patches will add more places that need this
fence.

Linux commit:
e355477ed9e4f401e3931043df97325d38552d54

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:15:03 +00:00
Hans Petter Selasky
f34f0a65b2 Report EQE data upon CQ completion in mlx5core.
Report EQE data upon CQ completion to let upper layers use this data.

Linux commit:
4e0e2ea1886afe8c001971ff767f6670312a9b04

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:10:53 +00:00
Hans Petter Selasky
ffdb195f31 Enhance the mlx5_core_create_cq() function in mlx5core.
Enhance mlx5_core_create_cq() to get the command out buffer from the
callers to let them use the output.

Linux commit:
38164b771947be9baf06e78ffdfb650f8f3e908e

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:06:10 +00:00
Hans Petter Selasky
4a64b690f1 Use mlx5core to create/destroy all Dynamically Connected Targets, DCTs.
To prevent a hardware memory leak when a DEVX DCT object is destroyed
without calling drain DCT before, (e.g. under cleanup flow), need to
manage its creation and destruction via mlx5 core.

Linux commit:
c5ae1954c47d3fd8815bd5a592aba18702c93f33

MFC after:	1 week
Sponsored by:	Mellanox Technologies // NVIDIA Networking
2020-11-16 10:03:18 +00:00
Hans Petter Selasky
194ddc011a Properly cleanup driver during remove_one() in mlx5core.
Cleanup all host resources, SYSCTLs, MSIX vectors and memory used
by the host and only leave the device allocated memory behind, if any,
because it may still be in use, when the PCI remove function is called.
Else future probe calls may fail due to SYSCTLs already existing.

MFC after:		1 week
Sponsored by:		Mellanox Technologies // NVIDIA Networking
2020-10-07 17:46:49 +00:00
Konstantin Belousov
596b98ba16 mlx5 sriov: Add controls for VFs to set port/node GUIDs.
Setting GUIDs make RoCE offloads functional on VFs.

Reported and tested by:	chuck
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:32:17 +00:00
Konstantin Belousov
cca1f7a12f mlx5 sriov: add error message for failed MAC programming on VF.
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:30:52 +00:00
Konstantin Belousov
2ea114b34e mlx5en: Implement SIOCGIFDOWNREASON.
Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:27:03 +00:00
Konstantin Belousov
62daa4b6e8 mlx5_core: add mlx5_query_pddr().
And use it in mlx5_query_pddr_range_info() instead of direct register
access.

Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:25:55 +00:00
Konstantin Belousov
e088db5eae mlx5_core: Import PDDR register definitions
PDDR (Port Diagnostics Database Register) is used to read the physical
layer debug database, which contains helpful troubleshooting information
regarding the state of the link.

PDDR register can only be queried when PCAM register reports it as
supported in its register mask. A new helper macro was added to
the MLX5_CAP_* infrastructure in order to access this mask.

Sponsored by:	Mellanox Technologies - Nvidia
MFC after:	1 week
2020-08-31 16:23:51 +00:00
Hans Petter Selasky
1866c98e64 Infiniband clients must be attached and detached in a specific order in ibcore.
Currently the linking order of the infiniband, IB, modules decide in which
order the clients are attached and detached. For example one IB client may
use resources from another IB client. This can lead to a potential deadlock
at shutdown. For example if the ipoib is unregistered after the ib_multicast
client is detached, then if ipoib is using multicast addresses a deadlock may
happen, because ib_multicast will wait for all its resources to be freed before
returning from the remove method.

Fix this by using module_xxx_order() instead of module_xxx().

Differential Revision:	https://reviews.freebsd.org/D23973
MFC after:		1 week
Sponsored by:		Mellanox Technologies
2020-07-06 08:50:11 +00:00
Konstantin Belousov
92d8df2f37 mlx5_core: remove unneccessary LFENCE instruction.
Use fence instead of barrier, which is optimized to take advantage of
the x86 TSO memory model.

Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2020-07-02 10:44:45 +00:00
Hans Petter Selasky
bf43f9812c Sync with Linux packet pacing enhancements in mlx5en(4).
Linux commit:
05d3ac978ed25b753bfe34fe76c50c31ee506a82

MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-05-26 07:41:46 +00:00
Hans Petter Selasky
9550e3403e Update the TLS capability bit after recent PRM changes in mlx5en(4).
A CX6-DX firmware version equal to or newer than 12.27.0372 is
now required.

Sponsored by:	Mellanox Technologies
2020-05-25 12:31:48 +00:00
Konstantin Belousov
d0a4068359 mlx5_core: add more port module event types to decode.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	3 days
2020-05-20 11:20:45 +00:00
Konstantin Belousov
6418350cf4 mlx5_core: add "PMD type not enabled" port module event type.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	3 days
2020-05-20 11:10:10 +00:00
Konstantin Belousov
4ad58ea85b mlx5_core: lower the severity of message noting that no SR-IOV cap is present.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:47:14 +00:00
Konstantin Belousov
bbcb656af2 mlx5: Route NIC_VPORT_CHANGE events to eswitch code.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:44:48 +00:00
Konstantin Belousov
90959e7e37 mlx5: Read number of VF ports from the SR-IOV cap.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:43:39 +00:00
Konstantin Belousov
18a70fa574 mlx5: Use eswitch interface to configure VFs steering.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:40:26 +00:00
Konstantin Belousov
f6ca0b216a mlx5: Integrate eswitch and mpfs management code.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:33:39 +00:00
Konstantin Belousov
91ad1bd953 mlx5: Restore eswitch management code from attic.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:30:56 +00:00
Konstantin Belousov
9dfa078252 mlx5: Basic PCIe side of SR-IOV support.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 22:17:01 +00:00
Konstantin Belousov
e19a968f15 mlx5_core: add sysctls to report device capabilities.
Reviewed by:	hselasky
Sponsored by:	Mellanox Technologies
MFC after:	2 weeks
2020-03-18 21:54:32 +00:00
Pawel Biernacki
7029da5c36 Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE.  All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by:	kib (mentor, blanket)
Commented by:	kib, gallatin, melifaro
Differential Revision:	https://reviews.freebsd.org/D23718
2020-02-26 14:26:36 +00:00
Hans Petter Selasky
f14d849862 Add support for disabling and polling MSIX interrupts in mlx5core.
MFC after:	1 week
Sponsored by:	Mellanox Technologies
2020-02-12 09:58:19 +00:00
Hans Petter Selasky
e48813009c Widen EPOCH(9) usage in mlx5en(4).
Make completion event path mostly lockless using EPOCH(9).

Implement a mechanism using EPOCH(9) which allows us to make
the callback path for completion events mostly lockless.

Simplify draining callback events using epoch_wait().

While at it make sure all receive completion callbacks are
covered by the network EPOCH(9), because this is required
when calling if_input() and ether_input() after r357012.

Sponsored by:	Mellanox Technologies
2020-01-30 12:35:13 +00:00
Konstantin Belousov
0cf6ff0a77 mlx5: Do not poke hardware for statistic after teardown is started.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-05 15:21:13 +00:00
Hans Petter Selasky
04f1690bf0 Add basic support for TCP/IP based hardware TLS offload to mlx5core.
The hardware offload is primarily targeted for TLS v1.2 and v1.3,
using AES 128/256 bit pre-shared keys. This patch adds all the needed
hardware structures, capabilites and firmware commands.

Sponsored by:	Mellanox Technologies
2019-12-05 15:16:19 +00:00
Konstantin Belousov
c47b170b02 mlx5: Do not try to enable fwdumps if scan space did not responded.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-02 14:22:55 +00:00
Konstantin Belousov
9a4af029b0 mlx5: Downgrade assert about misbehaving hardware to error message.
Sponsored by:	Mellanox Technologies
MFC after:	1 week
2019-12-02 14:21:40 +00:00
Hans Petter Selasky
67a254a074 Fix build failure for 32-bit platforms after r352991, due to
incorrect printf() formatter string.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 12:02:14 +00:00
Hans Petter Selasky
d3d47408fc Bump driver version for mlx5core, mlx5en(4) and mlx5ib(4).
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:15:35 +00:00
Hans Petter Selasky
fedc7bd218 Print numeric error_type and module_status in mlx5core
in case the strings are not available.

Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:06:01 +00:00
Hans Petter Selasky
63dd1be931 Improve mlx5_fwdump_prep logging in mlx5core.
Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 11:01:05 +00:00
Hans Petter Selasky
7bf46a63ce Randomize the delay when waiting for VSC flag in mlx5core.
The PRM suggests random 0 - 10ms to prevent multiple waiters on the same
interval in order to avoid starvation.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:59:44 +00:00
Hans Petter Selasky
59efbf791d Wait for FW readiness before initializing command interface in mlx5core.
Before attempting to initialize the command interface we must wait till
the fw_initializing bit is clear.

If we fail to meet this condition the hardware will drop our
configuration, specifically the descriptors page address.  This scenario
can happen when the firmware is still executing an FLR flow and did not
finish yet so the driver needs to wait for that to finish.

Linux commits:
6c780a0267b8
b8a92577f4be.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:53:28 +00:00
Hans Petter Selasky
3e40712eb0 Return an error from ioctl(MLX5_FW_RESET) if reset was rejected in mlx5core.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:24:13 +00:00
Hans Petter Selasky
048ddb58bc Move EEPROM information query from a sysctl in mlx5en(4) to an ioctl
in mlx5core. The EEPROM information is not only a property of the
mlx5en(4) driver.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:14:55 +00:00
Hans Petter Selasky
c28ef24918 Import Linux code to query/set buffer state in mlx5en(4).
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 10:05:34 +00:00
Hans Petter Selasky
a2f4f59ca8 Unify prints in mlx5core.
All prints in mlx5core should use on of the macros:
mlx5_core_err/dbg/warn

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:48:01 +00:00
Hans Petter Selasky
c9bb26aef1 Add proper print in case of 0x0 health syndrome in mlx5core.
In case of health counter fails to increment it indicates a bad device health.
In case when the syndrome indicated by firmware is 0x0, this indicates that
firmware is unable to respond to initialization segment reads.
Add proper print in this case.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:46:14 +00:00
Hans Petter Selasky
95c05e056f Add missing blank line at the end of the print in mlx5core.
Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:45:07 +00:00
Hans Petter Selasky
4bc8507b82 Remove no longer needed fwdump register tables from mlx5core.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:43:48 +00:00
Hans Petter Selasky
4745819044 Read rege map from crdump scan space in mlx5core.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:40:23 +00:00
Hans Petter Selasky
4d98df72cd Use the MLX5_VSC_DOMAIN_SEMAPHORES constant instead of hand-rolled symbol
in mlx5core.

Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:33:38 +00:00
Hans Petter Selasky
04910901bc Move mlx5_ifc_vsc_space_bits and mlx5_ifc_vsc_addr_bits to mlx5_ifc.h.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:32:41 +00:00
Hans Petter Selasky
e456decc55 Make the mlx5_vsc_wait_on_flag(9) function global.
Submitted by:	kib@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:31:36 +00:00
Hans Petter Selasky
111b57c359 Add port module event software counters in mlx5core.
While at it, fixup PME based on latest PRM defines.

Submitted by:	slavash@
MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:29:55 +00:00
Hans Petter Selasky
6226306b47 Cleanup naming of IRQ vectors in mlx5en.
Remove unused IRQ naming functions and arrays.

MFC after:	3 days
Sponsored by:	Mellanox Technologies
2019-10-02 09:23:33 +00:00