Commit Graph

211 Commits

Author SHA1 Message Date
Ryan Libby
da1c42cefe ntb: quiet gcc -Wreturn-type
Reviewed by:	cem, markj
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D27553
2020-12-11 22:51:48 +00:00
Mark Johnston
93fb2b060b ntb: Fix the 32-bit build after r366969
Reported by:	Jenkins
MFC with:	r366969
2020-10-23 15:12:06 +00:00
Mark Johnston
6660ef6e91 ntb: Add Intel Xeon Gen3 support
The NTB hardware starting with Skylake has some changes to the register
map and the doorbell interface.  Add a new NTB_XEON_GEN3 device type and
use it to conditionalize driver logic that differs from the existing
Xeon code.

Reviewed by:	vangyzen
Discussed with:	cem, Bret Ketchum <Bret.Ketchum@dell.com>
MFC after:	1 month
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
Differential Revision:	https://reviews.freebsd.org/D26683
2020-10-23 14:16:52 +00:00
Mark Johnston
97441fab87 ntb: Fix an assertion to permit >= 32 doorbells
MFC after:	1 week
Sponsored by:	NetApp, Inc.
Sponsored by:	Klara, Inc.
2020-10-23 14:15:58 +00:00
Eric van Gyzen
c59370f055 ntb_tool: ubuf is too small to hold a human readable 64 bit value
ubuf buffer is too small. It should be 18 if a NULL is not needed,
or 19 to hold the NULL terminator for the full 64-BIT value plus
the 0x prefix.

Submitted by:	bret_ketchum@dell.com
Reviewed by:	markj mav
MFC after:	2 weeks
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D26893
2020-10-21 17:11:57 +00:00
Mateusz Guzik
391486af75 ntb: clean up empty lines in .c and .h files 2020-09-01 22:03:55 +00:00
Ruslan Bukin
c8597a1f9f o Don't include headers from iommu.h, include them from the header
consumers instead;
o Order includes properly.

Reviewed by:	kib
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D25878
2020-07-29 22:08:54 +00:00
Ruslan Bukin
9abb926538 Fix build.
Reviewed by:	kib
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D25879
2020-07-29 15:46:17 +00:00
Ruslan Bukin
ea4c01156a o Move the buswide_ctxs bitmap to iommu_unit and rename related functions.
o Rename bus_dma_dmar_load_ident() as well.

Reviewed by:	kib
Sponsored by:	DARPA/AFRL
Differential Revision:	https://reviews.freebsd.org/D25852
2020-07-28 16:08:14 +00:00
Mark Johnston
dace381270 ntb: Stop checking for failures from malloc(M_WAITOK).
PR:		240545
Submitted by:	Andrew Reiter <arr@watson.org>
Reviewed by:	cem, mav
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D25768
2020-07-23 14:03:24 +00:00
Pawel Biernacki
7029da5c36 Mark more nodes as CTLFLAG_MPSAFE or CTLFLAG_NEEDGIANT (17 of many)
r357614 added CTLFLAG_NEEDGIANT to make it easier to find nodes that are
still not MPSAFE (or already are but aren’t properly marked).
Use it in preparation for a general review of all nodes.

This is non-functional change that adds annotations to SYSCTL_NODE and
SYSCTL_PROC nodes using one of the soon-to-be-required flags.

Mark all obvious cases as MPSAFE.  All entries that haven't been marked
as MPSAFE before are by default marked as NEEDGIANT

Approved by:	kib (mentor, blanket)
Commented by:	kib, gallatin, melifaro
Differential Revision:	https://reviews.freebsd.org/D23718
2020-02-26 14:26:36 +00:00
Gleb Smirnoff
e87c494015 Although most of the NIC drivers are epoch ready, due to peer pressure
switch over to opt-in instead of opt-out for epoch.

Instead of IFF_NEEDSEPOCH, provide IFF_KNOWSEPOCH. If driver marks
itself with IFF_KNOWSEPOCH, then ether_input() would not enter epoch
when processing its packets.

Now this will create recursive entrance in epoch in >90% network
drivers, but will guarantee safeness of the transition.

Mark several tested drivers as IFF_KNOWSEPOCH.

Reviewed by:		hselasky, jeff, bz, gallatin
Differential Revision:	https://reviews.freebsd.org/D23674
2020-02-24 21:07:30 +00:00
Alexander Motin
0d9cef0a94 Add support for Hygon NTB PCI device in ntb_hw_amd driver.
Submitted by:	Pu Wen <puwen@hygon.cn>
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D23565
2020-02-14 15:04:56 +00:00
Gleb Smirnoff
0921628ddc Introduce flag IFF_NEEDSEPOCH that marks Ethernet interfaces that
supposedly may call into ether_input() without network epoch.

They all need to be reviewed before 13.0-RELEASE.  Some may need
be fixed.  The flag is not planned to be used in the kernel for
a long time.
2020-01-23 01:41:09 +00:00
Alexander Motin
95ba48d973 Make DMAR allow Intel NTB device to access its own BAR0.
I have no good explanation why it happens, but I found that in B2B mode
at least Xeon v4 NTB leaks accesses to its configuration memory at BAR0
originated from the link side to its host side.  DMAR predictably blocks
those, making access to remote scratchpad registers in B2B mode impossible.

This change creates identity mapping in DMAR covering the BAR0 addresses,
making the NTB work fine with DMAR enabled.  It seems like allowing single
4KB range at 32KB offset may be enough, but I don't see a reason to be so
specific.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-11-28 02:40:12 +00:00
Alexander Motin
241da27e24 Make the code slightly more compact.
There should be no functional change.

MFC after:	1 week
2019-11-28 02:28:12 +00:00
Alexander Motin
62ba8e8469 Report XLAT0 register for completeness. 2019-11-25 01:00:51 +00:00
Alexander Motin
f0dd6a1787 Call bus_dma_dmar_set_buswide(9) added in r354830.
PLX NTB sends translated DMA requests not only from itsels, but from all
slots and functions of its bus.  By default DMAR blocks those additional.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-11-19 02:03:10 +00:00
Alexander Motin
7f215e071e Make ntb(4) send bus_get_dma_tag() requests to parent buses passing real
bus' child pointers instead of grandchilds.

DMAR does not like requests from devices not parented directly by PCI.

MFC after:	2 weeks
2019-11-14 04:34:58 +00:00
Alexander Motin
028d96899b Add compact scraptchpad protocol for ntb_transport(4).
Previously ntb_transport(4) required at least 6 scratchpad registers,
plus 2 more for each additional memory window.  That is too much for some
configurations, where several drivers have to share resources of the same
NTB hardware.  This patch introduces new compact version of the protocol,
requiring only 3 scratchpad registers, plus one more for each additional
memory window.  The optimization is based on fact that neither of version,
number of windows or number of queue pairs really need more then one byte
each, and window sizes of 4GB are not very useful now.  The new protocol
is activated automatically when the configuration is low on scratchpad
registers, or it can be activated explicitly with loader tunable.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-11-10 03:37:45 +00:00
Alexander Motin
7aafa7c368 Allow splitting PLX NTB BAR2 into several memory windows.
Address Lookup Table (A-LUT) being enabled allows to specify separate
translation for each 1/128th or 1/256th of the BAR2.  Previously it was
used only to limit effective window size by blocking access through some
of A-LUT elements.  This change allows A-LUT elements to also point
different memory locations, providing to upper layers several (up to 128)
independent memory windows.  A-LUT hardware allows even more flexible
configurations than this, but NTB KPI have no way to manage that now.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-11-10 03:24:53 +00:00
Alexander Motin
96f556f504 NTB Tool: Test driver for NTB hardware drivers.
NTB Tool driver is meant for testing NTB hardware driver functionalities,
such as doorbell interrupts, link events, scratchpad registers and memory
windows.  This is a port of ntb_tool driver from Linux.  It has been
verified on top of AMD and PLX NTB HW drivers.

Submitted by:	Arpan Palit <arpan.palit@amd.com>
Cleaned up by:	mav
MFC after:	2 weeks
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D18819
2019-08-16 20:14:37 +00:00
Alexander Motin
ccfbff6d59 Fix i386 build.
MFC after:	2 weeks
2019-08-15 23:56:19 +00:00
Alexander Motin
105a032977 Limit memory window usage in ntb_transport to 256MB.
This adds safety net for the case of misconfigured NTB with too big
memory window, for which we may be unable to allocate a memory buffer,
which does not make much sense for the network interface.  While there,
fix the code to really work with asymmetric window sizes setup.

This makes driver just print warning message on boot instead of hanging
if too large memory window is configured.

MFC after:	2 weeks
Sponsored by:	iXsystems, Inc.
2019-08-15 22:55:59 +00:00
Alexander Motin
d8335b5339 Fix i386 LINT build after r351056.
MFC after:	13 days
2019-08-15 16:22:15 +00:00
Alexander Motin
6ddecf2b75 Implement new methods for Intel and PLX NTB.
This restores parity with AMD NTB driver.  Though without any drivers
supporting more then one peer and respective KPI modification to pass
peer index to most of the calls this addition is pretty useless now.

MFC after:	2 weeks
2019-08-15 14:11:11 +00:00
Alexander Motin
e67b122307 Add support for PCI Device ID 0x148B in ntb_hw_amd driver.
Submitted by:	Rajesh Kumar <rajesh1.kumar@amd.com>
MFC after:	2 weeks
Differential Revision:	https://reviews.freebsd.org/D20892
2019-08-14 22:35:11 +00:00
Alexander Motin
3a76d901d6 Include sys/lock.h, as told by man page.
MFC after:	1 week
2019-07-02 15:01:54 +00:00
Alexander Motin
7b96ad44dd Fix i386 LINT after r349594.
MFC after:	1 month
2019-07-02 07:47:11 +00:00
Alexander Motin
6683132d54 Add driver for NTB in AMD SoC.
This patch is the driver for NTB hardware in AMD SoCs (ported from Linux)
and enables the NTB infrastructure like Doorbells, Scratchpads and Memory
window in AMD SoC. This driver has been validated using ntb_transport and
if_ntb driver already available in FreeBSD.

Submitted by:	Rajesh Kumar <rajesh1.kumar@amd.com>
MFC after:	1 month
Relnotes:	yes
Differential Revision:	https://reviews.freebsd.org/D18774
2019-07-02 05:25:18 +00:00
Conrad Meyer
e2e050c8ef Extract eventfilter declarations to sys/_eventfilter.h
This allows replacing "sys/eventfilter.h" includes with "sys/_eventfilter.h"
in other header files (e.g., sys/{bus,conf,cpu}.h) and reduces header
pollution substantially.

EVENTHANDLER_DECLARE and EVENTHANDLER_LIST_DECLAREs were moved out of .c
files into appropriate headers (e.g., sys/proc.h, powernv/opal.h).

As a side effect of reduced header pollution, many .c files and headers no
longer contain needed definitions.  The remainder of the patch addresses
adding appropriate includes to fix those files.

LOCK_DEBUG and LOCK_FILE_LINE_ARG are moved to sys/_lock.h, as required by
sys/mutex.h since r326106 (but silently protected by header pollution prior
to this change).

No functional change (intended).  Of course, any out of tree modules that
relied on header pollution for sys/eventhandler.h, sys/lock.h, or
sys/mutex.h inclusion need to be fixed.  __FreeBSD_version has been bumped.
2019-05-20 00:38:23 +00:00
Alexander Motin
a8bc5594db Allow I/OAT of present Xeon E5/E7 to work thorugh PLX NTB.
Its a hack, we can't know/list all DMA engines, but this covers all
I/OAT of Xeon E5/E7 at least from Sandy Bridge till Skylake I saw.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2019-02-21 14:10:14 +00:00
Warner Losh
329e817fcc Reapply, with minor tweaks, r338025, from the original commit:
Remove unused and easy to misuse PNP macro parameter

Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely.  The 'table' parameter is now required to
have correct pointer (or array) type.  Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

  $ cat modpnpsize0.cocci
    @normaltables@
    identifier b,c;
    expression a,d,e;
    declarer MODULE_PNP_INFO;
    @@
     MODULE_PNP_INFO(a,b,c,d,
    -sizeof(d[0]),
     e);

    @singletons@
    identifier b,c,d;
    expression a;
    declarer MODULE_PNP_INFO;
    @@
     MODULE_PNP_INFO(a,b,c,&d,
    -sizeof(d),
     1);

  $ rg -l MODULE_PNP_INFO -- sys | \
    xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not.  So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).
Approved by: re (glen)
2018-09-26 17:12:14 +00:00
Conrad Meyer
b8e771e97a Back out r338035 until Warner is finished churning GSoC PNP patches
I was not aware Warner was making or planning to make forward progress in
this area and have since been informed of that.

It's easy to apply/reapply when churn dies down.
2018-08-19 00:46:22 +00:00
Conrad Meyer
faa319436f Remove unused and easy to misuse PNP macro parameter
Inspired by r338025, just remove the element size parameter to the
MODULE_PNP_INFO macro entirely.  The 'table' parameter is now required to
have correct pointer (or array) type.  Since all invocations of the macro
already had this property and the emitted PNP data continues to include the
element size, there is no functional change.

Mostly done with the coccinelle 'spatch' tool:

  $ cat modpnpsize0.cocci
    @normaltables@
    identifier b,c;
    expression a,d,e;
    declarer MODULE_PNP_INFO;
    @@
     MODULE_PNP_INFO(a,b,c,d,
    -sizeof(d[0]),
     e);

    @singletons@
    identifier b,c,d;
    expression a;
    declarer MODULE_PNP_INFO;
    @@
     MODULE_PNP_INFO(a,b,c,&d,
    -sizeof(d),
     1);

  $ rg -l MODULE_PNP_INFO -- sys | \
    xargs spatch --in-place --sp-file modpnpsize0.cocci

(Note that coccinelle invokes diff(1) via a PATH search and expects diff to
tolerate the -B flag, which BSD diff does not.  So I had to link gdiff into
PATH as diff to use spatch.)

Tinderbox'd (-DMAKE_JUST_KERNELS).
2018-08-19 00:22:21 +00:00
Alexander Motin
6e76bc32d4 Try to preallocate receive memory early.
We may not have enough contiguous memory later, when NTB connection get
established.  It is quite likely that NTB windows are symmetric and this
allocation remain, but even if not, we will just reallocate it later.

MFC after:	2 weeks
2018-01-31 01:04:36 +00:00
Alexander Motin
3986af1f65 Use bus_dmamem_alloc(9) KPI instead of contigmalloc(9).
I suppose it should make this code NUMA-aware with recent NUMA drop-in,
trying to allocate shared memory buffers from domain closer to NT-bridge.

MFC after:	2 weeks
2018-01-26 22:22:26 +00:00
Warner Losh
6efa5583c7 Fix typos from last commit, these should have been #. 2017-12-22 20:48:49 +00:00
Warner Losh
d2064cf030 Use '#' rather than some made up name for fields we want to ignore. 2017-12-22 17:53:27 +00:00
Alexander Motin
21811c24c6 Add initial support for Address Lookup Table (A-LUT).
When enabled by EEPROM, use it to relax translation address/size alignment
requirements for BAR2 window by 128 or 256 times.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2017-10-01 09:48:31 +00:00
Conrad Meyer
a64bf59c49 Add PNP metadata to a few drivers
An eventual devd(8) or other component should be able to scan buses and
automatically load drivers that match device ids described in this metadata.

Reviewed by:	imp
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D12364
2017-09-14 15:34:45 +00:00
Alexander Motin
83feae78cc Add second entry to LUT on a link side in B2B mode.
Each of two entries on a virtual side should have its counterpart on a
peer's link side.

MFC after:	1 week
2017-09-14 04:51:17 +00:00
Alexander Motin
3dcd9d784a Increase negotiation polling period from 10ms to 100ms.
There is no big need to burn CPU if other side may be not there yet.  For
example, the PLX hardware by default enables the NTB link up on reset, not
dependig on driver to do it.  In case of Intel hardware this also reduces
race between MSI-X workaround negotiation and upper layers, using the same
scratchpad registers in different time.

MFC after:	12 days
2017-09-02 13:28:45 +00:00
Alexander Motin
b7dd3fbede Make NTB drivers report more info via NewBus methods.
MFC after:	12 days
2017-09-02 11:56:16 +00:00
Alexander Motin
08dc78166d Link Interface has no Link Error registers.
MFC after:	13 days
2017-09-01 09:48:19 +00:00
Alexander Motin
84f8cfec2f Clear doorbell bits after masking them before processing.
In theory this allows to avoid one more expensive doorbell register read
later in some scenarios.  But in practice it also significantly increases
packet rate on PLX hardware, that I can't explain yet, possibly work-
arounding some interrupt delays.

MFC after:	13 days
Sponsored by:	iXsystems, Inc.
2017-08-31 21:37:22 +00:00
Alexander Motin
171e3d1a1f Remove unneeded pmap_change_attr() calls.
Reported by:	kib
MFC after:	13 days
2017-08-31 17:02:06 +00:00
Alexander Motin
0faf59149c Add/polish some defines.
MFC after:	13 days
2017-08-31 16:32:11 +00:00
Alexander Motin
b6c46372af Fix port control for PEX 8749.
That chip has three Station Ports, so previous address math was incorrect.

MFC after:	13 days
Sponsored by:	iXsystems, Inc.
2017-08-31 13:41:44 +00:00
Alexander Motin
c7dabb6563 Make ntb_set_ctx() always generate fake link event.
It allows application driver get initial link state without racing with
hardware interrupts, thanks to the context rmlock held here.

MFC after:	1 week
Sponsored by:	iXsystems, Inc.
2017-08-31 10:59:39 +00:00