Commit Graph

113 Commits

Author SHA1 Message Date
Conrad Meyer
b23896a221 if_ntb: Don't roundup MW size to full BAR size unnecessarily
Note that the MW allocation still must be BAR *aligned*.  So, this only
loosens the constraints on MW allocation slightly.  BAR-aligned does not
play well with large (GB+) BAR sizes.

Going forward, if anyone cares about if_ntb on very large BARs, I
suggest they add functionality to allocate a smaller window than the BAR
size, and set the BAR range to cover a window much larger than the
allocated window.  This will require negotiating a window offset and
limit for protocol traffic.  None of this is implemented in this
revision.

Sponsored by:	EMC / Isilon Storage Division
2015-12-03 17:22:55 +00:00
Conrad Meyer
64448a2a50 if_ntb: Log error *before* zeroing relevant variables
Sponsored by:	EMC / Isilon Storage Division
2015-12-03 17:22:45 +00:00
Conrad Meyer
c4492f4758 NTB: WC/WB isn't enough; set MMR region as UC
And expose vm_memattr_t of current mapping to consumers (as well as the
ability to change it to one of UC, WB, WC).

After short discussion with:	jhb (but no review)
Sponsored by:	EMC / Isilon Storage Division
2015-11-25 01:59:08 +00:00
Conrad Meyer
e7f64f2e33 ntb: Add MW tunable for MMR Xeon errata workaround
Adds a new tunable, ntb.hw.b2b_mw_idx, which specifies the offset (from the
total number of memory windows) to use for register access on hardware with
the SDOORBELL_LOCKUP errata.  The default is -1, i.e., the last memory
window.

We map BARs before the b2b_mw_idx is selected, so map them all as memory
windows initially.  The register memory window should not be write-combined,
so we explicitly disable WC on the selected MW later.

This introduces a layer of abstraction between consumer memory window
indices, which exclude any exclusive errata-workaround BARs, and internal
memory window indices, which include such BARs.  An internal routine,
ntb_user_mw_to_idx(), converts the former to the latter.  Public APIs have
been updated to use this instead of assuming the exclusive workaround BAR is
the last available MW.

Sponsored by:	EMC / Isilon Storage Division
2015-11-24 18:51:17 +00:00
Conrad Meyer
70bca634d0 if_ntb: Initialize if_mtu to the correct MTU
Lower the payload data (IP) portion of the MTU from 0x10000 to
IP_MAXPACKET (0xFFFF) to avoid panicing the IP stack.

Sponsored by:	EMC / Isilon Storage Division
2015-11-19 19:53:19 +00:00
Conrad Meyer
f51a1fe048 if_ntb: Add Xeon link watchdog register writes
This feature is disabled by default.  To enable it, tune
hw.if_ntb.enable_xeon_watchdog to non-zero.

If enabled, writes an unused NTB register every second to demonstrate to
a hardware watchdog that the NTB device is still alive.  Most machines
with NTB will not need this -- you know who you are.

Sponsored by:	EMC / Isilon Storage Division
2015-11-19 19:53:09 +00:00
Conrad Meyer
e90df06388 if_ntb: Reuse receive buffers correctly
Discard the unused rx_free_q.  Instead, reuse inputed packets by putting
them back on the *pend* queue after reinitialization.

If tx or rx handlers are unavailable, free mbufs rather than leaking
them.

With this change, if_ntb can receive more than 100
(NTB_QP_DEF_NUM_ENTRIES) packets.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:49 +00:00
Conrad Meyer
c95b032ec6 NTB: Expose 32-bit BAR limits to consumers
32-bit BARs can only address memory mapped in the low 32 bits of
physical RAM.  Expose this as a 'plimit' out parameter from
ntb_mw_get_range().

Fix if_ntb to allocate memory within this limit.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:40 +00:00
Conrad Meyer
4e1eea459a NTB: Mask off the low 12 bits of address/range registers
Sometimes they'll read spurious values (observed: 0xc on Broadwell-DE),
failing link negotiation.

Discussed with:	Dave Jiang, Allen Hubbe
Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:31 +00:00
Conrad Meyer
57ba792a67 ntb_hw: Add programmatic interface to enable/disable WC
Enable users to enable/disable WC on memory windows programmatically.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:21 +00:00
Conrad Meyer
3062870897 ntb_hw: Add tunable to disable write-combining
The tunable 'hw.ntb.enable_writecombine' may be set to zero to
administratively disable write combining the mapped NTB region.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:13 +00:00
Conrad Meyer
552573e11c NTB: Fix 32-bit BAR size validation
Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:20:04 +00:00
Conrad Meyer
30994d9192 if_ntb: Diff reduce with Linux
Use bus_space_write instead of (non-volatile) C pointer writes via an
iowrite32() shim in the same places as the Dual BSD/GPL Linux driver.

Update some types to fixed 32-bit sizes.

Sponsored by:	EMC / Isilon Storage Division
2015-11-18 22:19:55 +00:00
Conrad Meyer
efd5acf04b if_ntb: Clear the right QP in the free bitmap
Now it can ping back and forth.

Sponsored by:	EMC / Isilon Storage Division
2015-11-14 01:23:13 +00:00
Conrad Meyer
dc8579df1c NTB: MFV 8b782fab: unify translation addresses
There is no need for the upstream and downstream addresses to be
different for the NTB configs.  Go to using a single set of address. It
is still possible to configure them differently using module parameter
override however (CEM: tunable).

Authored by:	Dave Jiang <dave.jiang@intel.com>
Reviewed by:	Allen Hubbe <Allen.Hubbe@emc.com>
Reviewed by:	Jon Mason <jdmason@kudzu.us>
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-11-12 19:07:03 +00:00
Conrad Meyer
28a0666622 if_ntb: MFV c92ba3c5: invalid buf pointer in multi-MW setups
Order of operations issue with the QP Num and MW count, which would
result in the receive buffer pointer being invalid if there are more
than 1 MW.  Corrected with parenthesis to enforce the proper order of
operations.

Reported by:	John I. Kading <John.Kading@gd-ms.com>
Reported by:	Conrad Meyer <cem@FreeBSD.org>
Authored by:	Jon Mason <jdmason@kudzu.us>
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:56:21 +00:00
Conrad Meyer
13e751cf1a NTB: Add more HW registers to device sysctl tree
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:56:11 +00:00
Conrad Meyer
fe92caaf23 ntb: volatile some members set by interrupt routines
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:56:02 +00:00
Conrad Meyer
bce32f9268 ntb_hw: Similarly, add a debug-leveled macro for ntb_hw
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:53 +00:00
Conrad Meyer
98bdb1ce51 if_ntb: Add module-specific log level
Rather than relying on the quite accurately named 'bootverbose'.

Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:44 +00:00
Conrad Meyer
b22ecf1ff8 if_ntb: Transport link cleanup needs to be on a taskqueue
Because it can sleep drainking link work callout(s).  Linux (dual
BSD/GPL driver) does something very similar.

At the same time, switch the NTB CTX lock to a non-spin mutex, because
the taskqueue_swi lock can't be taken after a spin mutex.

Suggested by:	Witness
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:34 +00:00
Conrad Meyer
9cf310367f NTB: Diff reduce with Linux
No functional change.

Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:25 +00:00
Conrad Meyer
e4a818a922 ntb_hw: Correctly detect DSD/USD
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:16 +00:00
Conrad Meyer
2219047bc1 ntb_hw: In INTx fallback, correct db shift from 15 to 16
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:55:07 +00:00
Conrad Meyer
6fde27ecee ntb: Use caddr_t to simplify pointer arithmetic
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:54:58 +00:00
Conrad Meyer
1a60b0c88e NTB: Skip db_valid validation writing DB link bit
In ntb_poll_link, we are intentionally writing the link bit, which is
absent from db_valid_mask.  Don't panic on a kassert when we do so.

The Linux version of this (dual BSD/GPL) driver has the db_valid_mask
assertions in callers of db_iowrite() rather than db_iowrite() itself;
it skips the assertions in the equivalent of ntb_poll_link().  Rather
than duplicating the assertions in every caller, add a db_iowrite_raw()
that doesn't check and use it from ntb_poll_link().

Suggested by:	kassert_panic
Sponsored by:	EMC / Isilon Storage Division
2015-11-11 18:54:49 +00:00
Conrad Meyer
217b098a1e ntb: Revert r290130 now that r290156 has landed
Nagged by:	vangyzen
Sponsored by:	EMC / Isilon Storage Division
2015-10-29 19:35:01 +00:00
Conrad Meyer
79c1a0199f ntb: Do not attempt to set write-combining on MWs
AMD64 pmap assumes ranges will be in the DMAP, which isn't necessarily
true for NTB memory windows (especially 64-bit BARs).

Suggested by:	pmap_change_attr_locked -> kassert_panic
Sponsored by:	EMC / Isilon Storage Division
2015-10-29 04:16:28 +00:00
Conrad Meyer
f09b75d4b5 if_ntb: Use ffsll() now that it is broadly available
Sponsored by:	EMC / Isilon Storage Division
2015-10-22 23:03:24 +00:00
Conrad Meyer
0a7663c2c0 NTB: Add device introspection sysctl hierarchy
This should export all of the same information as the Linux ntb_hw_intel
debugfs info file, but with a bit more structure, in the sysctl tree
rooted at 'dev.ntb_hw.<N>.debug_info'.

Raw registers are marked as OPAQUE because reading them on some hardware
revisions may cause a hard lockup (NTB errata).  They can be read with
'sysctl -x dev.ntb_hw.<N>.debug_info.registers'.  On Xeon platforms,
some additional registers are available under 'registers.xeon_stats' and
'registers.xeon_hw_err'.  They are exported as big-endian values so that
the 'sysctl -x' output is legible.

Shrink the feature mask to 32 bits so we can use the %b formatter in
'debug_info.features'.

Sponsored by:	EMC / Isilon Storage Division
2015-10-22 23:03:15 +00:00
Conrad Meyer
a69ff5b1ee NTB: Revert r289645
Per Benno, this is a Linuxism we do not need in FreeBSD.

Suggested by:	benno
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:32:16 +00:00
Conrad Meyer
d9503a7fcd if_ntb: Diff-reduce with Linux; add queue index type
Add ntb_q_idx_t so it is more clear which struct members are of the same
type (some bogus uint64_ts snuck in that should have been unsigned int).

Add tx_err_no_buf and s/ENOMEM/EBUSY/ in tx_enqueue to match Linux.

Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:21:01 +00:00
Conrad Meyer
2d6501b281 NTB: MFV 8c9edf63: Fix zero size or integer overflow in ntb_set_mw
A plain 32 bit integer will overflow for values over 4GiB.

Change the plain integer size to the appropriate size type in
ntb_set_mw.  Change the type of the size parameter and two local
variables used for size.

Even if there is no overflow, a size of zero is invalid here.

Authored by:	Allen Hubbe
Reported by:	Juyoung Jung
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:52 +00:00
Conrad Meyer
4994fe1295 NTB: MFV da2e5ae5: Fix ntb_transport out-of-order RX update
It was possible for a synchronous update of the RX index in the error
case to get ahead of the asynchronous RX index update in the normal
case.  Change the RX processing to preserve an RX completion order.

There were two error cases.  First, if a buffer is not present to
receive data, there would be no queue entry to preserve the RX
completion order.  Instead of dropping the RX frame, leave the RX frame
in the ring.  Schedule RX processing when RX entries are enqueued, in
case there are RX frames waiting in the ring to be received.

Second, if a buffer is too small to receive data, drop the frame in the
ring, mark the RX entry as done, and indicate the error in the RX entry
length.  Check for a negative length in the receive callback in
ntb_netdev, and count occurrences as rx_length_errors.

Authored by:	Allen Hubbe
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:42 +00:00
Conrad Meyer
8a26cf17c6 if_ntb: Correct over-long lines, use qmin()
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:33 +00:00
Conrad Meyer
bbea13c586 if_ntb: Use if_printf instead of device_printf
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:24 +00:00
Conrad Meyer
eccd1f0a14 NTB: MFV 2f887b9a: Rename Intel code names to platform names
Mechanically replace "SOC" with "ATOM" to match Linux.  No functional
change.  Original Linux commit log follows:

Instead of using the platform code names, use the correct platform names
to identify the respective Intel NTB hardware.

Authored by:	Dave Jiang
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:15 +00:00
Conrad Meyer
3a8a0a9dfa NTB: Don't abort if setting a MW write-combine fails
Also log BAR mapping results more verbosely.

Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:20:06 +00:00
Conrad Meyer
217453caed NTB: Fix typo in bar5 tunables
Typo introduced in r289614.

Pointy-hat:	cem
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:19:57 +00:00
Conrad Meyer
43a191ed78 NTB: MFV 7eb38781: Print driver name in module init
Prints driver name to indicate what is being loaded.

Authored by:	Dave Jiang
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 19:19:48 +00:00
Conrad Meyer
5165422dff NTB: Clean up safely if attach fails early
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:52 +00:00
Conrad Meyer
98580c064c NTB: MFV 9891417d: Increase transport MTU to 64k from 16k
Benchmarking showed a significant performance increase with the MTU size
to 64k instead of 16k.  Change the driver default to 64k.

Authored by:	Dave Jiang
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:43 +00:00
Conrad Meyer
6a88fb3ef8 if_ntb: Fix typo in qp_link_work to match Linux
Throw away the result of the peer SPAD read.  The peer will write our
local SPAD and we need to keep the locally read SPAD value to check if
the remote side is up.

Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:34 +00:00
Conrad Meyer
d5bd08b091 NTB: MFV 42fefc86: Add parameters for Intel SNB B2B addresses
Add module parameters for the addresses to be used in B2B topology.

Authored by:	Allen Hubbe
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:25 +00:00
Conrad Meyer
25ff5df7c4 if_ntb: MFV 2849b5d7: Reset transport QP link stats on down
Reset the link stats when the link goes down.  In particular, the TX and
RX index and count must be reset, or else the TX side will be sending
packets to the RX side where the RX side is not expecting them.  Reset
all the stats, to be consistent.

Authored by:	Allen Hubbe
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:16 +00:00
Conrad Meyer
0326e4d6b5 NTB: Change Atom (BWD/SoC) pci_id name to match others
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:54:08 +00:00
Conrad Meyer
fb6531b056 NTB: MFV 5ae0beb6: Enable link for Intel root port mode in probe
We skip actually bringing up Rootport/Transparent configurations, so
most of this doesn't apply.  Original Linux commit log:

Link training should be enabled in the driver probe for root port mode.
We should not have to wait for transport to be loaded for this to
happen.  Otherwise the ntb device will not show up on the transparent
bridge side of the link.

Authored by:	Dave Jiang
Obtained from:	Linux (Dual BSD/GPL driver)
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:46:14 +00:00
Conrad Meyer
6849146f9d NTB: enum-ify some HW constants
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:46:05 +00:00
Conrad Meyer
2d53c202a1 NTB: Pull copy of soc_link_is_err out of recover_soc_link
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:45:57 +00:00
Conrad Meyer
e638b59556 NTB: Drop some dead softc members
Sponsored by:	EMC / Isilon Storage Division
2015-10-20 01:45:48 +00:00