Commit Graph

1288 Commits

Author SHA1 Message Date
John Baldwin
0677f5ccbb cxgbe ddp: Trim stale function prototype
Sponsored by:	Chelsio Communications
2023-08-23 14:30:16 -07:00
Marius Strobl
dc485b968d tcp_info: Add and export more FreeBSD-specific fields
This change adds struct tcp_info fields corresponding to the following
struct tcpcb ones:
- snd_una
- snd_max
- rcv_numsacks
- rcv_adv
- dupacks

Note that while both tcp_fill_info() and fill_tcp_info_from_tcb() are
extended accordingly, no counterpart of rcv_numsacks is available in
the cxgbe(4) TOE PCB, though.

Sponsored by:	NetApp, Inc. (originally)
2023-08-22 20:34:01 +02:00
Marius Strobl
8c6104c48e tcp_fill_info(): Change lock assertion on INPCB to locked only
This function actually only ever reads from the TCP PCB. Consequently,
also make the pointer to its TCP PCB parameter const.

Sponsored by:	NetApp, Inc. (originally)
2023-08-22 20:33:49 +02:00
Warner Losh
78d146160d sys: Remove $FreeBSD$: one-line bare tag
Remove /^\s*\$FreeBSD\$$\n/
2023-08-16 11:55:17 -06:00
Warner Losh
031beb4e23 sys: Remove $FreeBSD$: one-line sh pattern
Remove /^\s*#[#!]?\s*\$FreeBSD\$.*$\n/
2023-08-16 11:54:58 -06:00
Warner Losh
685dc743dc sys: Remove $FreeBSD$: one-line .c pattern
Remove /^[\s*]*__FBSDID\("\$FreeBSD\$"\);?\s*\n/
2023-08-16 11:54:36 -06:00
Warner Losh
95ee2897e9 sys: Remove $FreeBSD$: two-line .h pattern
Remove /^\s*\*\n \*\s+\$FreeBSD\$$\n/
2023-08-16 11:54:11 -06:00
Navdeep Parhar
8a2e880ab0 cxgbe(4): Update firmwares to 1.27.4.0
This is the list of changes since last release, taken from the release
notes of Chelsio Unified Wire 3.18.0.1.

Version : 1.27.4.0
Date    : 07/05/2023
=======================================
Fixes
-----

BASE:
- Handle 40G to 100G cable change.
- Avoid unnecessary i2c read.
=======================================

Obtained from:	Chelsio Communications
Sponsored by:	Chelsio Communications
MFC after:	1 week
2023-08-16 08:43:43 -07:00
John Baldwin
884eaacd24 ddb: Rework macros to make it easier to add new command tables.
- Add new DB_DEFINE_TABLE and DB_DECLARE_TABLE macros to define new
  command tables.  DB_DECLARE_TABLE is intended for use in headers
  similar to MALLOC_DECLARE and SYSCTL_DECL.

  DB_DEFINE_TABLE takes three arguments, the name of the parent table,
  the command name, and the name of the table itself, e.g.
  DB_DEFINE_TABLE(show, foo, show_foo) defines a new "show foo" table.

- DB_TABLE_COMMAND, DB_TABLE_COMMAND_FLAGS, DB_TABLE_ALIAS, and
  DB_ALIAS_FLAGS allow new commands and aliases to be defined.  These
  are similar to the existing DB_COMMAND, etc. except that they take
  an initial argument giving the name of the parent table, e.g.:

  DB_TABLE_COMMAND(show_foo, bar, db_show_foo_bar)

  defines a new "show foo bar" command.

This provides a cleaner interface than the ad-hoc use of internal
macros like _DB_SET that was required previously (e.g. in cxgbe(4)).

This retires DB_FUNC macro as well as the internal _DB_FUNC macro.

Reviewed by:	melifaro, kib, markj
Differential Revision:	https://reviews.freebsd.org/D40819
2023-07-05 16:02:01 -07:00
John Baldwin
c98146ae22 cxgbe: Move ethofld_transmit under #ifdef INET || INET.
This fixes a -Wunused-function warning when building LINT-NOIP.

Reported by:	rpokala
2023-05-20 09:27:40 -07:00
Warner Losh
4d846d260e spdx: The BSD-2-Clause-FreeBSD identifier is obsolete, drop -FreeBSD
The SPDX folks have obsoleted the BSD-2-Clause-FreeBSD identifier. Catch
up to that fact and revert to their recommended match of BSD-2-Clause.

Discussed with:		pfg
MFC After:		3 days
Sponsored by:		Netflix
2023-05-12 10:44:03 -06:00
John Baldwin
4961faaacc pmap_{un}map_io_transient: Use bool instead of boolean_t.
Reviewed by:	imp, kib
Differential Revision:	https://reviews.freebsd.org/D39920
2023-05-04 12:29:48 -07:00
Navdeep Parhar
ca5391bd85 cxgbe(4): Update firmwares to version 1.27.3.0
These are the changes since the last update (copy-pasted from the
release notes for Chelsio Unified Wire v3.18.0.0):

====================
Version : 1.27.3.0
Date    : 04/07/2023

Fixes
-----
BASE:
- Fixed a hang if module eeprom reads gives invalid data.
- KR backlplane no-fec link problem fixed.
OFLD:
- iscsi ddp errors fixed.
- iwarp connection abort in rare cases causing NIC traffic hang fixed.

ENHANCEMENTS
------------
BASE:
- Cisco GLC-TE 1G modules support added.

====================
Version : 1.27.1.0
Date    : 12/02/2022

Fixes
-----
BASE:
- memwrite dsgl cannot be used for T5.
OFLD:
- Enabled FCoE in SO adapters.
- TOE-TLS crash fixed.
- iscsi hang fixed.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
2023-04-20 20:57:38 -07:00
Navdeep Parhar
2791335104 cxgbe(4): Dump the firmware log before falling back to a minimal config.
It might have errors that explain why the attempted configuration
failed.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2023-04-20 12:56:24 -07:00
Navdeep Parhar
7adf138ba9 cxgbe/iw_cxgbe: debug routines to dump STAG (steering tag) entries.
t4_dump_stag to dump hw state for a known STAG.

t4_dump_all_stag to dump hw state for all valid STAGs.  This routine
walks the entire STAG region looking for valid entries and this can take
a while for some configurations.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2023-04-19 09:38:14 -07:00
Navdeep Parhar
9f354cd3d0 cxgbe(4): Allow tracing filters on loopback ports.
Each physical port has an associated loopback tx channel and anything
transmitted over that channel by the driver is looped back internally by
the hardware as if received on that physical port.  This change allows
tracing filters to be installed in this loopback path.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2023-04-01 17:50:46 -07:00
Navdeep Parhar
531ef35241 cxgbe/iw_cxgbe: Always set a vnet around calls to IN_LOOPBACK.
This is catch up with efe58855f3.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2023-04-01 16:19:10 -07:00
Navdeep Parhar
21b778fbeb cxgbe(4): Remove dead code.
Fixes:	e7e0844422 cxgbe(4): Replace T4_PKT_TIMESTAMP with something slightly less hackish.
MFC after:	1 week
Sponsored by:	Chelsio Communications
2023-03-30 14:13:07 -07:00
Justin Hibbits
954712e8f6 Mechanically convert cxgb(4) and cxgbe(4) to IfAPI
Reviewed by:	np
Sponsored by:	Juniper Networks, Inc.
Differential Revision: https://reviews.freebsd.org/D38597
2023-03-07 15:31:48 -05:00
John Baldwin
ca457729cc cxgbe: Queue NIC TLS mbufs internally.
Set internal mbuf properties and queue NIC TLS mbuf chains to the
connection's TX queue directly.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D38579
2023-02-17 09:07:17 -08:00
John Baldwin
b919bf0596 cxgbe: Move helper functions for mbuf metadata to adapter.h.
Previously private to t4_sge.c, this allows other parts of the driver
(such as NIC TLS) to use these helpers directly.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D38578
2023-02-17 09:06:12 -08:00
John Baldwin
8afd23de92 cxgbe: Allow parse_pkt to internally queue a packet.
If parse_pkt returns EINPROGRESS, return from cxgbe_transmit
without queueing the packet in a txq.  Use this to move the call
to ethofld_transmit for packet pacing into parse_pkt.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D38577
2023-02-17 09:03:23 -08:00
John Baldwin
d2070e5fa9 cxgbe: Don't leak memory resource if t4iov attach fails.
Co-authored by:	np
Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D38580
2023-02-15 13:34:00 -08:00
Gleb Smirnoff
caf32b260a pfil: add pfil_mem_{in,out}() and retire pfil_run_hooks()
The 0b70e3e78b changed the original design of a single entry point
into pfil(9) chains providing separate functions for the filtering
points that always provide mbufs and know the direction of a flow.
The motivation was to reduce branching.  The logical continuation
would be to do the same for the filtering points that always provide
a memory pointer and retire the single entry point.

o Hooks now provide two functions: one for mbufs and optional for
  memory pointers.
o pfil_hook_args() has a new member and pfil_add_hook() has a
  requirement to zero out uninitialized data. Bump PFIL_VERSION.
o As it was before, a hook function for a memory pointer may realloc
  into an mbuf.  Such mbuf would be returned via a pointer that must
  be provided in argument.
o The only hook that supports memory pointers is ipfw:default-link.
  It is rewritten to provide two functions.
o All remaining uses of pfil_run_hooks() are converted to
  pfil_mem_in().
o Transparent union of pfil_packet_t and tricks to fix pointer
  alignment are retired. Internal pfil_realloc() reduces down to
  m_devget() and thus is retired, too.

Reviewed by:		mjg, ocochard
Differential revision:	https://reviews.freebsd.org/D37977
2023-02-14 10:02:49 -08:00
Jung-uk Kim
47131a3278 cxgbe: fix module loading
After c034143269, if_cxgbe.ko fails to load if crypto is not compiled
in kernel, e.g., MINIMAL.

link_elf_obj: symbol hmac_init_ipad undefined
linker_load_file: /boot/kernel/if_cxgbe.ko - unsupported file type
kldload: an error occurred while loading module if_cxgbe. Please check dmesg(8) for more details.

Reviewed by:		jhb
Differential Revision:	https://reviews.freebsd.org/D38482
2023-02-10 13:27:20 -05:00
John Baldwin
e396612511 cxgbe T6 KTLS: Use intotcpcb().
Fixes:		e68b379244 tcp: embed inpcb into tcpcb
Sponsored by:	Chelsio Communications
2023-02-06 17:07:53 -08:00
John Baldwin
70efe1a2fe ccr,ccp: Fix argument order to sglist_append_vmpages.
The offset comes before the byte count.

Reported by:	br
Reviewed by:	asomers, markj
MFC after:	1 week
Sponsored by:	DARPA
Differential Revision:	https://reviews.freebsd.org/D38375
2023-02-06 13:51:57 -08:00
Justin Hibbits
c255d1a401 IfAPI: Add if_llsoftc member accessors for TOEDEV
Summary:
Keep TOEDEV() macro for backwards compatibility, and add a SETTOEDEV()
macro to complement with the new accessors.

Sponsored by:	Juniper Networks, Inc.
Reviewed by:	glebius
Differential Revision: https://reviews.freebsd.org/D38199
2023-01-31 15:02:16 -05:00
Gordon Bergling
f273b45613 cxgbe(4): Fix a few common typos in source code comments
- s/Reqests/Requests/

MFC after:	3 days
2023-01-11 10:53:07 +01:00
John Baldwin
7063b9974f ccr: Retire ccr_softc member in struct adapter.
Prior to Conrad's changes to replace session integer IDs with a
pointer to the driver-specific state in commit 1b0909d51a, the
driver had to find the softc pointer from the adapter before it could
locate the ccr_session structure for a completed request.  Since
Conrad's changes, the ccr_session pointer can now be obtained directly
from the crp.  Add a backpoint from ccr_session back to ccr_softc and
use this in place of the ccr_softc member in cxgbe's struct adapter.

Sponsored by:	Chelsio Communications
2022-12-29 11:39:28 -08:00
Gleb Smirnoff
e68b379244 tcp: embed inpcb into tcpcb
For the TCP protocol inpcb storage specify allocation size that would
provide space to most of the data a TCP connection needs, embedding
into struct tcpcb several structures, that previously were allocated
separately.

The most import one is the inpcb itself.  With embedding we can provide
strong guarantee that with a valid TCP inpcb the tcpcb is always valid
and vice versa.  Also we reduce number of allocs/frees per connection.
The embedded inpcb is placed in the beginning of the struct tcpcb,
since in_pcballoc() requires that.  However, later we may want to move
it around for cache line efficiency, and this can be done with a little
effort.  The new intotcpcb() macro is ready for such move.

The congestion algorithm data, the TCP timers and osd(9) data are
also embedded into tcpcb, and temprorary struct tcpcb_mem goes away.
There was no extra allocation here, but we went through extra pointer
every time we accessed this data.

One interesting side effect is that now TCP data is allocated from
SMR-protected zone.  Potentially this allows the TCP stacks or other
TCP related modules to utilize that for their own synchronization.

Large part of the change was done with sed script:

s/tp->ccv->/tp->t_ccv./g
s/tp->ccv/\&tp->t_ccv/g
s/tp->cc_algo/tp->t_cc/g
s/tp->t_timers->tt_/tp->tt_/g
s/CCV\(ccv, osd\)/\&CCV(ccv, t_osd)/g

Dependency side effect is that code that needs to know struct tcpcb
should also know struct inpcb, that added several <netinet/in_pcb.h>.

Differential revision:	https://reviews.freebsd.org/D37127
2022-12-07 09:00:48 -08:00
Mateusz Guzik
09d7f260d4 cxgbe: stop open-coding kref access
Sponsored by:	Rubicon Communications, LLC ("Netgate")
2022-12-05 17:38:17 +00:00
John Baldwin
2ff447ee3b cxgbe: Enable TOE TLS RX when an RX key is provided via setsockopt().
Rather than requiring a socket to be created as a TLS socket from the
get go, switch a TOE socket from "plain" TOE to TLS mode when a
receive key is added to the socket.

The firmware is only able to switch a "plain" TOE connection to TLS
mode if the head of the pending socket data is the start of a TLS
record, so the connection is migrated to TLS mode as a multi-step
process.

When TOE TLS RX is enabled, the associated connection's receive side
is frozen via a flag in the TCB.  The state of the socket buffer is
then examined to determine if the pending data in the socket buffer
ends on a TLS record boundary.  If so, the connection is migrated to
TLS mode and unfrozen.  Otherwise, the connection is unfrozen
temporarily until more data arrives.  Once more data arrives, the
receive queue is frozen again and rechecked.  This continues until the
connection is paused at a record boundary.  Any records received
before TLS mode is enabled are decrypted as software records.

Note that this removes the 'rx_tls_ports' sysctl.  TOE TLS offload for
receive is now enabled automatically on existing TOE connections when
using a KTLS-aware SSL library just as it was previously enabled
automatically for TLS transmit.  This also enables TLS offload for TOE
connections which enable TLS after passing initial data in the clear
(e.g. STARTTLS with SMTP).

Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D37351
2022-11-15 12:08:51 -08:00
John Baldwin
21186bdb2d cxgbe: Various whitespace fixes.
Mostly trailing whitespace and spaces before tabs.

Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D37350
2022-11-15 12:03:57 -08:00
John Baldwin
e4bc19b2fa cxgbe tom: Fix jobtotid() compilation.
The previous commit lost an implicit struct socket * cast.  Use an
inline function instead as the macro is already rather long.

Fixes:		e1401f7579 cxgbe: use standard sototcpcb() accessor macro to get socket's tcpcb
Sponsored by:	Chelsio Communications
2022-11-08 11:25:58 -08:00
Gleb Smirnoff
9eb0e8326d tcp: provide macros to access inpcb and socket from a tcpcb
There should be no functional changes with this commit.

Reviewed by:		rscheff
Differential revision:	https://reviews.freebsd.org/D37123
2022-11-08 10:24:40 -08:00
Gleb Smirnoff
b2c558c898 cxgbe: include headers required to include t4_tom.h
Before the change we would get struct tcpcb forward declaration
only with help of pollution via in_pcb.h.
2022-10-19 15:15:53 -07:00
Gleb Smirnoff
e1401f7579 cxgbe: use standard sototcpcb() accessor macro to get socket's tcpcb
Reviewed by:		np
Differential revision:	https://reviews.freebsd.org/D37041
2022-10-19 15:15:32 -07:00
Gleb Smirnoff
53af690381 tcp: remove INP_TIMEWAIT flag
Mechanically cleanup INP_TIMEWAIT from the kernel sources.  After
0d7445193a, this commit shall not cause any functional changes.

Note: this flag was very often checked together with INP_DROPPED.
If we modify in_pcblookup*() not to return INP_DROPPED pcbs, we
will be able to remove most of this checks and turn them to
assertions.  Some of them can be turned into assertions right now,
but that should be carefully done on a case by case basis.

Differential revision:	https://reviews.freebsd.org/D36400
2022-10-06 19:24:37 -07:00
Navdeep Parhar
661faa1f77 cxgbe/cxgbei: Do not validate the hardware iSCSI tag mask.
This was added in 7cba15b16e in 2016 and firmwares at that time were
already setting up the iSCSI tag mask properly.  Since then it has also
become possible to split the iSCSI region between multiple PCIE PFs but
the driver's calculation takes only its own PF's allocation into account
and that means this code is incorrect and not just a harmless no-op.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2022-10-06 10:12:00 -07:00
Navdeep Parhar
8d2c13931b cxgbe/tom: Fix assertions in the code that maintains TCB history.
The tids used for TOE connections start from tid_base, not 0.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2022-09-28 20:01:14 -07:00
John Baldwin
cee4fc7cad cxgbe: Use secq(9) to manage the timestamp generations.
This is mostly cosmetic, but it also doesn't leave a gap of time where
no structures are valid.  Instead, we permit the ISR to continue to
use the previous structure if the write to update cal_current isn't
yet visible.

Reviewed by:	gallatin
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D36669
2022-09-26 14:58:41 -07:00
John Baldwin
2c74c9dac3 cxgbe: Compute timestamps via sbintime_t.
This uses fixed-point math already used elsewhere in the kernel for
sub-second time values.  To avoid overflows this does require updating
the calibration once a second rather than once every 30 seconds.  Note
that the cxgbe driver already queries multiple registers once a second
for the statistics timers.  This version also uses fewer instructions
with no branches (for the math portion) in the per-packet fast path.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D36663
2022-09-26 14:58:30 -07:00
Navdeep Parhar
02fac928ab cxgbe(4): Fix potential integer overflow in t4_tstmp_to_ns.
Coverity flagged this in its latest run but it is not a problem in
practice as the card's core clock would have to be > 4.2GHz for any
overflow to occur.

CID 1498303:  Integer handling issues  (OVERFLOW_BEFORE_WIDEN)
Potentially overflowing expression "sc->params.vpd.cclk * 1000U" with type "unsigned int" (32 bits, unsigned) is evaluated using 32-bit arithmetic, and then used in a context that expects an expression of type "uint64_t" (64 bits, unsigned).

Reported by:	Coverity Scan (CID 1498303)
Sponsored by:	Chelsio Communications
2022-09-22 09:47:11 -07:00
Konstantin Belousov
8d42019058 cxgbe: fix kernel build without TCP_OFFLOAD or RATELIMIT
Reviewed by:	np
Fixes:	e398922eaf
Sponsored by:	Nvidia networking
Differential revision:	https://reviews.freebsd.org/D36648
2022-09-21 19:19:28 +03:00
Mateusz Guzik
d0b235c715 cxgbe: fix the build after e398922eaf 2022-09-20 21:31:49 +00:00
Randall Stewart
e398922eaf Enable M_TSTMP in Chelsio cxgbe driver by creating a mechanism that can sync the time.
Chelsio has always been recording a timestamp in the mbuf (rcv_tstmp) but
not setting the M_TSTMP bit in the mbuf flags. This is because the timestamp
was just the free running 60bit clock. This change fixes that so that
we keep a synchronization by periodically (every 30 seconds after startup)
getting the timestamp and the current nanosecond time. We always keep
several sets around and the current one we always keep the current pair
and the previous pair of timestamps. This allows us to setup a ratio
between the two so we can correctly translate the time. Note that
we use special care to split the timestamp into seconds (per the clock tick)
and nanoseconds otherwise 64bit math would overflow.

Reviewed by: np
Sponsored by: Netflix Inc
Differential Revision: https://reviews.freebsd.org/D36315
2022-09-20 15:13:16 -04:00
Navdeep Parhar
998eb37aca cxgbe(4): Add knob to control congestion behavior for TOE queues.
hw.cxgbe.ofld_cong_drop works just like hw.cxgbe.cong_drop but for TOE
rx queues.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
2022-09-12 11:40:35 -07:00
Navdeep Parhar
df275ae509 cxgbe(4): Add another setting to the knob that controls congestion.
hw.cxgbe.cong_drop=2 will generate backpressure *and* drop frames for
queues that are congested.

MFC after:	2 weeks
Sponsored by:	Chelsio Communications
2022-09-12 11:40:29 -07:00
Navdeep Parhar
c387ff0045 cxgbe(4): Specify the ingress queue's type when creating it.
The firmware takes the type into account when setting up the PCIe
channel for the queue.

MFC after:	1 week
Sponsored by:	Chelsio Communications
2022-09-12 11:40:25 -07:00