* New error_flags that can be used from the error ithread and elsewhere
without a synch_op.
* Stop the adapter immediately in t4_fatal_err but defer most of the
rest of the handling to a task. The task is allowed to sleep, unlike
the ithread. Remove async_event_task as it is no longer needed.
* Dump the devlog, CIMLA, and PCIE_FW exactly once on any fatal error
involving the firmware or the CIM block. While here, dump some
additional info (see dump_cim_regs) for these errors.
* If both reset_on_fatal_err and panic_on_fatal_err are set then attempt
a reset first and do not panic the system if it is successful.
MFC after: 1 week
Sponsored by: Chelsio Communications
cxgbe_refresh_stats takes into account VI_SKIP_STATS but not
VI_INIT_DONE when deciding whether to read the hardware stats. But
before this change VI_SKIP_STATS was set only for VIs with VI_INIT_DONE.
That meant that cxgbe_refresh_stats always accessed the hardware for
uninitialized VIs, and this is a problem if the adapter is suspended or
in the middle of a reset.
Fix this by setting VI_SKIP_STATS on all VIs during suspend. While
here, ignore VI_INIT_DONE in vi_refresh_stats too to be consistent with
cxgbe_refresh_stats.
MFC after: 1 week
Sponsored by: Chelsio Communications
Having a single pool of worker threads adds extra complexity and
overhead. The software backend also uses per-connection kthreads.
Sponsored by: Chelsio Communications
Previously the driver was called to send PDUs to the NIC synchronously
from the icl_conn_pdu_queue_cb callback. However, this performed a
fair bit of work while holding the icl connection lock. Instead,
change the callback to add sent PDUs to a STAILQ and defer dispatching
of PDUs to the NIC to a helper thread similar to the scheme used in
the TCP iSCSI backend.
- Replace rx_flags int and the sole RXF_ACTIVE flag with a simple
rx_active bool.
- Add a pool of transmit worker threads for cxgbei.
- Fix worker thread exit to depend on the wakeup in kthread_exit()
to fix a race with module unload.
Reported by: mav
Sponsored by: Chelsio Communications
sbcut() returns mbufs in reverse order so is not suitable for reading
data from the socket buffer. Instead, check for already-received data
in the receive worker thread before passing offload PDUs up to the
iSCSI layer. This uses soreceive() to read data from the socket and
is also to use M_WAITOK since it now runs from a worker thread instead
of an interrupt thread.
Also, fix decoding of the data segment length for pre-offload PDUs.
Reported by: Jithesh Arakkan @ Chelsio
Fixes: a8c4147edc cxgbei: Parse all PDUs received prior to enabling offload mode.
Sponsored by: Chelsio Communications
Previously this would only handle a single PDU that did not contain
any data. This should now handle an arbitrary number of PDUs.
While here check for these PDUs in the T6-specific CPL_RX_ISCSI_CMP
handler in addition to CPL_RX_ISCSI_DDP.
Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications
Otherwise we end up copying one uninitialized byte into the socket
buffer.
Reported by: KMSAN
Reviewed by: jhb
MFC after: 1 week
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33953
The default sysctl context setup by newbus for a device is eventually
freed by device_sysctl_fini, which runs after the device driver's detach
routine. sysctl nodes associated with this context must not use any
resources (like driver locks, hardware access, counters, etc.) that are
released by driver detach.
There are a lot of sysctl nodes like this in cxgbe(4) and the fix is to
hang them off a context that is explicitly freed by the driver before it
releases any resource that might be used by a sysctl.
This fixes panics when running "sysctl dev.t6nex dev.cc" in a tight loop
and loading/unloading the driver in parallel.
Reported by: Suhas Lokesha
MFC after: 1 week
Sponsored by: Chelsio Communications
This ensures that the driver reports an error instead of failing
silently when an invalid media is requested.
Reported by: Suhas Lokesha @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications
This eliminates error messages like this from the driver when running at
50Gbps with 100G cables:
[3726] cc0: l1cfg failed: 71
[4407] cc0: l1cfg failed: 71
Note that link comes up anyway with or without this change.
Reported by: Suhas Lokesha @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications
This matches icl_conn_task_setup() which passes the PDU and avoids the
need for a layering violation in cxgbei to fetch the request PDU from
the ctl_io.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33746
Previously the driver duplicated code from cryptosoft.c to handle
certain edge case AES-CCM and AES-GCM requests. However, this
approach has a few downsides:
1) It only uses "plain" software and not accelerated software since it
uses enc_xform directly.
2) It performs the operation synchronously even though the caller
believes it is invoking an async driver. This was fine for the
original use case of requests with only AAD and no payload that
execute quickly, but is a bit more disingenuous for large requests
which fall back due to exceeding the size of a firmware work
request (e.g. due to large scatter/gather lists).
3) It has required several updates since ccr(4) was added to the tree.
Instead, allocate a software session for AES-CCM and AES-GCM sessions
and dispatch a cloned request asynchronusly to the software session.
Reviewed by: markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D33608
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
CHANGES
++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Version : 1.26.6.0
Date : 01/03/2022
================================================================================
Fixes
-----
BASE:
- Fixed one module eeprom read failure.
- Fixed an issue with speed selection when 40G and 25G are advertised and
supported.
- Fixed a random traffic hang when T5 receives invalid ets BW in dcbx
messages from a switch.
- Fixed very long link up time with few switches.
================================================================================
Obtained from: Chelsio Communications
MFC after: 2 weeks
Sponsored by: Chelsio Communications
This fixes a driver panic during stats collection when a port's id does
not match its tx channel. The bug affected only the T580 card running
with a non-default VPD.
Reported by: Suhas Lokesha @ Chelsio
MFC after: 1 week
Sponsored by: Chelsio Communications
sge->ctrlq is not always allocated during attach (eg. if firmware
initialization fails) and detach should be able to deal with this.
MFC after: 1 week
Sponsored by: Chelsio Communications
The advanced TCP stacks (bbr, rack) may decide to drop a TCP connection
when they do output on it. The default stack never does this, thus
existing framework expects tcp_output() always to return locked and
valid tcpcb.
Provide KPI extension to satisfy demands of advanced stacks. If the
output method returns negative error code, it means that caller must
call tcp_drop().
In tcp_var() provide three inline methods to call tcp_output():
- tcp_output() is a drop-in replacement for the default stack, so that
default stack can continue using it internally without modifications.
For advanced stacks it would perform tcp_drop() and unlock and report
that with negative error code.
- tcp_output_unlock() handles the negative code and always converts
it to positive and always unlocks.
- tcp_output_nodrop() just calls the method and leaves the responsibility
to drop on the caller.
Sweep over the advanced stacks and use new KPI instead of using HPTS
delayed drop queue for that.
Reviewed by: rrs, tuexen
Differential revision: https://reviews.freebsd.org/D33370
When the initiator is reconnecting to the target, the connection may
temporarily be marked disconnected or not have an associated socket.
New I/O requests received by the initiator in this state should not
fail with ECONNRESET as that results in an I/O error back to userland.
Instead, they need to still succeed so that CAM can queue the requests
and send them once the connection is re-established.
Setting up DDP for zero-copy receive requires a socket, so just punt
on using DDP for these transfers.
Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications
ccr(4) can handle requests for AES-CTR (a stream cipher), not just
block ciphers, so make the function and structure names more generic.
Sponsored by: Chelsio Communications
This centralizes the check for valid nonce lengths for AES-GCM.
While here, remove some duplicate checks for valid AES-GCM tag lengths
from ccp(4) and ccr(4).
Reviewed by: markj
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D33194
(Rest is from the README that came with the firmware)
Version : 1.26.4.0
Date : 12/02/2021
Fixes
-----
BASE:
- Fixed error on setting 25G speed on 100G copper with multiple FEC set in
firmware commands.
- Handle link of unknown optics modules by enabling module tx unconditionally.
- Fixed link not coming up for 25G CRS phys. Firmware incorrectly tried to
bring up the link in RS-FEC but as per IEEE spec, it must be BASER FEC.
- Fixed an issue where firmware doesn't automatically retry next FEC if driver
asks to bring up the link using RS-FEC and link doesn't come up.
Obtained from: Chelsio Communications
MFC after: 1 month
Sponsored by: Chelsio Communications
Modify the GPIO pins only on the Base-T cards and even there drive all
of them low instead of putting them in hi-z state. For the rest (this
is the common case), directly power off the PLLs of the high speed
serdes. This is the simplest method that does not involve or conflict
with the firmware but still works with all T4-T6 cards regardless of
what's plugged into the port.
This fixes a problem where the peer wouldn't always see a link down if
it is connected to the device using a -CR4 copper cable.
MFC after: 3 weeks
Sponsored by: Chelsio Communications
Recent firmwares have support for autonomous FEC selection and a "force"
knob to let the driver control this behavior (or not) in a fine grained
manner. This change adds a driver knob so that all the different ways of
configuring the link FEC can be exercised. Note that this controls the
internal driver/firmware interaction for link configuration and is not
meant for general use.
MFC after: 1 week
Sponsored by: Chelsio Communications
Recent firmwares have more leeway in FEC selection and there is a need
to track the FECs requested by the driver separately from the FEC in use
on the link. The existing dev.<port>.<inst>.fec sysctl can read both but
its behavior depends on the link state and it is sometimes hard to find
out what was requested when the link is up.
Split the fec sysctl into two (requested_fec and link_fec) to get access
to both pieces of information regardless of the link state.
MFC after: 1 week
Sponsored by: Chelsio Communications
Previously, sorele() always required the socket lock and dropped the
lock if the released reference was not the last reference. Many
callers locked the socket lock just before calling sorele() resulting
in a wasted lock/unlock when not dropping the last reference.
Move the previous implementation of sorele() into a new
sorele_locked() function and use it instead of sorele() for various
places in uipc_socket.c that called sorele() while already holding the
socket lock.
The sorele() macro now uses refcount_release_if_not_last() try to drop
the socket reference without locking the socket. If that shortcut
fails, it locks the socket and calls sorele_locked().
Reviewed by: kib, markj
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32741
These macros are not backend-specific but reference a
backend-independent field in struct icl_conn.
Reviewed by: mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32858
Previously the body of ktls_tick was a nop when NIC TLS was disabled,
but the callout was still scheduled consuming power on otherwise-idle
systems with Chelsio T6 adapters. Now the callout only runs while NIC
TLS is enabled on at least one interface of an adapter.
Reported by: mav
Reviewed by: np, mav
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32491
The tag length is included as one of the values in the flags byte of
block 0 passed to CBC_MAC, so merely copying the first N bytes is
insufficient.
To avoid adding more sideband data to the CBC MAC software context,
pull the generation of block 0, the AAD length, and AAD padding out of
cbc_mac.c and into cryptosoft.c. This matches how GCM/GMAC are
handled where the length block is constructed in cryptosoft.c and
passed as an input to the Update callback. As a result, the CBC MAC
Update() routine is now much simpler and simply performs the
XOR-and-encrypt step on each input block.
While here, avoid a copy to the staging block in the Update routine
when one or more full blocks are passed as input to the Update
callback.
Reviewed by: sef
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32120
Rather than copying crp_iv to a local array on the stack that is then
passed to xform reinit routines, pass crp_iv directly and remove the
local copy.
Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32106
Add a 'len' argument to the reinit hook in 'struct enc_xform' to
permit support for AEAD ciphers such as AES-CCM and Chacha20-Poly1305
which support different nonce lengths.
Reviewed by: markj
Sponsored by: Chelsio Communications, The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32105
While here, use crypto_read_iv() in a few more places in ccr(4) that I
missed previously.
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D32104
The firmwares and the following changelog are from the "Chelsio Unified
Wire v3.15.0.0 for Linux."
Version : 1.26.2.0
Date : 09/24/2021
====================
FIXES
-----
BASE:
- Added support for SFP+ RJ45 (0x1C).
- Fixing backward compatibility issue with older drivers when multiple
speeds are passed to firmware.
OFLD:
- Do not touch tp_plen_max if driver is supplying tp_plen_max. This
fixes a connection reset issue in iscsi.
ENHANCEMENTS
------------
BASE:
- Firmware header modified to add firmware binary signature.
MFC after: 1 month
Sponsored by: Chelsio Communications
The KMSAN runtime needs to have its shadow maps updated when devices
update host memory, otherwise it assumes that device-populated memory is
uninitialized. For most drivers this is handled transparently by
busdma, but cxgbe doesn't make use of dma maps for receive buffers and
so requires special treatment.
Reported by: mjg
Tested by: mjg
Reviewed by: np
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D32102
The -NOIP builds fail because cxgbe_tls_tag_free() has no prototype (if
neither INET nor INET6 are defined). The function isn't actually used
in that case, so we can just remove the stub implementation.
Sponsored by: Rubicon Communications, LLC ("Netgate")
For TCP DDP, handle_ddp_close() needs to see the pre-FIN rcv_nxt to
determine how much data was placed in the local buffer before the FIN
was received. The changes in d59f1c49e2 broke this by updating
rcv_nxt before calling handle_ddp_close().
Fixes: d59f1c49e2 cxgbe tom: Permit rcv_nxt mismatches on FIN for iSCSI connections on T6.
Sponsored by: Chelsio Communications
This is similar to the fixes in 141fe2dcee. One difference is that
TOE sockets do not change states (listen vs non-listen) once created,
so no lock is needed for SOLISTENING().
Sponsored by: Chelsio Communications
Move the type and function pointers for operations on existing send
tags (modify, query, next, free) out of 'struct ifnet' and into a new
'struct if_snd_tag_sw'. A pointer to this structure is added to the
generic part of send tags and is initialized by m_snd_tag_init()
(which now accepts a switch structure as a new argument in place of
the type).
Previously, device driver ifnet methods switched on the type to call
type-specific functions. Now, those type-specific functions are saved
in the switch structure and invoked directly. In addition, this more
gracefully permits multiple implementations of the same tag within a
driver. In particular, NIC TLS for future Chelsio adapters will use a
different implementation than the existing NIC TLS support for T6
adapters.
Reviewed by: gallatin, hselasky, kib (older version)
Sponsored by: Chelsio Communications
Differential Revision: https://reviews.freebsd.org/D31572
Reject attempts to convert a connection using a different ULP
mode: (e.g. DDP or TLS) to ISCSI.
Reported by: Jithesh Arakkan @ Chelsio
Sponsored by: Chelsio Communications
