Commit Graph

469 Commits

Author SHA1 Message Date
Navdeep Parhar
c149da1661 cxgbe/cxgbei: There is no need for multiple modules in the KLD.
Sponsored by:	Chelsio Communications
2016-08-26 01:28:31 +00:00
Navdeep Parhar
a95e0bfb6c cxgbe/cxgbei: Convert the driver-private PDU flags to enums and replace
pdu_ prefix with icp_ in struct icl_cxgbei_pdu.

Sponsored by:	Chelsio Communications
2016-08-25 23:06:12 +00:00
Navdeep Parhar
5d83392ac5 cxgbe/cxgbei: Read the chip's configuration to determine the actual
hardware send and receive PDU limits.  Report these limits to ICL and
take them into account when setting the socket's send and receive buffer
sizes.  The driver used a single hardcoded limit everywhere prior to
this change.

Sponsored by:	Chelsio Communications
2016-08-25 21:55:17 +00:00
Navdeep Parhar
97b84d344d Make the iSCSI parameter negotiation more flexible.
Decouple the send and receive limits on the amount of data in a single
iSCSI PDU.  MaxRecvDataSegmentLength is declarative, not negotiated, and
is direction-specific so there is no reason for both ends to limit
themselves to the same min(initiator, target) value in both directions.

Allow iSCSI drivers to report their send, receive, first burst, and max
burst limits explicitly instead of using hardcoded values or trying to
derive all of them from the receive limit (which was the only limit
reported by the drivers prior to this change).

Display the send and receive limits separately in the userspace iSCSI
utilities.

Reviewed by:	jpaetzel@ (earlier version), trasz@
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7279
2016-08-25 05:22:53 +00:00
John Baldwin
bd6ff0807e Reorder sysctls so that nodes shared with the VF driver are added first.
This permits a single early return for VF devices in the routines that
add sysctl nodes.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7512
2016-08-19 17:54:51 +00:00
John Baldwin
301abe0f3f Adjust t4_port_init() to work with VF devices.
Specifically, the FW_PORT_CMD may or may not work for a VF (the PF
driver can choose whether or not to permit access to this command),
so don't attempt to fetch port information on a VF if permission is
denied by the PF.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7511
2016-08-19 17:52:48 +00:00
John Baldwin
577422fc53 Add structures for VF-specific adapter parameters.
While here, mark which parameters are PF-specific and which are
VF-specific.

Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7508
2016-08-19 17:49:49 +00:00
John Baldwin
8b2246b349 Add support for register dumps on VF devices.
- Add handling of VF register sets to t4_get_regs_len() and t4_get_regs().
- While here, use t4_get_regs_len() in the ioctl handler for regdump
  instead of inlining it.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7484
2016-08-15 17:42:54 +00:00
John Baldwin
e5e9897c5f Update mailbox writes to work with VF devices.
- Use alternate register locations for the data and control registers for
  VFs.
- Do a dummy read to force the writes to the  mailbox data registers to
  post before the write to the control register on VFs.
- Do not check the PCI-e firmware register for errors on VFs.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7483
2016-08-15 17:41:34 +00:00
John Baldwin
59c1e950b9 Make SGE parameter handling more VF-friendly.
Add fields to hold the SGE control register and free list buffer sizes to
the sge_params structure.  Populate these new fields in
t4_init_sge_params() for PF devices and change t4_read_chip_settings() to
pull these values out of the params structure instead of reading
registers directly.  This will permit t4_read_chip_settings() to be reused
for VF devices which cannot read SGE registers directly.

While here, move the call to t4_init_sge_params() to
get_params__post_init().  The VF driver will populate the SGE parameters
structure via a different method before calling t4_read_chip_settings().

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7476
2016-08-15 17:40:05 +00:00
John Baldwin
ec55567ce6 Track the base absolute ID of ingress and egress queues.
Use this to map an absolute queue ID to a logical queue ID in interrupt
handlers.  For the regular cxgbe/cxl drivers this should be a no-op as
the base absolute ID should be zero.  VF devices have a non-zero base
absolute ID and require this change.  While here, export the absolute ID
of egress queues via a sysctl.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7446
2016-08-09 17:49:42 +00:00
John Baldwin
a56745092c Reserve an adapter flag IS_VF to mark VF devices vs PF devices.
Sponsored by:	Chelsio Communications
2016-08-08 21:45:39 +00:00
John Baldwin
8f6690d385 Fix a typo. 2016-08-08 21:28:02 +00:00
John Baldwin
f454e7ebf5 Add __printflike() to bus_describe_intr() to enable -Wformat checks.
Fix a few places that were passing a raw string as the format to use
a "%s" format string instead.

MFC after:	2 months
2016-08-04 18:29:16 +00:00
Navdeep Parhar
9217931fb4 cxgbe/t4_tom: The page pod arena allocates from pod address space and
not index space.  The minimum valid allocation out of this arena is the
size of a single page pod.

Sponsored by:	Chelsio Communications
2016-08-04 17:29:42 +00:00
John Baldwin
847bfa8eb7 Use the port device name for the iov device for Chelsio T4/T5 cards.
Chelsio T4/T5 adapters are multifunction cards.  The main driver uses
physical function 4 (PF4).  However, VF devices for SR-IOV are only
supported on physical functions 0 through 3, where PF0 creates VFs tied
to port 0, etc.  The t4iov/t5iov driver was previously added to
create VF devices for ports that are present on each adapter.  This
change uses the recently added pci_iov_attach_name() function to
name the character device in /dev/iov after the associated port on
the card (e.g. /dev/iov/cxl0 is used to create VFs that share the
cxl0 port).  With this in place, mark the t4iov/t5iov devices quiet
to prevent them from cluttering dmesg.

Reviewed by:	rstone
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7402
2016-08-03 17:11:08 +00:00
Navdeep Parhar
515b36c5b5 cxgbe/t4_tom: Read the chip's DDP page sizes and save them in a
per-adapter data structure.  This replaces a global array with hardcoded
page sizes.

Sponsored by:	Chelsio Communications
2016-08-02 23:54:21 +00:00
John Baldwin
315048f2ad Store the offset of the KDOORBELL and GTS registers in the softc.
VF devices use a different register layout than PF devices.  Storing
the offset in a value in the softc allows code to be shared between the
PF and VF drivers.

Reviewed by:	np
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D7389
2016-08-01 22:39:51 +00:00
John Baldwin
8dd07eab0e Various fixes to the t4/5nex character device.
- Remove null open/close methods.
- Don't set d_flags to 0 explicitly.
- Remove t5_cdevsw as the .d_name member isn't really used and doesn't
  warrant a separate cdevsw just for the name.
- Use ENOTTY as the error value for an unknown ioctl request.
- Use make_dev_s() to close race with setting si_drv1.

Sponsored by:	Chelsio Communications
2016-07-29 22:11:29 +00:00
John Baldwin
29c229e9fc Mark spg_len and fl_pktshift static.
These variables are no longer exported to t4_netmap.c after r296478.
2016-07-28 17:37:12 +00:00
John Baldwin
07159830be Add support for zero-copy aio_write() on TOE sockets.
AIO write requests for a TOE socket on a Chelsio T4+ adapter can now
DMA directly from the user-supplied buffer.  This is implemented by
wiring the pages backing the user-supplied buffer and queueing special
mbufs backed by raw VM pages to the socket buffer.  The TOE code
recognizes these special mbufs and builds a sglist from the VM page
array associated with the mbuf when queueing a work request to the TOE.

Because these mbufs do not have an associated virtual address, m_data
is not valid.  Thus, the AIO handler does not invoke sosend() directly
for these mbufs but instead inlines portions of sosend_generic() and
tcp_usr_send().

An aiotx_buffer structure is used to describe the user buffer (e.g.
it holds the array of VM pages and a reference to the AIO job).  The
special mbufs reference this structure via m_ext.  Note that a single
job might be split across multiple mbufs (e.g. if it is larger than
the socket buffer size).  The 'ext_arg2' member of each mbuf gives an
offset relative to the backing aiotx_buffer.  The AIO job associated
with an aiotx_buffer structure is completed when the last reference to
the structure is released.

Zero-copy aio_write()'s for connections associated with a given
adapter can be enabled/disabled at runtime via the
'dev.t[45]nex.N.toe.tx_zcopy' sysctl.

MFC after:	1 month
Relnotes:	yes
Sponsored by:	Chelsio Communications
2016-07-27 18:29:35 +00:00
Navdeep Parhar
17146cd543 cxgbe(4): Initialize the adapter queues (fwq and mgmtq) instead of
returning EAGAIN if they aren't available when the user tries to program
a filter.  Do this after validating the filter so that the driver
doesn't bring up the queues if it doesn't have to.
2016-07-26 23:29:37 +00:00
John Baldwin
f91fca5ba7 Add a driver to create VF devices on Chelsio T4/T5 NICs.
Chelsio NICs are a bit unique compared to some other NICs in that they
expose different functionality on different physical functions.  In
particular, PF4 is used to manage the NIC interfaces ('t4nex' and 't5nex').
However, PF4 is not able to create VF devices.  Instead, VFs are only
supported by physical functions 0 through 3.  This commit adds 't4iov'
and 't5iov' drivers that attach to PF0-3.

One extra wrinkle is that the iov devices cannot enable SR-IOV until the
firwmare has been initialized by the main PF4 driver.  To handle this
case, a new t4_if kobj interface has been added to permit cross-calls
between the PF drivers.  The PF4 driver notifies sibling drivers when it
is fully attached.  It also requests sibling drivers to detach before it
detaches.  Sibling drivers query the PF4 driver during their attach
routine to see if it is attached.  If not, the sibling drivers defer
their attach actions until the PF4 driver informs them it is attached.

VF devices are associated with a single port on the NIC.  VF devices
created from PF0 are associated with the first port on the NIC, VFs
from PF1 are associated with the second port, etc.  VF devices can
only be created from a PF device that has an associated port.  Thus,
on a 2-port card, VFs are only supported on PF0 and PF1.

Reviewed by:	np (earlier versions)
MFC after:	1 month
Sponsored by:	Chelsio Communications
2016-07-22 22:46:41 +00:00
John Baldwin
069af0eb14 Install a handler for firmware work request error messages.
If a driver sends an malformed or disallowed work request, the firmware
responds with a work request error.  Previously the driver treated this is
as an unexpected message and panicked.  Now it decodes the error message
to aid in debugging.

Reviewed by:	np (older version)
MFC after:	1 month
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D6950
2016-07-22 21:52:07 +00:00
Enji Cooper
092af585e1 Remove redundant declaration for tcp_dooptions, similar to r302576
netinet/tcp_var.h already defines this function

Differential Revision:	https://reviews.freebsd.org/D7189
MFC after:	1 week
PR:	209920
Reported by:	Mark Millard <markmi@dsl-only.net>
Reviewed by:	np
Tested with:	clang 3.8.0, gcc 4.2.1, gcc 5.3.0
Sponsored by:	EMC / Isilon Storage Division
2016-07-11 17:11:18 +00:00
Navdeep Parhar
a33b046750 cxgbe(4): Add sysctl to display the RSS indirection table size for an
interface.

dev.cxl.<n>.rss_size
dev.vcxl.<n>.rss_size

MFC after:	3 days
2016-07-08 18:13:23 +00:00
Navdeep Parhar
671bf2b8b2 cxgbe(4): Changes to the CPL-handler registration mechanism and code
related to "shared" CPLs.

a) Combine t4_set_tcb_field and t4_set_tcb_field_rpl into a single
function.  Allow callers to direct the response to any iq.  Tidy up
set_ulp_mode_iscsi while there to use names from t4_tcb.h instead of
magic constants.

b) Remove all CPL handler tables from struct adapter.  This reduces its
size by around 2KB.  All handlers are now registered at MOD_LOAD instead
of attach or some kind of initialization/activation.  The registration
functions do not need an adapter parameter any more.

c) Add per-iq handlers to deal with CPLs whose destination cannot be
determined solely from the opcode.  There are 2 such CPLs in use right
now: SET_TCB_RPL and L2T_WRITE_RPL.  The base driver continues to send
filter and L2T_WRITEs over the mgmtq and solicits the reply on fwq.
t4_tom (including the DDP code) now uses the port's ctrlq to send
L2T_WRITEs and SET_TCB_FIELDs and solicits the reply on an ofld_rxq.
fwq and ofld_rxq have different handlers that know what kind of tid to
expect in the reply.  Update t4_write_l2e and callers to to support any
wrq/iq combination.

Approved by:	re@ (kib@)
Sponsored by:	Chelsio Communications
2016-07-05 01:29:24 +00:00
Navdeep Parhar
bf9363d72c cxgbe(4): Avoid a NULL dereference while dumping the L2 table. Entries
used by switching filters that rewrite L2 information do not have any
associated ifnet.

Approved by:	re@ (gjb@)
Sponsored by:	Chelsio Communications
2016-07-01 23:18:49 +00:00
Navdeep Parhar
5e03372b18 cxgbe(4): Do not bring up an interface when IFCAP_TOE is enabled on it.
The interface's queues are functional after VI_INIT_DONE (which is short
of interface-up) and that's all that's needed for t4_tom to communicate
with the chip.

Approved by:	re@ (gjb@)
Sponsored by:	Chelsio Communications
2016-06-29 06:55:30 +00:00
Navdeep Parhar
62291463de cxgbe(4): Merge netmap support from the ncxgbe/ncxl interfaces to the
vcxgbe/vcxl interfaces and retire the 'n' interfaces.  The main
cxgbe/cxl interfaces and tunables related to them are not affected by
any of this and will continue to operate as usual.

The driver used to create an additional 'n' interface for every
cxgbe/cxl interface if "device netmap" was in the kernel.  The 'n'
interface shared the wire with the main interface but was otherwise
autonomous (with its own MAC address, etc.).  It did not have normal
tx/rx but had a specialized netmap-only data path.  r291665 added
another set of virtual interfaces (the 'v' interfaces) to the driver.
These had normal tx/rx but no netmap support.

This revision consolidates the features of both the interfaces into the
'v' interface which now has a normal data path, TOE support, and native
netmap support.  The 'v' interfaces need to be created explicitly with
the hw.cxgbe.num_vis tunable.  This means "device netmap" will not
result in the automatic creation of any virtual interfaces.

The following tunables can be used to override the default number of
queues allocated for each 'v' interface.  nofld* = 0 will disable TOE on
the virtual interface and nnm* = 0 to will disable native netmap
support.

# number of normal NIC queues
hw.cxgbe.ntxq_vi
hw.cxgbe.nrxq_vi

# number of TOE queues
hw.cxgbe.nofldtxq_vi
hw.cxgbe.nofldrxq_vi

# number of netmap queues
hw.cxgbe.nnmtxq_vi
hw.cxgbe.nnmrxq_vi

hw.cxgbe.nnm{t,r}xq{10,1}g tunables have been removed.

--- tl;dr version ---
The workflow for netmap on cxgbe starting with FreeBSD 11 is:
1) "device netmap" in the kernel config.
2) "hw.cxgbe.num_vis=2" in loader.conf.  num_vis > 2 is ok too, you'll
end up with multiple autonomous netmap-capable interfaces for every
port.
3) "dmesg | grep vcxl | grep netmap" to verify that the interface has
netmap queues.
4) Use any of the 'v' interfaces for netmap.  pkt-gen -i vcxl<n>... .
One major improvement is that the netmap interface has a normal data
path as expected.
5) Just ignore the cxl interfaces if you want to use netmap only.  No
need to bring them up.  The vcxl interfaces are completely independent
and everything should just work.
---------------------

Approved by:	re@ (gjb@)
Relnotes:	Yes
Sponsored by:	Chelsio Communications
2016-06-23 02:53:00 +00:00
John Baldwin
b1012d8036 Account for AIO socket operations in thread/process resource usage.
File and disk-backed I/O requests store counts of read/written disk
blocks in each AIO job so that they can be charged to the thread that
completes an AIO request via aio_return() or aio_waitcomplete().  This
change extends AIO jobs to store counts of received/sent messages and
updates socket backends to set these counts accordingly.  Note that
the socket backends are careful to only charge a single messages for
each AIO request even though a single request on a blocking socket might
invoke sosend or soreceive multiple times.  This is to mimic the
resource accounting of synchronous read/write.

Adjust the UNIX socketpair AIO test to verify that the message resource
usage counts update accordingly for aio_read and aio_write.

Approved by:	re (hrs)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D6911
2016-06-21 22:19:06 +00:00
John Baldwin
ae0b1ccbab Use sbused() instead of sbspace() to avoid signed issues.
Inserting a full mbuf with an external cluster into the socket buffer
resulted in sbspace() returning -MLEN.  However, since sb_hiwat is
unsigned, the -MLEN value was converted to unsigned in comparisons.  As a
result, the socket buffer was never autosized.  Note that sb_lowat is signed
to permit direct comparisons with sbspace(), but sb_hiwat is unsigned.
Follow suit with what tcp_output() does and compare the value of sbused()
with sb_hiwat instead.

Approved by:	re (gjb)
Sponsored by:	Chelsio Communications
2016-06-15 21:08:51 +00:00
John Baldwin
fe0bdd1d2c Move backend-specific fields of kaiocb into a union.
This reduces the size of kaiocb slightly. I've also added some generic
fields that other backends can use in place of the BIO-specific fields.

Change the socket and Chelsio DDP backends to use 'backend3' instead of
abusing _aiocb_private.status directly. This confines the use of
_aiocb_private to the AIO internals in vfs_aio.c.

Reviewed by:	kib (earlier version)
Approved by:	re (gjb)
Sponsored by:	Chelsio Communications
Differential Revision:	https://reviews.freebsd.org/D6547
2016-06-15 20:56:45 +00:00
Navdeep Parhar
c4765d2743 cxgbe/t4_tom: Fix inverted assertion in r300895. It is RDMA
connections and not others that are allowed to fail the receive window
check.

Approved by:	re (gjb@)
2016-06-14 21:09:00 +00:00
Navdeep Parhar
390006c70f iw_cxgbe: Make sure that send_abort results in a TCP RST and not a FIN.
Release the hold on ep->com immediately after sending the RST.  This
fixes a bug that sometimes leaves userspace iWARP tools hung when the
user presses ^C.

Submitted by:	Krishnamraju Eraparaju @ Chelsio
Approved by:	re (gjb@)
Sponsored by:	Chelsio Communications
2016-06-14 21:02:36 +00:00
Navdeep Parhar
02f972e8f3 cxgbe(4): Add a sysctl to manage the binding of a txq to a traffic class.
Sponsored by:	Chelsio Communications
2016-06-08 14:15:29 +00:00
Navdeep Parhar
468646f716 cxgbe(4): A couple of fixes to set_sched_queue.
- Validate the scheduling class against the actual limit (which is chip
  specific) instead of a magic number.

- Return an error if an attempt is made to manipulate the tx queues of a
  VI that hasn't been initialized.

Sponsored by:	Chelsio Communications
2016-06-07 07:48:36 +00:00
Navdeep Parhar
0bc01d66b6 cxgbe(4): Provide information about traffic classes in the sysctl mib.
Sponsored by:	Chelsio Communications
2016-06-07 06:42:35 +00:00
Navdeep Parhar
46464b95b0 cxgbe(4): Track the state of the hardware traffic schedulers in the
driver.  This works as long as everyone uses set_sched_class_params
to program them.

Sponsored by:	Chelsio Communications
2016-06-07 00:27:55 +00:00
Navdeep Parhar
cfbe2911c9 cxgbe(4): Break up set_sched_class. Validate the channel number and
min/max rates against their actual limits (which are chip and port
specific) instead of hardcoded constants.

Sponsored by:	Chelsio Communications
2016-06-06 22:51:44 +00:00
Navdeep Parhar
db80d07377 cxgbe(4): Create a reusable struct type for scheduling class parameters.
Sponsored by:	Chelsio Communications
2016-06-06 20:42:46 +00:00
Navdeep Parhar
e6b775f48f iw_cxgbe: Fix panic that occurs when c4iw_ev_handler tries to acquire
comp_handler_lock but c4iw_destroy_cq has already freed the CQ memory
(which is where the lock resides).

Submitted by:	Krishnamraju Eraparaju @ Chelsio
Sponsored by:	Chelsio Communications
2016-06-01 18:46:54 +00:00
Edward Tomasz Napierala
ac3453eba0 Reduce the priority of cxgbei(4) driver, so it doesn't get chosen
by default.  This is a workaround for a too simplistic ICL module
choosing mechanism.  To use it, specify offload in ctl.conf
or iscsi.conf.

This fixes a problem where "kldload cxgbei" wedges the iSCSI stack,
if you don't have a Chelsio card installed, or the endpoints of the
iSCSI session are not reachable through addresses configured
on that interface.

Reviewed by:	np@
MFC after:	1 month
2016-06-01 12:04:04 +00:00
Navdeep Parhar
addd6a52c4 cxgbe/t4_tom: Exempt RDMA connections from a TCP sanity test for now, to
avoid panicking debug kernels.

t4_tom does not keep track of a connection once it switches to ULP mode
iWARP.  If the connection falls out of ULP mode the driver/hardware seq#
etc. are out of sync.  A better fix would be to figure out what the
current seq# are, update the driver's state, and perform all sanity
checks as usual.
2016-05-28 00:38:17 +00:00
Navdeep Parhar
fc511a3302 iw_cxgbe: Plug a lock leak in process_mpa_request().
If the parent is DEAD or connect_request_upcall() fails, the parent
mutex is left locked.  This leads to a hang when process_mpa_request()
is called again for another child of the listening endpoint.

Submitted by:	Krishnamraju Eraparaju @ Chelsio
Obtained from:	upstream iw_cxgb4
Sponsored by:	Chelsio Communications
2016-05-27 23:44:33 +00:00
Navdeep Parhar
69b913d68f iw_cxgbe: Use vmem(9) to manage PBL and RQT allocations.
Submitted by:	Krishnamraju Eraparaju at Chelsio
Reviewed by:	Steve Wise
Sponsored by:	Chelsio Communications
2016-05-27 21:26:26 +00:00
Hans Petter Selasky
fa201e28fc Prepare for activation of LinuxKPI module parameters as read-only
tunable SYSCTL's. Linux module parameters are associated with the
module they belong to. FreeBSD does not share this concept of a parent
module. Instead add macros which define the prefix to use for the
module parameters in the LinuxKPI consumers.

While at it convert all "bool" LinuxKPI module parameters to "byte"
type, because we don't have a "bool" type of SYSCTL in FreeBSD.

Sponsored by:	Mellanox Technologies
MFC after:	1 week
2016-05-25 12:03:21 +00:00
Edward Tomasz Napierala
b891159418 Add mechanism for choosing iSER-capable ICL modules.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-05-24 08:44:45 +00:00
Edward Tomasz Napierala
7deb68ab2c Provide a way for ICL modules to declare they support PIM_UNMAPPED.
MFC after:	1 month
Sponsored by:	The FreeBSD Foundation
2016-05-21 11:10:48 +00:00
John Baldwin
1081d2766c Move the KTR for the update of ddp_active_id on each completion under
VERBOSE_TRACES.

Sponsored by:	Chelsio Communications
2016-05-20 23:08:22 +00:00