Commit Graph

75 Commits

Author SHA1 Message Date
Gleb Smirnoff
28ebe80cab Provide functions to determine presence of a given address
configured on a given interface.

Discussed with:	np
Sponsored by:	Nginx, Inc.
2015-04-17 11:57:06 +00:00
Navdeep Parhar
7ef00d7884 cxgbe/tom: return rx credits promptly if the socket buffer's low water
mark cannot be reached because the window advertised to the peer isn't
wide enough.  While here, tweak the normal credit return too.

MFC after:	1 month
2015-03-31 01:22:20 +00:00
John Baldwin
b12c0a9eeb Move special DDP handling for closing a connection into a new
handle_ddp_close() function in t4_ddp.c as the logic is similar
to handle_ddp_data().  This allows all knowledge of the special
DDP mbufs to be private to t4_ddp.c as well.
2015-03-16 15:56:06 +00:00
John Baldwin
69a088631e Resize receive socket buffers that support autosizing when receiving
TCP data via direct data placement.

Sponsored by:	Chelsio
MFC after:	1 week
2015-03-11 17:35:07 +00:00
Navdeep Parhar
b3d44a6800 cxgbe(4): tidy up some of the interaction between the Upper Layer
Drivers (ULDs) and the base if_cxgbe driver.

Track the per-adapter activation of ULDs in a new "active_ulds" field.
This was done pretty arbitrarily before this change -- via TOM_INIT_DONE
in adapter->flags for TOM, and the (1 << MAX_NPORTS) bit in
adapter->offload_map for iWARP.

iWARP and hw-accelerated iSCSI rely on the TOE (supported by the TOM
ULD).  The rules are:
a) If the iWARP and/or iSCSI ULDs are available when TOE is enabled then
   iWARP and/or iSCSI are enabled too.
b) When the iWARP and iSCSI modules are loaded they go looking for
   adapters with TOE enabled and enable themselves on that adapter.
c) You cannot deactivate or unload the TOM module from underneath iWARP
   or iSCSI.  Any such attempt will fail with EBUSY.

MFC after:	2 weeks
2015-02-08 09:28:55 +00:00
John Baldwin
86f05ea6cf Lock the socket buffer before jumping to the 'out' label if sblock()
fails in t4_soreceive_ddp().
2015-01-26 16:32:41 +00:00
John Baldwin
de5a10ecbc - Update a disabled KASSERT() to use sbused() instead of accessing
the no-longer existant sb_cc sockbuf member.
- Use sbavail() instead of sbused() in t4_soreceive_ddp() to match the
  usage in soreceive_stream() on which it is based.

Discussed with:	glebius (2)
2015-01-26 16:29:14 +00:00
Navdeep Parhar
db8bcd1b21 cxgbe/tom: allocate page pod addresses instead of ppod#.
MFC after:	2 weeks
2015-01-07 06:20:33 +00:00
Navdeep Parhar
f8c479085f cxgbe/tom: use vmem(9) as the DDP page pod allocator.
MFC after:	1 month
2015-01-06 01:30:32 +00:00
Navdeep Parhar
79b93bf6a3 cxgbe/tom: do not engage the TOE's payload chopper for payload < 2 MSS
or for 10Gbps ports.

MFC after:	2 weeks
2015-01-03 00:09:21 +00:00
Navdeep Parhar
402873f32a cxgbe/tom: fix the MSS calculation for IPv6 connections handled by the TOE.
MFC after:	1 week
2015-01-02 21:13:24 +00:00
Navdeep Parhar
dd1be4d418 cxgbe/tom: log some more details in send_flowc_wr.
MFC after:	1 week
2015-01-02 20:52:51 +00:00
John Baldwin
5ad25ceb41 Check for SS_NBIO in so->so_state instead of sb->sb_flags in
soreceive_stream().

Differential Revision:	https://reviews.freebsd.org/D1299
Reviewed by:	bz, gnn
MFC after:	1 week
2014-12-15 17:52:08 +00:00
Navdeep Parhar
a7570ee305 Move KTR_CXGBE from t4_tom.h to adapter.h so that the base if_cxgbe
code can use it too.

MFC after:	1 week
2014-12-12 21:54:59 +00:00
Gleb Smirnoff
651e4e6a30 Merge from projects/sendfile: extend protocols API to support
sending not ready data:
o Add new flag to pru_send() flags - PRUS_NOTREADY.
o Add new protocol method pru_ready().

Sponsored by:	Nginx, Inc.
Sponsored by:	Netflix
2014-11-30 13:24:21 +00:00
Gleb Smirnoff
0f9d0a73a4 Merge from projects/sendfile:
o Introduce a notion of "not ready" mbufs in socket buffers.  These
mbufs are now being populated by some I/O in background and are
referenced outside.  This forces following implications:
- An mbuf which is "not ready" can't be taken out of the buffer.
- An mbuf that is behind a "not ready" in the queue neither.
- If sockbet buffer is flushed, then "not ready" mbufs shouln't be
  freed.

o In struct sockbuf the sb_cc field is split into sb_ccc and sb_acc.
  The sb_ccc stands for ""claimed character count", or "committed
  character count".  And the sb_acc is "available character count".
  Consumers of socket buffer API shouldn't already access them directly,
  but use sbused() and sbavail() respectively.
o Not ready mbufs are marked with M_NOTREADY, and ready but blocked ones
  with M_BLOCKED.
o New field sb_fnrdy points to the first not ready mbuf, to avoid linear
  search.
o New function sbready() is provided to activate certain amount of mbufs
  in a socket buffer.

A special note on SCTP:
  SCTP has its own sockbufs.  Unfortunately, FreeBSD stack doesn't yet
allow protocol specific sockbufs.  Thus, SCTP does some hacks to make
itself compatible with FreeBSD: it manages sockbufs on its own, but keeps
sb_cc updated to inform the stack of amount of data in them.  The new
notion of "not ready" data isn't supported by SCTP.  Instead, only a
mechanical substitute is done: s/sb_cc/sb_ccc/.
  A proper solution would be to take away struct sockbuf from struct
socket and allow protocols to implement their own socket buffers, like
SCTP already does.  This was discussed with rrs@.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-11-30 12:52:33 +00:00
Gleb Smirnoff
cfa6009e36 In preparation of merging projects/sendfile, transform bare access to
sb_cc member of struct sockbuf to a couple of inline functions:

sbavail() and sbused()

Right now they are equal, but once notion of "not ready socket buffer data",
will be checked in, they are going to be different.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2014-11-12 09:57:15 +00:00
Navdeep Parhar
527e4e62ac Always request a completion for every work request for iWARP. The
initial MPA exchange must be tracked this way so that t4_tom's state for
the tid is all clean at the time the tid transitions to RDMA mode.  Once
it does, t4_tom is out of the way and iw_cxgbe uses the qp endpoints
directly.

Sponsored by:	Chelsio Communications
2014-10-28 18:10:57 +00:00
Navdeep Parhar
19abdd0654 cxgbe/tom: don't leak resources tied to an active open request that
cannot be sent to the chip because a prerequisite L2 resolution
failed.

Submitted by:	Hariprasad at chelsio dot com (original version)
MFC after:	2 weeks.
2014-10-07 21:26:22 +00:00
Navdeep Parhar
3a260c2a19 Update comment (missed this bit in r272079). 2014-09-24 20:08:43 +00:00
Navdeep Parhar
13251b21e1 cxgbe/tom: Catch up with r271119, syncache_add doesn't need tcbinfo lock. 2014-09-24 20:04:11 +00:00
Navdeep Parhar
0fe982772d Some hooks in cxgbe(4) for the offloaded iSCSI driver.
(I'm committing this on behalf of my colleagues in the Storage team
at Chelsio).

Submitted by:	Sreenivasa Honnur <shonnur at chelsio dot com>
Sponsored by:	Chelsio Communications.
2014-07-24 18:39:08 +00:00
Attilio Rao
3ae10f7477 - Modify vm_page_unwire() and vm_page_enqueue() to directly accept
the queue where to enqueue pages that are going to be unwired.
- Add stronger checks to the enqueue/dequeue for the pagequeues when
  adding and removing pages to them.

Of course, for unmanaged pages the queue parameter of vm_page_unwire() will
be ignored, just as the active parameter today.
This makes adding new pagequeues quicker.

This change effectively modifies the KPI.  __FreeBSD_version will be,
however, bumped just when the full cache of free pages will be
evicted.

Sponsored by:	EMC / Isilon storage division
Reviewed by:	alc
Tested by:	pho
2014-06-16 18:15:27 +00:00
Bjoern A. Zeeb
255cd9fd58 Move the tcp_fields_to_host() and tcp_fields_to_net() (inline)
functions to the tcp_var.h header file in order to avoid further
duplication with upcoming commits.

Reviewed by:	np
MFC after:	2 weeks
2014-05-23 20:15:01 +00:00
Navdeep Parhar
88bb82e511 Do not create a hardware IPv6 server if the listen address is not
in6addr_any and is not in the CLIP table either.  This fixes a reported
TOE+IPv6 NULL-dereference panic in do_pass_open_rpl().

While here, stop creating hardware servers for any loopback address.
It's just a waste of server tids.

MFC after:	1 week
2013-12-17 21:41:23 +00:00
Navdeep Parhar
93e9cae3fa Read card capabilities after firmware initialization, instead of setting
them up as part of firmware initialization (which the driver gets to do
only if it's the master driver).

Read the range of tids available for the ETHOFLD functionality if it's
enabled.

New is_ftid() and is_etid() functions to test whether a tid falls within
the range of filter tids or ETHOFLD tids respectively.

MFC after:	2 weeks
2013-12-14 03:08:03 +00:00
Gleb Smirnoff
66e01d73cd - Provide necessary includes.
- Remove unnecessary includes.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-29 11:17:49 +00:00
Gleb Smirnoff
c3322cb91c Include necessary headers that now are available due to pollution
via if_var.h.

Sponsored by:	Netflix
Sponsored by:	Nginx, Inc.
2013-10-28 07:29:16 +00:00
Navdeep Parhar
48d05478bf cxgbe(4): Update T4 and T5 firmwares to 1.9.12.0 2013-10-14 21:25:07 +00:00
Navdeep Parhar
eb22728291 Rework the tx credit mechanism between the cxgbe/tom driver
and the card.  This helps smooth out some burstiness in the
exchange.

Approved by:	re (glebius)
2013-09-09 04:38:57 +00:00
Navdeep Parhar
c81d56a0aa Fix a miscalculation that caused cxgbe/tom to auto-increment
a TOE socket's tx buffer size too aggressively.

Approved by:	re (delphij)
2013-09-09 00:16:59 +00:00
Navdeep Parhar
4f641559c7 For TOE connections, the window scale factor in CPL_PASS_ACCEPT_REQ is
set to 15 to indicate that the peer did not send a window scale option
with its SYN.  Do not send a window scale option in the SYN|ACK reply
in that case.
2013-09-03 23:34:04 +00:00
Navdeep Parhar
319a31ea18 Change t4_list_lock and t4_uld_list_lock from mutexes to sx'es.
- tom_uninit had to be reworked not to hold the adapter lock (a mutex)
  around t4_deactivate_uld, which acquires the uld_list_lock.
- the ifc_match for the interface cloner that creates the tracer ifnet
  had to be reworked as the kernel calls ifc_match with the global
  if_cloners_mtx held.
2013-08-28 20:59:22 +00:00
Navdeep Parhar
9800517691 Add hooks in base cxgbe(4) for the iWARP upper-layer driver. Update a
couple of assertions in the TOE driver as well.
2013-08-28 20:45:45 +00:00
Navdeep Parhar
6eb3180fb2 - Make note of interface MTU change if the rx queues exist, and not just
when the interface is up.
- Add a tunable to control the TOE's rx coalesce feature (enabled by
  default as it always has been).  Consider the interface MTU or the
  coalesce size when deciding which cluster zone to use to fill the
  offload rx queue's free list.  The tunable is:
  dev.{t4nex,t5nex}.<N>.toe.rx_coalesce

MFC after:	1 day
2013-07-04 21:19:01 +00:00
Navdeep Parhar
50ce3d40aa Pay attention to TCP_NODELAY when it's set/unset after the connection
is established.

MFC after:	1 day
2013-07-04 19:44:30 +00:00
Navdeep Parhar
054a2dc11c The T5 allows the driver to specify the ISS. Do so; use the ISS picked
by the kernel.

MFC after:	1 day
2013-07-04 18:41:21 +00:00
Navdeep Parhar
c337fa30af - Read all TP parameters in one place.
- Read the filter mode, calculate various shifts, and use them
  properly during active open (in select_ntuple).

MFC after:	1 day
2013-07-04 17:55:52 +00:00
Navdeep Parhar
f81cb396de cxgbe/tom: Allow caller to select the queue (control or data) used to
send the CPL_SET_TCB_FIELD request in t4_set_tcb_field().

MFC after:	1 week
2013-06-11 21:20:23 +00:00
Navdeep Parhar
e0f8a7f4da cxgbe/tom: Fix bad signed/unsigned mixup in the stid allocator. This
fixes a panic when allocating a mixture of IPv6 and IPv4 stids.

MFC after:	1 week
2013-06-08 07:23:26 +00:00
Navdeep Parhar
688dba74a5 cxgbe/tom: Do not use M_PROTO1 to mark rx zero-copy mbufs as special.
All the M_PROTOn flags are clobbered when an mbuf is appended to the
socket buffer.

MFC after:	1 week
2013-05-03 18:37:50 +00:00
Navdeep Parhar
88c4ff7bf1 Fix DDP breakage introduced in r248925. Bitwise OR has higher
precedence than ternary conditional.

MFC after:	1 week
2013-04-30 19:57:21 +00:00
Navdeep Parhar
dd181b2652 cxgbe/tom: Update the CLIP table on the chip when there are changes
to the list of IPv6 addresses on the system.  The table is used for
TOE+IPv6 only.
2013-04-18 19:52:11 +00:00
Navdeep Parhar
b7a7c6d0c3 cxgbe/tom: Slight simplification of code that calculates options2.
MFC after:	3 days
2013-04-11 21:36:01 +00:00
Navdeep Parhar
d14b0ac129 cxgbe(4): Add support for Chelsio's Terminator 5 (aka T5) ASIC. This
includes support for the NIC and TOE features of the 40G, 10G, and
1G/100M cards based on the T5.

The ASIC is mostly backward compatible with the Terminator 4 so cxgbe(4)
has been updated instead of writing a brand new driver.  T5 cards will
show up as cxl (short for cxlgb) ports attached to the t5nex bus driver.

Sponsored by:	Chelsio
2013-03-30 02:26:20 +00:00
Navdeep Parhar
d92ed49c94 Install an extra hold on the newly allocated synq entry so that it
cannot be freed while do_pass_accept_req is running.  This closes a race
where do_pass_establish on another CPU (the driver chose a different
queue for the new tid) expands the synq entry into a full PCB and then
releases the only hold on it, all while do_pass_accept_req is still
running.

MFC after:	3 days
2013-01-26 03:23:28 +00:00
Navdeep Parhar
dfd1b3a02f Add a couple of missing error codes. Treat CPL_ERR_KEEPALV_NEG_ADVICE as
negative advice and not a fatal error.

MFC after:	3 days
2013-01-26 03:01:51 +00:00
Navdeep Parhar
c995301b2d cxgbe/tom: Add support for fully offloaded TCP/IPv6 connections (passive open).
MFC after:	1 week
2013-01-15 18:50:40 +00:00
Navdeep Parhar
8be0815671 cxgbe/tom: Add support for fully offloaded TCP/IPv6 connections (active open).
MFC after:	1 week
2013-01-15 18:38:51 +00:00
Navdeep Parhar
87aa6825ff cxgbe/tom: Basic CLIP table management.
This is the Compressed Local IPv6 table on the chip.  To save space, the
chip uses an index into this table instead of a full IPv6 address in
some of its hardware data structures.

For now the driver fills this table with all the local IPv6 addresses
that it sees at the time the table is initialized.  I'll improve this
later so that the table is updated whenever new IPv6 addresses are
configured or existing ones deleted.

MFC after:	1 week
2013-01-15 07:07:29 +00:00