Commit Graph

337 Commits

Author SHA1 Message Date
np
2a87fdbf3a Initialize the response queue mutex a bit earlier to avoid a panic that
occurs if t3_sge_alloc_qset fails and then t3_free_qset attempts to
destroy an uninitialized mutex.

Submitted by:	Vijay Singh <vijju dot singh at gmail>
MFC after:	3 days
2012-10-25 18:11:04 +00:00
eadler
3f7a414911 remove duplicate semicolons where possible.
Approved by:	cperciva
MFC after:	1 week
2012-10-22 03:00:37 +00:00
ed
dc8da81941 Prefer __containerof() over __member2struct().
The former works better with qualifiers, but also properly type checks
the input pointer.
2012-10-19 13:26:40 +00:00
kevlo
ceb08698f2 Revert previous commit...
Pointyhat to:	kevlo (myself)
2012-10-10 08:36:38 +00:00
kevlo
8747a46991 Prefer NULL over 0 for pointers 2012-10-09 08:27:40 +00:00
gavin
5005c75c5d Align the PCI Express #defines with the style used for the PCI-X
#defines.  This also has the advantage that it makes the names more
compact, iand also allows us to correct the non-uniform naming of
the PCIM_LINK_* defines, making them all consistent amongst themselves.

This is a mostly mechanical rename:
  s/PCIR_EXPRESS_/PCIER_/g
  s/PCIM_EXP_/PCIEM_/g
  s/PCIM_LINK_/PCIEM_LINK_/g

When this is MFC'd, #defines will be added for the old names to assist
out-of-tree drivers.

Discussed with:	jhb
MFC after:	1 week
2012-09-18 22:04:59 +00:00
jhb
353407bd01 Attach interrupt handlers during attach instead of during the first time
the interface is brought up.  Without this, the boot time interrupt
round-robin assignment does not think the allocated interrupt resources
are active and leaves them assigned to CPU 0.

While here, add descriptive tags to each interrupt handler when MSI-X
is used.

Reviewed by:	np
MFC after:	1 week
2012-08-30 17:47:39 +00:00
np
aac78a84d9 Deal with the case where a syncache entry added by the TOE driver is
evicted from the syncache but a later syncache_expand succeeds because
of syncookies.  The TOE driver has to resort to more direct means to
install its hooks in the socket in this case.
2012-08-21 22:23:17 +00:00
np
7a7bbaad5a Correctly handle the case where an inp has already been dropped by the time
the TOE driver reports that an active open failed.  toe_connect_failed is
supposed to handle this but it should be provided the inpcb instead of the
tcpcb which may no longer be around.
2012-08-21 18:09:33 +00:00
dim
c31e7aae74 In cxgb(4), in function iwch_reregister_phys_mem(), initialize the
'npages' variable to zero, to avoid using it uninitialized in certain
cases.

Found by:	clang
Reviewed by:	np
MFC after:	1 week
2012-08-06 18:51:14 +00:00
np
87ea126c75 cxgb(4): IPv6 rx/tx hw checksum, IPv6 TSO and LRO too.
(Some parts already worked, this makes it complete).
2012-06-30 02:11:53 +00:00
np
67d5f1a727 - Updated TOE support in the kernel.
- Stateful TCP offload drivers for Terminator 3 and 4 (T3 and T4) ASICs.
  These are available as t3_tom and t4_tom modules that augment cxgb(4)
  and cxgbe(4) respectively.  The cxgb/cxgbe drivers continue to work as
  usual with or without these extra features.

- iWARP driver for Terminator 3 ASIC (kernel verbs).  T4 iWARP in the
  works and will follow soon.

Build-tested with make universe.

30s overview
============
What interfaces support TCP offload?  Look for TOE4 and/or TOE6 in the
capabilities of an interface:
# ifconfig -m | grep TOE

Enable/disable TCP offload on an interface (just like any other ifnet
capability):
# ifconfig cxgbe0 toe
# ifconfig cxgbe0 -toe

Which connections are offloaded?  Look for toe4 and/or toe6 in the
output of netstat and sockstat:
# netstat -np tcp | grep toe
# sockstat -46c | grep toe

Reviewed by:	bz, gnn
Sponsored by:	Chelsio communications.
MFC after:	~3 months (after 9.1, and after ensuring MFC is feasible)
2012-06-19 07:34:13 +00:00
bz
1bc3ff1445 MFp4 bz_ipv6_fast:
Allow LRO to work on IPv6 as well.
  Fix the module Makefile to at least properly inlcude opt_inet6.h
  and allow builds without INET or INET6.

  Sponsored by:	The FreeBSD Foundation
  Sponsored by:	iXsystems

Reviewed by:	gnn (as part of the whole)
MFC After:	3 days
2012-05-25 03:00:34 +00:00
bz
9efa72fd42 MFp4 bz_ipv6_fast:
Significantly update tcp_lro for mostly two things:
  1) introduce basic support for IPv6 without extension headers.
  2) try hard to also get the incremental checksum updates right,
     especially also in the IPv4 case for the IP and TCP header.

  Move variables around for better locality, factor things out into
  functions, allow checksum updates to be compiled out, ...

  Leave a few comments on further things to look at in the future,
  though that is not the full list.

  Update drivers with appropriate #includes as needed for IPv6 data
  type in LRO.

  Sponsored by:	The FreeBSD Foundation
  Sponsored by:	iXsystems

Reviewed by:	gnn (as part of the whole)
MFC After:	3 days
2012-05-24 23:03:23 +00:00
scottl
32e16cef40 Convert a number of drivers to obtaining their parent DMA tag from their
PCI device attachment.
2012-03-12 08:03:51 +00:00
np
1a089455d6 Add IPv6 TSO (including TSO+VLAN) support to cxgb(4).
If an IPv6 packet has extension headers the kernel needs to deal with it
itself.  For the rest it can set various CSUM_XXX flags and the driver
will act on them.
2012-02-09 23:19:09 +00:00
np
e200cb94e2 Allocate the BAR for userspace doorbells after the is_offload check
is functional.

MFC after:	3 days
2012-02-08 03:02:12 +00:00
np
e2807c4b0c Remove if_start from cxgb and cxgbe.
Submitted by:	jhb
MFC after:	3 days
2012-02-07 07:32:39 +00:00
np
96756edba8 Fix return value of function.
Submitted by:	Dan McGregor <dan dot mcgregor at usask dot ca>
2011-12-23 01:39:10 +00:00
marius
17e14c6132 - There's no need to overwrite the default device method with the default
one. Interestingly, these are actually the default for quite some time
  (bus_generic_driver_added(9) since r52045 and bus_generic_print_child(9)
  since r52045) but even recently added device drivers do this unnecessarily.
  Discussed with: jhb, marcel
- While at it, use DEVMETHOD_END.
  Discussed with: jhb
- Also while at it, use __FBSDID.
2011-11-22 21:28:20 +00:00
ed
0c56cf839d Mark all SYSCTL_NODEs static that have no corresponding SYSCTL_DECLs.
The SYSCTL_NODE macro defines a list that stores all child-elements of
that node. If there's no SYSCTL_DECL macro anywhere else, there's no
reason why it shouldn't be static.
2011-11-07 15:43:11 +00:00
np
34341b0783 Update T3 firmware to 7.11.0
Changes since 7.8.0 (from the official changelog):

- Fixed sporadic interrupt generation for associated CQ when processing
  a local invalidate work request
- Changes to core scheduling to avoid starving requests from the host
  under heavy RDMA Read Request load (e.g. packets to the wire)

- Programmed the tp tx resource limiter in function of the traffic (only
  affects iWarp)

- Increased the egress NIC gather list length from 36 to 46 entries

MFC after:	1 week
2011-03-25 20:53:02 +00:00
np
9b158a1ef4 t3_free_sge_resources should be given the number of qsets it needs to free.
MFC after:	1 week
2011-03-24 01:16:48 +00:00
np
12faa8eb0c T3C initialization should setup the parity fence too.
MFC after:	1 week
2011-03-24 01:13:28 +00:00
jhb
00c3c01f4f Do a sweep of the tree replacing calls to pci_find_extcap() with calls to
pci_find_cap() instead.
2011-03-23 13:10:15 +00:00
brucec
6d9b42b486 Fix typos - remove duplicate "the".
PR:	bin/154928
Submitted by:	Eitan Adler <lists at eitanadler.com>
MFC after: 	3 days
2011-02-21 09:01:34 +00:00
mdf
7fc649fc41 Explicitly wire the user buffer rather than doing it implicitly in
sbuf_new_for_sysctl(9).  This allows using an sbuf with a SYSCTL_OUT
drain for extremely large amounts of data where the caller knows that
appropriate references are held, and sleeping is not an issue.

Inspired by:	rwatson
2011-01-27 00:34:12 +00:00
mdf
6b5f615b7c Introduce signed and unsigned version of CTLTYPE_QUAD, renaming
existing uses.  Rename sysctl_handle_quad() to sysctl_handle_64().
2011-01-19 23:00:25 +00:00
mdf
6648b8cede sysctl(8) should use the CTLTYPE to determine the type of data when
reading.  (This was already done for writing to a sysctl).  This
requires all SYSCTL setups to specify a type.  Most of them are now
checked at compile-time.

Remove SYSCTL_*X* sysctl additions as the print being in hex should be
controlled by the -x flag to sysctl(8).

Succested by:	bde
2011-01-19 17:04:07 +00:00
mdf
2978c8b7ae sysctl(9) cleanup checkpoint: amd64 GENERIC builds cleanly.
Commit the cxgb driver piece.
2011-01-12 19:53:44 +00:00
alc
971b02b7bc Introduce and use a new VM interface for temporarily pinning pages. This
new interface replaces the combined use of vm_fault_quick() and
pmap_extract_and_hold() throughout the kernel.

In collaboration with:	kib@
2010-12-25 21:26:56 +00:00
alc
9769bb9f83 The local variable "rv" is still required by vm_fault_hold_user_pages(). 2010-12-20 23:41:31 +00:00
alc
be5201b0d1 Introduce vm_fault_hold() and use it to (1) eliminate a long-standing race
condition in proc_rwmem() and to (2) simplify the implementation of the
cxgb driver's vm_fault_hold_user_pages().  Specifically, in proc_rwmem()
the requested read or write could fail because the targeted page could be
reclaimed between the calls to vm_fault() and vm_page_hold().

In collaboration with:	kib@
MFC after:	6 weeks
2010-12-20 22:49:31 +00:00
alc
303f816df2 Implement and use a single optimized function for unholding a set of pages.
Reviewed by:	kib@
2010-12-17 22:41:22 +00:00
avg
6d67849166 fix incorrect use of atomic_set_xxx in cxgb
There is no need to use an atomic operation at structure initialization
time.
Note that the file changed is not connected to the build at this time.

Reviewed by:	jhb (general issue)
Approved by:	np
MFC after:	2 weeks
2010-12-11 10:14:08 +00:00
mdf
5695ef4698 Re-add r212370 now that the LOR in powerpc64 has been resolved:
Add a drain function for struct sysctl_req, and use it for a variety
of handlers, some of which had to do awkward things to get a large
enough SBUF_FIXEDLEN buffer.

Note that some sysctl handlers were explicitly outputting a trailing
NUL byte.  This behaviour was preserved, though it should not be
necessary.

Reviewed by:    phk (original patch)
2010-09-16 16:13:12 +00:00
np
efe139f428 Fix t3_gate_rx_traffic and t3_open_rx_traffic. Parts of them always operated
on XGMAC0 instead of the specified XGMAC.

MFC after:	3 days
2010-09-15 21:50:37 +00:00
mdf
3ed6eac561 Revert r212370, as it causes a LOR on powerpc. powerpc does a few
unexpected things in copyout(9) and so wiring the user buffer is not
sufficient to perform a copyout(9) while holding a random mutex.

Requested by: nwhitehorn
2010-09-13 18:48:23 +00:00
mdf
bc54684253 Add a drain function for struct sysctl_req, and use it for a variety of
handlers, some of which had to do awkward things to get a large enough
FIXEDLEN buffer.

Note that some sysctl handlers were explicitly outputting a trailing NUL
byte.  This behaviour was preserved, though it should not be necessary.

Reviewed by:	phk
2010-09-09 18:33:46 +00:00
np
35fdf713ab Fix tx pause quanta and timer calculations.
MFC after:	3 days
2010-08-15 20:52:15 +00:00
np
82b574175c Always reset the XGMAC's XAUI PCS on a link up.
MFC after:	3 days
2010-08-15 20:45:16 +00:00
np
18dccf2b73 wakeup is required if the adapter lock is released anywhere during
init and not just for the may_sleep case.

Pointed out by:	Isilon
MFC after:	3 days
2010-08-15 20:34:51 +00:00
jhb
ba4940d821 - Change the warning about PCI-e links narrower than x8 to only apply to
10G cards.  1G cards are x4 only.
- Use constants from pcireg.h for reading the current link width.
- Use pci_set_max_read_req() rather than implementing it by hand.

Reviewed by:	np
MFC after:	1 week
2010-07-26 17:31:15 +00:00
np
ad7dfe8fd7 Improve cxgb(4)'s behaviour when faced with temporarily "bouncy" links:
- Run the adapter's tick at 1Hz and remove link state checks from it.
  Instead, have each port check its link state.  Delay the check so that
  it takes place slightly after the driver is notified of a change in
  link state.  This is a cheap way to debounce these notifications if
  many are received in rapid succession.  POLL_LINK_1ST_TIME flag can
  also be eliminated as a side effect of these changes.
- Do not reset the PHY when link goes down.
- Clear port's link_fault flag if the PHY indicates link is down.
- get_link_status_r should leave speed and duplex alone when link is down.

MFC after:	1 month
2010-07-09 00:38:00 +00:00
np
acfd54a769 Eliminate ext_intr_task. The "slow" interrupt handler is already
running on the adapter's task queue.  Just do what the task does
instead of enqueueing it.

MFC after:	3 days
2010-07-09 00:36:35 +00:00
np
1ed3d6a605 Fix bufsize calculation so that cxgbtool can display information for the
last I/O queue too.

MFC after:	3 days
2010-07-09 00:35:09 +00:00
alc
21a2cb3935 Catch up with the page and page queues locking changes. 2010-06-18 23:14:16 +00:00
np
a3db6f34d7 cxgb(4): add knob to get packet timestamps from the hardware.
The T3 ASIC can provide an incoming packet's timestamp instead of its RSS hash.
The timestamp is just a counter running off the card's clock.  With a 175MHz
clock an increment represents ~5.7ns and the 32 bit value wraps around in ~25s.

# sysctl -d dev.cxgbc.0.pkt_timestamp
dev.cxgbc.0.pkt_timestamp: provide packet timestamp instead of connection hash

# sysctl -d dev.cxgbc.0.core_clock
dev.cxgbc.0.core_clock: core clock frequency (in KHz)
# sysctl dev.cxgbc.0.core_clock
dev.cxgbc.0.core_clock: 175000
2010-06-12 22:33:04 +00:00
np
6a6e0cff3d make format string a string literal.
Reported by:	clang
2010-06-12 22:24:39 +00:00
np
39b4053381 cxgb(4): add an 'nfilters' tunable that lets the user place an upper
limit on the number of hardware filters (and thus the amount of TCAM
reserved for filtering).
2010-06-07 08:23:16 +00:00