* Use ATH_RC_NUM instead of '4' when iterating over the ratecontrol series
array.
* A few style(9) fixes, hopefully no regressions here.
* Add some comments that better describe what's going on.
The existing code tries to use the beacon miss timer to signal that the AP
has gone away. Unfortunately this doesn't seem to be behaving itself.
I'll try to investigate why this is for the sake of completeness.
The result is the STA will stay "associated" to the AP it was associated
with when it suspended. It never receives a bmiss notification so it
never tries reassociating.
PR: kern/169084
* Resize some types. In particular, bfs_seqno can be uint16_t for now.
Previous work would assign the unassigned seqno a value of -1, which
I obviously can't do here.
* Remove bfs_pktdur. It was in the original code but nothing so far uses
it.
This gets ath_buf down (on my i386 system) to 292 bytes from 300 bytes.
I'd rather it be much, much smaller.
fixed for 802.11n TX, this needs to be disabled or users wlil see randomly
hanging aggregation sessions.
Whilst I'm here, remove the warning about 802.11n being full of dragons.
It's nowhere near that scary now.
ath_start() is called.
This (defaults to 10 frames) gives for a little headway in the TX ath_buf
allocation, so buffer cloning is still possible.
This requires a lot omre experimenting and tuning.
It also doesn't stop a node/TID from consuming all of the available
ath_buf's, especially when the node is going through high packet loss
or only talking at a low TX rate. It also doesn't stop a paused TID
from taking all of the ath_bufs. I'll look at fixing that up in subsequent
commits.
PR: kern/168170
traffic.
* Create sc_mgmt_txbuf and sc_mgmt_txdesc, initialise/free them appropriately.
* Create an enum to represent buffer types in the API.
* Extend ath_getbuf() and _ath_getbuf_locked() to take the above enum.
* Right now anything sent via ic_raw_xmit() allocates via ATH_BUFTYPE_MGMT.
This may not be very useful.
* Add ATH_BUF_MGMT flag (ath_buf.bf_flags) which indicates the current buffer
is a mgmt buffer and should go back onto the mgmt free list.
* Extend 'txagg' to include debugging output for both normal and mgmt txbufs.
* When checking/clearing ATH_BUF_BUSY, do it on both TX pools.
Tested:
* STA mode, with heavy UDP injection via iperf. This filled the TX queue
however BARs were still going out successfully.
TODO:
* Initialise the mgmt buffers with ATH_BUF_MGMT and then ensure the right
type is being allocated and freed on the appropriate list. That'd save
a write operation (to bf->bf_flags) on each buffer alloc/free.
* Test on AP mode, ensure that BAR TX and probe responses go out nicely
when the main TX queue is filled (eg with paused traffic to a TID,
awaiting a BAR to complete.)
PR: kern/168170
(or direct dispatch) behind the TXQ lock (which, remember, is doubling
as the TID lock too for now.)
This ensures that:
(a) the sequence number and the CCMP PN allocation is done together;
(b) overlapping transmit paths don't interleave frames, so we don't
end up with the original issue that triggered kern/166190.
Ie, that we don't end up with seqno A, B in thread 1, C, D in
thread 2, and they being queued to the software queue as "A C D B"
or similar, leading to the BAW stalls.
This has been tested:
* both STA and AP modes with INVARIANTS and WITNESS;
* TCP and UDP TX;
* both STA->AP and AP->STA.
STA is a Routerstation Pro (single CPU MIPS) and the AP is a dual-core
Centrino.
PR: kern/166190
scheduled from the head of the software queue rather than trying to
queue the newly given frame.
This leads to some rather unfortunate out of order (but still valid
as it's inside the BAW) frame TX.
This now:
* Always queues the frame at the end of the software queue;
* Tries to direct dispatch the frame at the head of the software queue,
to try and fill up the hardware queue.
TODO:
* I should likely try to queue as many frames to the hardware as I can
at this point, rather than doing one at a time;
* ath_tx_xmit_aggr() may fail and this code assumes that it'll schedule
the TID. Otherwise TX may stall.
PR: kern/166190
This is an unfortunate byproduct of how the routine is used - it's called
with the head frame on the queue, but if the frame is failed, it's inserted
into the tail of the queue.
Because of this, the sequence numbers would get all shuffled around and
the BAW would be bumped past this sequence number, that's now at the
end of the software queue. Then, whenever it's time for that frame
to be transmitted, it'll be immediately outside of the BAW and TX will
stall until the BAW catches up.
It can also result in all kinds of weird duplicate BAW frames, leading
to hilarious panics.
PR: kern/166190
This showed up when doing heavy UDP throughput on SMP machines.
The problem with this is because the 802.11 sequence number is being
allocated separately to the CCMP PN replay number (which is assigned
during ieee80211_crypto_encap()).
Under significant throughput (200+ MBps) the TX path would be stressed
enough that frame TX/retry would force sequence number and PN allocation
to be out of order. So once the frames were reordered via 802.11 seqnos,
the CCMP PN would be far out of order, causing most frames to be discarded
by the receiver.
I've fixed this in some local work by being forced to:
(a) deal with the issues that lead to the parallel TX causing out of
order sequence numbers in the first place;
(b) fix all the packet queuing issues which lead to strange (but mostly
valid) TX.
I'll begin fixing these in a subsequent commit or five.
PR: kern/166190
it turns out that it negatively affects performance. I'm stil investigating
exactly why deferring the IO causes such negative TCP performance but
doesn't affect UDP preformance.
Leave the ath_tx_kick() change in there however; it's going to be useful
to have that there for if_transmit() work.
PR: kern/168649
called to "kick" along TX.
For now, schedule a taskqueue call.
Later on I may go back to the direct call of ath_rx_tasklet() - but for
now, this will do.
I've tested UDP and TCP TX. UDP TX still achieves 240MBit, but TCP
TX gets stuck at around 100MBit or so, instead of the 150MBit it should
be at. I'll re-test with no ACPI/power/sleep states enabled at startup
and see what effect it has.
This is in preparation for supporting an if_transmit() path, which will
turn ath_tx_kick() into a NUL operation (as there won't be an ifnet
queue to service.)
Tested:
* AR9280 STA
TODO:
* test on AR5416, AR9160, AR928x STA/AP modes
PR: kern/168649
implementing parallel TX and TX/RX completion can be done without
simply abusing long-held locks.
Right now, multiple concurrent ath_start() entries can result in
frames being dequeued out of order. Well, they're dequeued in order
fine, but if there's any preemption or race between CPUs between:
* removing the frame from the ifnet, and
* calling and runningath_tx_start(), until the frame is placed on a
software or hardware TXQ
Then although dequeueing the frame is in-order, queueing it to the hardware
may be out of order.
This is solved in a lot of other drivers by just holding a TX lock over
a rather long period of time. This lets them continue to direct dispatch
without races between dequeue and hardware queue.
Note to observers: if_transmit() doesn't necessarily solve this.
It removes the ifnet from the main path, but the same issue exists if
there's some intermediary queue (eg a bufring, which as an aside also
may pull in ifnet when you're using ALTQ.)
So, until I can sit down and code up a much better way of doing parallel
TX, I'm going to leave the TX path using a deferred taskqueue task.
What I will likely head towards is doing a direct dispatch to hardware
or software via if_transmit(), but it'll require some driver changes to
allow queues to be made without using the really large ath_buf / ath_desc
entries.
TODO:
* Look at how feasible it'll be to just do direct dispatch to
ath_tx_start() from if_transmit(), avoiding doing _any_ intermediary
serialisation into a global queue. This may break ALTQ for example,
so I have to be delicate.
* It's quite likely that I should break up ath_tx_start() so it
deposits frames onto the software queues first, and then only fill
in the 802.11 fields when it's being queued to the hardware.
That will make the if_transmit() -> software queue path very
quick and lightweight.
* This has some very bad behaviour when using ACPI and Cx states.
I'll do some subsequent analysis using KTR and schedgraph and file
a follow-up PR or two.
PR: kern/168649
These aren't strictly needed at the moment as we're not doing APSM
and forcing the NIC in and out of network sleep. But, they don't hurt.
Tested:
* AR9280 (mini-PCIe)
Obtained from: Qualcomm Atheros, Linux ath9k
* Now that ah_configPCIE is called for both power on and suspend/resume,
make sure the right bit(s) are cleared and set when suspending and
resuming. Specifically:
+ force disable/enable the PCIe PHY upon suspend/resume;
+ reprogram the PCIe WAR register when resuming and upon power-on.
* Add a recipe which powers down any PCIe PHY hardware inside the AR5416
(which is the PCI variant) to save on power. I have (currently) no way
to test exactly how much power is saved, if any.
Tested on:
* AR5416 cardbus - although unfortunately pccard/cbb/cardbus currently
detaches the NIC upon suspend, I don't think it's a proper test case.
* AR5418 PCIe attached to expresscard - since we're not doing PCIe APSM,
it's also not likely a full/good test case.
In both instances I went through a handful of suspend/resume cycles and
ensured that the STA vap reassociated correctly.
TODO:
* Setup a laptop to simply sit in a suspend/resume loop, making sure that
the NIC always correctly comes back;
* Start doing suspend/resume tests with actual traffic going on in the
background, as I bet this process is all quite racy at the present;
* Test adhoc/hostap mode, just to be completely sure it's working correctly;
* See if I can jury rig an external power source to an AR5416 to test out
whether ah_disablePCIE() works.
Obtained from: Qualcomm Atheros
* Add some other WAR bits (very usefully described too) in preparation for
porting over some suspend/resume fixes from ath9k/Atheros.
Obtained from: Qualcomm Atheros
not to disable the PCIe PHY in prepration for reset.
Extend the enablepci method to have a "poweroff" flag, which if equal
to true means the hardware is about to go to sleep.
* Flesh out the pcie disable method for 11n chips, as they were defaulting
to the AR5212 (empty) PCIe disable method.
* Add accessor macros for the HAL PCIe enable/disable calls.
* Call disable on ath_suspend()
* Call enable on ath_resume()
NOTE:
* This has nothing to do with the NIC sleep/run state - the NIC still
will stay in network-run state rather than supporting network-sleep
state. This is preparation work for supporting correct suspend/resume
WARs for the 11n PCIe NICs.
TODO:
* It may be feasible at this point to keep the chip powered down during
initial probe/attach and only power it up upon the first configure/reset
pass. This however would require correct (for values of "correct")
tracking of the NIC power configuration state from the driver and that
just isn't attempted at the moment.
Tested:
* AR9280 on my Lenovo T60, but with no suspend/resume pass (yet).
I'll have to leave this high for now, until I've done some significant
surgery with how ath_bufs (and descriptors) are handled.
This should significantly cut down on the opportunities for a full TX
queue hanging traffic. I'll continue making things work though; I'm
mostly doing this for users. :)
I've come across a weird scenario in net80211 where two TX streams will
happily attempt to setup an aggregation session together.
If we're very lucky, it happens concurrently on separate CPUs and the
total lack of locking in the net80211 aggregation code causes this stuff
to race. Badly.
So >1 call would occur to the ath(4) addba start, but only one call would
complete to addba complete or timeout. The TID would thus stay paused.
The real fix is to implement some proper per-node (or maybe per-TID)
locking in net80211, which then could be leveraged by the ath(4) TX
aggregation code.
Whilst I'm at it, shuffle around the debugging messages a bit.
I like to keep people on their toes.
There's some TX path TDMA code in if_ath_tx.c which should be migrated
out, but first I should likely try and verify/fix/repair the TDMA support
in 9.x and -HEAD.
* migrate the rx processing out into if_ath_rx.c
* migrate the TSF functions into if_ath_tsf.h, as inlines
This is in prepration for supporting the EDMA RX routines, required to
support the AR93xx series NICs.
TODO:
* ath_start() shouldn't be private, but it's called as part of
the RX path. I should likely migrate ath_rx_tasklet() back into
if_ath.c and then return this to be 'static'. The RX code really
shouldn't need to see TX routines (and vice versa.)
* ath_beacon_* should be in if_ath_beacon.[ch].
* ath_tdma_* should be in if_ath_tdma.[ch] ...
add some more BAR debugging logic.
* Change the definition of ath_debug and ath_softc.sc_debug from
int to uint64_t;
* Change the relevant sysctls;
* Add a new BAR TX debugging field;
* Use this in if_ath_tx.
This has been tested by using the sysctl program, which happily allows
for fields > 32 bits to be configured.
Although I _should_ handle the other errors in various ways (specifically
errors like FILT), treating them as having transmitted successfully
is completely wrong. Here, they'd be counted as successful and the BAW
would be advanced.. but the RX side wouldn't have received them.
The specific errors I've been seeing here are HAL_TXERR_FILT.
This patch does fix the issue - I've tested it using -i 0.001 pings
(enough to start aggregation) and now the behaviour is correct:
* The RX side never sees a "moved window" error, and
* The TX side sends BARs as needed, with the RX side correctly handling
them.
PR: kern/167902
TX and RX PCU stop/drain routines have been thoroughly debugged.
It's also very likely that I should add hooks back up to the
interface glue (if_ath_pci / if_ath_ahb) to do any relevant
bus flushes that are required. A WMAC DDR flush may be required
for the AR9130 SoC.