a version that i posted earlier on the -current mailing list,
and subsequent feedback received.
The core of the change is just in sys/firmware.h and kern/subr_firmware.c,
while other files are just adaptation of the clients to the ABI change
(const-ification of some parameters and hiding of internal info,
so this is fully compatible at the binary level).
In detail:
- reduce the amount of information exported to clients in struct firmware,
and constify the pointer;
- internally, document and simplify the implementation of the various
functions, and make sure error conditions are dealt with properly.
The diffs are large, but the code is really straightforward now (i hope).
Note also that there is a subtle issue with the implementation of
firmware_register(): currently, as in the previous version, we just
store a reference to the 'imagename' argument, but we should rather
copy it because there is no guarantee that this is a static string.
I realised this while testing this code, but i prefer to fix it in
a later commit -- there is no regression with respect to the past.
Note, too, that the version in RELENG_6 has various bugs including
missing locks around the module release calls, mishandling of modules
loaded by /boot/loader, and so on, so an MFC is absolutely necessary
there. I was just postponing it until this cleanup to avoid doing
things twice.
MFC after: 1 week
front of isp_init so we can read NVRAM even if we're role ISP_NONE.
Prepare for reintroduction of channels (for FC) for N-Port
Virtualization.
Fix a botch in handle assignment that caused us to nuke one device
when a new one arrives and end up with two devices with the same
identity in the virtual target mapping table.
gone device timers and zombie state entries. There are tunables
that can be used to select a number of parameters.
loop_down_limit - how long to wait for loop to come back up before
declaring
all devices dead (default 300 seconds)
gone_device_time- how long to wait for a device that has appeared
to leave the loop or fabric to reappear (default 30 seconds)
Internal tunables include (which should be externalized):
quick_boot_time- how long to wait when booting for loop to come up
change_is_bad- whether or not to accept devices with the same
WWNN/WWPN that reappear at a different PortID as being the 'same'
device.
Keen students of some of the subtle issues here will ask how
one can keep devices from being re-accepted at all (the answer
is to set a gone_device_time to zero- that effectively would
be the same thing).
loading for the QLogic cards.
Because isp(4) exists before the root is mounted, it's not really
possible for us to use the kernel's linker to load modules directly
from disk- that's really too bad.
However, the this is still a net win in in that the firmware has
been split up on a per chip (and in some cases, functionality)
basis, so the amount of stuff loaded *can* be substantially less
than the 1.5MB of firmware images that ispfw now manages. That is,
each specific f/w set is now also built as a module. For example,
QLogic 2322 f/w is built as isp_2322.ko and Initiator/Target 1080
firmware is built as isp_1080_it.ko.
For compatibility purposes (i.e., to perturb folks the least), we
also still build all of the firmware as one ispfw.ko module.
This allows us to let 'ispfw_LOAD' keep on working in existing
loader.conf files. If you now want to strip this down to just
the firmware for your h/w, you can then change loader.conf to
load the f/w you specifically want.
We also still allow for ispfw to be statically built (e.g., for
PAE and sparc64).
Future changes will look at f/w unloading and also role switching
that then uses the kernel linker to load different ips f/w sets.
MFC after: 2 months
Pull in some target mode changes from a private branch.
Pull in some more RELENG_4 compilation changes.
A lot of lines changed, but not much content change yet.
to getting rid u_int for uint and so on).
b) Turn back on 64 bit DAC support. Cheeze it a bit in that we have two
DMA callback functions- one when we have bus_addr_t > 4 bits in width and
the other which should be normal. Even Cheezier in that we turn off setting
up DMA maps to be BUS_SPACE_MAXADDR if we're in ISP_TARGET_MODE. More work
on this in a week or so.
c) Tested under amd64 and 1MB DFLTPHYS, sparc64, i386 (PAE, but insufficient
memory to really test > 4GB). LINT check under amd64.
MFC after: 1 month
are bus_addr_t, not bus_size_t.
In any case, turn off DAC support entirely until it is revamped to actually
work *correctly* for 64 bit platforms (not using a PAE definition and for
both initiator and target mode).
up to date. Principle changes for this reelase is to support 2K Port Login
firmware. This allows us to support the 2322 (and 2422 4Gb) cards which only
come with the 2K Port Login firmware. The 2322 should now work- but we don't
have firmware sets for it in ispfw (as the change to load 2K Port Login f/w
hasn't been made- that f/w is so big it has to be loaded in more than one
chunk).
Other changes are the beginnings of cleaning up some long standing target
mode issues. The next changes here will incorporate a lot of bug fixes
from others.
Finally, some copyright cleanup and attempts to make the parts of the
driver that are FreeBSD specific start conforming more to FreeBSD style.
MFC after: 1 month
Until we can have perfect knowledge that all callers above us think it's okay
for us to sleep, releasing *our* locks of course, we don't dare try and sleep.
flags include INTR_MPSAFE. Put the flags in a common place so that
both isp_sbus && isp_pci DTRT.
In isp_mbxdma setup, drop any locks prior to calling things like
bus_dmatag_create. This gets rid of these obnoxious WITNESS messages
about 'sleeping with locks held' blah blah blah blah blah.
This code does not imply that SBus cards work yet. They hang for me.
But I can't netboot the latest snapshot on my ultra1e, and things
hang at bus_setup_intr time.
Since I'm offline for a while, I thought I'd toss this in in case somebody
else who has a bit better luck wants to fart around with it. Please try
and wait until I get back to check things in.
Oops; I forgot for previous delta... If we're and FC or ULTRA2 or better
card, we can have a 1024 element request queue instead of 256.
MFC after: 1 week
Remove sim queue freezes for resource shortages. I've had too many
strange race conditions where I freeze on a resource shortage but
never get unfrozen.
Consolidate the remaining sim queue freeze condition (for loopdown)
into an inline with debug messages that allows us to track problems
at ISP_LOGDEBUG0 level easier. Change a bunch of debug messages about
loop down/up conditions to ISP_LOGDEBUG0 level.
Remove dead isp_relsim code.
Change some internal flag stuff for efficiency.
Complain vociferously if we try and use our FC scratch area while it's
busy being used already (I mean, if we don't have solaris' ability
to sleep as an interrupt thread which would allow us to just use
a p/v semaphore, at least *say* when you've just borked yourself).
Add infrastructure to allow overrides of hard loopid && initiator
id from boot variables.
Fix the usual quota of silly bugs:
+ 'ktmature' needs to be per-instance. Argh.
+ When entering isp_watchdog, set intsok to zero, preserving
old value to restore later. It's not nice to try and sleep
from splsoftclock.
+ Fix tick overflow buglet in checking timeout value.
MFC after: 1 week
(so we can, when things get lost, find out who currently is processing
on behalf of this open exchange. Invariably, when things are lost and
wedged, it's CAM).
Keep an atio resource counter locally.
MFC after: 1 week
Grumble. I've seen better documented architectures out of Redmond.
Redo fabric evaluation to not use GET ALL NEXT (GA_NXT). Switches seem
to be trying to wriggle out of supporting this well. Instead, use
GID_FT to get a list of Port IDs and then use GPN_ID/GNN_ID to find the
port and node wwn. This should make working on fabrics a bit cleaner and
more stable.
This also caused some cleanup of SNS subcommand canonicalization so that
we can actually check for FS_ACC and FS_RJT, and if we get an FS_RJT,
print out the reason and explanation codes.
We'll keep the old GA_NXT method around if people want to uncomment a
controlling definition in ispvar.h.
This also had us clean up ISPASYNC_FABRICDEV to use a local lportdb argument
and to have the caller explicitly say that a device is at the end of the
fabric list.
MFC after: 1 week
stuff was right, but the busdma stuff was massively not right.
Didn't really test on ia64 or i386- don't have the former h/w and my
FreeBSD-current disk is unwell right now. Hope that this is okay.
MFC after: 1 week
disable MWI on 2300
based on function code, set an 'isp_port' for the 2312- it's a
separate instance, but the NVRAM is shared, and the second port's
NVRAM is at offset 256.
+ Enable RIO operation for LVD SCSI cards. This makes a *big* difference
as even under reasonable load we get batched completions of about 30
commands at a time on, say, an ISP1080.
+ Do 'continuation' mailbox commands- this allows us to specify a work
area within the softc and 'continue' repeated mailbox commands. This is
more or less on an ad hoc basis and is currently only used for firmware
loading (which f/w now loads substantially faster becuase the calling
thread is only woken when all the f/w words are loaded- not for each
one of the 40000 f/w words that gets loaded).
+ If we're about to return from isp_intr with a 'bogus interrupt' indication,
and we're not a 23XX card, check to see whether the semaphore register is
currently *2* (not *1* as it should be) and whether there's an async completion
sitting in outgoing mailbox0. This seems to capture cases of lost fast posting
and RIO interrupts that the 12160 && 1080 have been known to pump out under
extreme load (extreme, as in > 250 active commands).
+ FC_SCRATCH_ACQUIRE/FC_SCRATCH_RELEASE macros.
+ Endian correct swizzle/unswizzle of an ATIO2 that has a WWPN in it.
MFC after: 1 week
the response queue. Instead of the ad hoc ISP_SWIZZLE_REQUEST, we now have
a complete set of inline functions in isp_inline.h. Each platform is
responsible for providing just one of a set of ISP_IOX_{GET,PUT}{8,16,32}
macros.
The reason this needs to be done is that we need to have a single set of
functions that will work correctly on multiple architectures for both little
and big endian machines. It also needs to work correctly in the case that
we have the request or response queues in memory that has to be treated
specially (e.g., have ddi_dma_sync called on it for Solaris after we update
it or before we read from it). It also has to handle the SBus cards (for
platforms that have them) which, while on a Big Endian machine, do *not*
require *most* of the request/response queue entry fields to be swizzled
or unswizzled.
One thing that falls out of this is that we no longer build requests in the
request queue itself. Instead, we build the request locally (e.g., on the
stack) and then as part of the swizzling operation, copy it to the request
queue entry we've allocated. I thought long and hard about whether this was
too expensive a change to make as it in a lot of cases requires an extra
copy. On balance, the flexbility is worth it. With any luck, the entry that
we build locally stays in a processor writeback cache (after all, it's only
64 bytes) so that the cost of actually flushing it to the memory area that is
the shared queue with the PCI device is not all that expensive. We may examine
this again and try to get clever in the future to try and avoid copies.
Another change that falls out of this is that MEMORYBARRIER should be taken
a lot more seriously. The macro ISP_ADD_REQUEST does a MEMORYBARRIER on the
entry being added. But there had been many other places this had been missing.
It's now very important that it be done.
Additional changes:
Fix a longstanding buglet of sorts. When we get an entry via isp_getrqentry,
the iptr value that gets returned is the value we intend to eventually plug
into the ISP registers as the entry *one past* the last one we've written-
*not* the current entry we're updating. All along we've been calling sync
functions on the wrong index value. Argh. The 'fix' here is to rename all
'iptr' variables as 'nxti' to remember that this is the 'next' pointer-
not the current pointer.
Devote a single bit to mboxbsy- and set aside bits for output mbox registers
that we need to pick up- we can have at least one command which does not
have any defined output registers (MBOX_EXECUTE_FIRMWARE).
MFC after: 2 weeks
appropriate cache flush that provides MEMORY_BARRIER in between handoffs
between host && RISC processor for the shared memory request/response
queues.
Submitted by: dfr@nlsystems.com
per-command component that we *don't* try and pass thru CAM. CAM just
is too risky and too much of a pain- structures get copied, but not
all info of interest can be considered safely transported thru all
consumers (including user space) from the incoming ATIO to the outgoing
CTIO- it's just much safer to have a buddy structure, identified by the
command's tag which *does* make it thru safely.
Pay attention to link speed and report 200MB/s xfer speed for a
23XX card in 2GPs mode.
MFC after: 1 week
SIM (as is true for the 1280 and the 12160), then I have to have separate
flags && status for *both* busses. *Whap*.
Implement condition variables for coordination with some target mode
events. It's nice to use these and not panic in obscure little places
in the kernel like 'propagate_priority' just because we went to sleep
holding a mutex, or some other absurd thing.
MFC after: 4 weeks
some reworking (and consequent cleanup) of the interrupt service code.
Also begin to start a cleanup of target mode support that will (eventually)
not require more inforamtion routed with the ATIO to come back with the
CTIO other than tag.
MFC after: 4 weeks
----
Make a device for each ISP- really usable only with devfs and add an ioctl
entry point (this can be used to (re)set debug levels, reset the HBA,
rescan the fabric, issue lips, etc).
----
Add in a kernel thread for Fibre Channel cards. The purpose of this
thread is to be woken up to clean up after Fibre Channel events
block things. Basically, any FC event that casts doubt on the
location or identify of FC devices blocks the queues. When, and
if, we get the PORT DATABASE CHANGED or NAME SERVER DATABASE CHANGED
async event, we activate the kthread which will then, in full thread
context, re-evaluate the local loop and/or the fabric. When it's
satisfied that things are stable, it can then release the blocked
queues and let commands flow again.
The prior mechanism was a lazy evaluation. That is, the next command
to come down the pipe after change events would pay the full price
for re-evaluation. And if this was done off of a softcall, it really
could hang up the system.
These changes brings the FreeBSD port more in line with the Solaris,
Linux and NetBSD ports. It also, more importantly, gets us being
more proactive about topology changes which could then be reflected
upwards to CAM so that the periph driver can be informed sooner
rather than later when things arrive or depart.
---
Add in the (correct) usage of locking macros- we now have lock transition
macros which allow us to transition from holding the CAM lock (Giant)
and grabbing the softc lock and vice versa. Switch over to having this
HBA do real locking. Some folks claim this won't be a win. They're right.
But you have to start somewhere, and this will begin to teach us how
to DTRT for HBAs, etc.
--
Start putting in prototype 2300 support. Add back in LIP
and Loop Reset as async events that each platform will handle.
Add in another int_bogus instrumentation point.
Do some more substantial target mode cleanups.
MFC after: 8 weeks
