file objects calling a user-specified predicate function on each object.
The iteration terminates either when the entire list has been iterated
over or the predicate function returns a non-zero value.
linker_file_foreach() returns the value returned by the last invocation
of the predicate function. It also accepts a void * context pointer that
is passed to the predicate function as well. Using an iterator function
avoids exposing linker internals to the rest of the kernel making locking
simpler.
- Use linker_file_foreach() instead of walking the list of linker files
manually to lookup ndis files in ndis(4).
- Use linker_file_foreach() to implement linker_hwpmc_list_objects().
for a Windows ISR is 'BOOLEAN isrfunc(KINTERRUPT *, void *)' meaning
the ISR get a pointer to the interrupt object and a context pointer,
and returns TRUE if the ISR determines the interrupt was really generated
by the associated device, or FALSE if not.
I had mistakenly used 'void isrfunc(void *)' instead. It happens the
only thing this affects is the internal ndis_intr() ISR in subr_ndis.c,
but it should be fixed just in case we ever need to register a real
Windows ISR vi IoConnectInterrupt().
For NDIS miniports that provide a MiniportISR() method, the 'is_our_intr'
value returned by the method serves as the return value from ndis_isr(),
and 'call_isr' is used to decide whether or not to schedule the interrupt
handler via DPC. For drivers that only supply MiniportEnableInterrupt()
and MiniportDisableInterrupt() methods, call_isr is always TRUE and
is_our_intr is always FALSE.
In the end, there should be no functional changes, except that now
ntoskrnl_intr() can terminate early once it finds the ISR that wants
to service the interrupt.
rather than in ifindex_table[]; all (except one) accesses are
through ifp anyway. IF_LLADDR() works faster, and all (except
one) ifaddr_byindex() users were converted to use ifp->if_addr.
- Stop storing a (pointer to) Ethernet address in "struct arpcom",
and drop the IFP2ENADDR() macro; all users have been converted
to use IF_LLADDR() instead.
for code to start out on one CPU when thunking into Windows
mode in ctxsw_utow(), and then be pre-empted and migrated to another
CPU before thunking back to UNIX mode in ctxsw_wtou(). This is
bad, because then we can end up looking at the wrong 'thread environment
block' when trying to come back to UNIX mode. To avoid this, we now
pin ourselves to the current CPU when thunking into Windows code.
Few other cleanups, since I'm here:
- Get rid of the ndis_isr(), ndis_enable_interrupt() and
ndis_disable_interrupt() wrappers from kern_ndis.c and just invoke
the miniport's methods directly in the interrupt handling routines
in subr_ndis.c. We may as well lose the function call overhead,
since we don't need to export these things outside of ndis.ko
now anyway.
- Remove call to ndis_enable_interrupt() from ndis_init() in if_ndis.c.
We don't need to do it there anyway (the miniport init routine handles
it, if needed).
- Fix the logic in NdisWriteErrorLogEntry() a little.
- Change some NDIS_STATUS_xxx codes in subr_ntoskrnl.c into STATUS_xxx
codes.
- Handle kthread_create() failure correctly in PsCreateSystemThread().
and ndis_halt_nic(). It's been disabled for some time anyway, and
it turns out there's a possible deadlock in NdisMInitializeTimer() when
acquiring the miniport block lock to modify the timer list: it's
possible for a driver to call NdisMInitializeTimer() when the miniport
block lock has already been acquired by an earlier piece of code. You
can't acquire the same spinlock twice, so this can deadlock.
Also, implement MmMapIoSpace() and MmUnmapIoSpace(), and make
NdisMMapIoSpace() and NdisMUnmapIoSpace() use them. There are some
drivers that want MmMapIoSpace() and MmUnmapIoSpace() so that they can
map arbitrary register spaces not directly associated with their
device resources. For example, there's an Atheros driver for
a miniPci card (0x168C:0x1014) on the IBM Thinkpad x40 that wants
to map some I/O spaces at 0xF00000 and 0xE00000 which are held by
the acpi0 device. I don't know what it wants these ranges for,
but if it can't map and access them, the MiniportInitialize() method
fails.
- Change ndis_return() from a DPC to a workitem so that it doesn't
run at DISPATCH_LEVEL (with the dispatcher lock held).
- In if_ndis.c, submit packets to the stack via (*ifp->if_input)() in
a workitem instead of doing it directly in ndis_rxeof(), because
ndis_rxeof() runs in a DPC, and hence at DISPATCH_LEVEL. This
implies that the 'dispatch level' mutex for the current CPU is
being held, and we don't want to call if_input while holding
any locks.
- Reimplement IoConnectInterrupt()/IoDisconnectInterrupt(). The original
approach I used to track down the interrupt resource (by scanning
the device tree starting at the nexus) is prone to problems when
two devices share an interrupt. (E.g removing ndis1 might disable
interrupts for ndis0.) The new approach is to multiplex all the
NDIS interrupts through a common internal dispatcher (ntoskrnl_intr())
and allow IoConnectInterrupt()/IoDisconnectInterrupt() to add or
remove interrupts from the dispatch list.
- Implement KeAcquireInterruptSpinLock() and KeReleaseInterruptSpinLock().
- Change the DPC and workitem threads to use the KeXXXSpinLock
API instead of mtx_lock_spin()/mtx_unlock_spin().
- Simplify the NdisXXXPacket routines by creating an actual
packet pool structure and using the InterlockedSList routines
to manage the packet queue.
- Only honor the value returned by OID_GEN_MAXIMUM_SEND_PACKETS
for serialized drivers. For deserialized drivers, we now create
a packet array of 64 entries. (The Microsoft DDK documentation
says that for deserialized miniports, OID_GEN_MAXIMUM_SEND_PACKETS
is ignored, and the driver for the Marvell 8335 chip, which is
a deserialized miniport, returns 1 when queried.)
- Clean up timer handling in subr_ntoskrnl.
- Add the following conditional debugging code:
NTOSKRNL_DEBUG_TIMERS - add debugging and stats for timers
NDIS_DEBUG_PACKETS - add extra sanity checking for NdisXXXPacket API
NTOSKRNL_DEBUG_SPINLOCKS - add test for spinning too long
- In kern_ndis.c, always start the HAL first and shut it down last,
since Windows spinlocks depend on it. Ntoskrnl should similarly be
started second and shut down next to last.
First and most importantly, I threw out the thread priority-twiddling
implementation of KeRaiseIrql()/KeLowerIrq()/KeGetCurrentIrql() in
favor of a new scheme that uses sleep mutexes. The old scheme was
really very naughty and sought to provide the same behavior as
Windows spinlocks (i.e. blocking pre-emption) but in a way that
wouldn't raise the ire of WITNESS. The new scheme represents
'DISPATCH_LEVEL' as the acquisition of a per-cpu sleep mutex. If
a thread on cpu0 acquires the 'dispatcher mutex,' it will block
any other thread on the same processor that tries to acquire it,
in effect only allowing one thread on the processor to be at
'DISPATCH_LEVEL' at any given time. It can then do the 'atomic sit
and spin' routine on the spinlock variable itself. If a thread on
cpu1 wants to acquire the same spinlock, it acquires the 'dispatcher
mutex' for cpu1 and then it too does an atomic sit and spin to try
acquiring the spinlock.
Unlike real spinlocks, this does not disable pre-emption of all
threads on the CPU, but it does put any threads involved with
the NDISulator to sleep, which is just as good for our purposes.
This means I can now play nice with WITNESS, and I can safely do
things like call malloc() when I'm at 'DISPATCH_LEVEL,' which
you're allowed to do in Windows.
Next, I completely re-wrote most of the event/timer/mutex handling
and wait code. KeWaitForSingleObject() and KeWaitForMultipleObjects()
have been re-written to use condition variables instead of msleep().
This allows us to use the Windows convention whereby thread A can
tell thread B "wake up with a boosted priority." (With msleep(), you
instead have thread B saying "when I get woken up, I'll use this
priority here," and thread A can't tell it to do otherwise.) The
new KeWaitForMultipleObjects() has been better tested and better
duplicates the semantics of its Windows counterpart.
I also overhauled the IoQueueWorkItem() API and underlying code.
Like KeInsertQueueDpc(), IoQueueWorkItem() must insure that the
same work item isn't put on the queue twice. ExQueueWorkItem(),
which in my implementation is built on top of IoQueueWorkItem(),
was also modified to perform a similar test.
I renamed the doubly-linked list macros to give them the same names
as their Windows counterparts and fixed RemoveListTail() and
RemoveListHead() so they properly return the removed item.
I also corrected the list handling code in ntoskrnl_dpc_thread()
and ntoskrnl_workitem_thread(). I realized that the original logic
did not correctly handle the case where a DPC callout tries to
queue up another DPC. It works correctly now.
I implemented IoConnectInterrupt() and IoDisconnectInterrupt() and
modified NdisMRegisterInterrupt() and NdisMDisconnectInterrupt() to
use them. I also tried to duplicate the interrupt handling scheme
used in Windows. The interrupt handling is now internal to ndis.ko,
and the ndis_intr() function has been removed from if_ndis.c. (In
the USB case, interrupt handling isn't needed in if_ndis.c anyway.)
NdisMSleep() has been rewritten to use a KeWaitForSingleObject()
and a KeTimer, which is how it works in Windows. (This is mainly
to insure that the NDISulator uses the KeTimer API so I can spot
any problems with it that may arise.)
KeCancelTimer() has been changed so that it only cancels timers, and
does not attempt to cancel a DPC if the timer managed to fire and
queue one up before KeCancelTimer() was called. The Windows DDK
documentation seems to imply that KeCantelTimer() will also call
KeRemoveQueueDpc() if necessary, but it really doesn't.
The KeTimer implementation has been rewritten to use the callout API
directly instead of timeout()/untimeout(). I still cheat a little in
that I have to manage my own small callout timer wheel, but the timer
code works more smoothly now. I discovered a race condition using
timeout()/untimeout() with periodic timers where untimeout() fails
to actually cancel a timer. I don't quite understand where the race
is, using callout_init()/callout_reset()/callout_stop() directly
seems to fix it.
I also discovered and fixed a bug in winx32_wrap.S related to
translating _stdcall calls. There are a couple of routines
(i.e. the 64-bit arithmetic intrinsics in subr_ntoskrnl) that
return 64-bit quantities. On the x86 arch, 64-bit values are
returned in the %eax and %edx registers. However, it happens
that the ctxsw_utow() routine uses %edx as a scratch register,
and x86_stdcall_wrap() and x86_stdcall_call() were only preserving
%eax before branching to ctxsw_utow(). This means %edx was getting
clobbered in some cases. Curiously, the most noticeable effect of this
bug is that the driver for the TI AXC110 chipset would constantly drop
and reacquire its link for no apparent reason. Both %eax and %edx
are preserved on the stack now. The _fastcall and _regparm
wrappers already handled everything correctly.
I changed if_ndis to use IoAllocateWorkItem() and IoQueueWorkItem()
instead of the NdisScheduleWorkItem() API. This is to avoid possible
deadlocks with any drivers that use NdisScheduleWorkItem() themselves.
The unicode/ansi conversion handling code has been cleaned up. The
internal routines have been moved to subr_ntoskrnl and the
RtlXXX routines have been exported so that subr_ndis can call them.
This removes the incestuous relationship between the two modules
regarding this code and fixes the implementation so that it honors
the 'maxlen' fields correctly. (Previously it was possible for
NdisUnicodeStringToAnsiString() to possibly clobber memory it didn't
own, which was causing many mysterious crashes in the Marvell 8335
driver.)
The registry handling code (NdisOpen/Close/ReadConfiguration()) has
been fixed to allocate memory for all the parameters it hands out to
callers and delete whem when NdisCloseConfiguration() is called.
(Previously, it would secretly use a single static buffer.)
I also substantially updated if_ndis so that the source can now be
built on FreeBSD 7, 6 and 5 without any changes. On FreeBSD 5, only
WEP support is enabled. On FreeBSD 6 and 7, WPA-PSK support is enabled.
The original WPA code has been updated to fit in more cleanly with
the net80211 API, and to eleminate the use of magic numbers. The
ndis_80211_setstate() routine now sets a default authmode of OPEN
and initializes the RTS threshold and fragmentation threshold.
The WPA routines were changed so that the authentication mode is
always set first, followed by the cipher. Some drivers depend on
the operations being performed in this order.
I also added passthrough ioctls that allow application code to
directly call the MiniportSetInformation()/MiniportQueryInformation()
methods via ndis_set_info() and ndis_get_info(). The ndis_linksts()
routine also caches the last 4 events signalled by the driver via
NdisMIndicateStatus(), and they can be queried by an application via
a separate ioctl. This is done to allow wpa_supplicant to directly
program the various crypto and key management options in the driver,
allowing things like WPA2 support to work.
Whew.
struct ifnet or the layer 2 common structure it was embedded in have
been replaced with a struct ifnet pointer to be filled by a call to the
new function, if_alloc(). The layer 2 common structure is also allocated
via if_alloc() based on the interface type. It is hung off the new
struct ifnet member, if_l2com.
This change removes the size of these structures from the kernel ABI and
will allow us to better manage them as interfaces come and go.
Other changes of note:
- Struct arpcom is no longer referenced in normal interface code.
Instead the Ethernet address is accessed via the IFP2ENADDR() macro.
To enforce this ac_enaddr has been renamed to _ac_enaddr.
- The second argument to ether_ifattach is now always the mac address
from driver private storage rather than sometimes being ac_enaddr.
Reviewed by: sobomax, sam
works again.
This driver uses NdisScheduleWorkItem(), and we have to take special steps
to insure that its workitems don't collide with any of the other workitems
used by the NDISulator. In particular, if one of the driver's work jobs
blocks, it can prevent NdisMAllocateSharedMemoryAsync() from completing
when expected.
The original hack to fix this was to have NdisMAllocateSharedMemoryAsync()
defer its work to the DPC queue instead of the general task queue. To
fix it now, I decided to add some additional workitem threads. (There's
supposed to be a pool of worker threads in Windows anyway.) Currently,
there are 4. There should be at least 2. One is reserved for the legacy
ExQueueWorkItem() API, while the others are used in round-robin by the
IoQueueWorkItem() API. NdisMAllocateSharedMemoryAsync() uses the latter
API while NdisScheduleWorkItem() uses the former, so the deadlock is
avoided.
Fixed NdisMRegisterDevice()/NdisMDeregisterDevice() to work a little
more sensibly with the new driver_object/device_object framework. It
doesn't really register a working user-mode interface, but the existing
code was completely wrong for the new framework.
Fixed a couple of bugs dealing with the cancellation of events and
DPCs. When cancelling an event that's still on the timer queue (i.e.
hasn't expired yet), reset dh_inserted in its dispatch header to FALSE.
Previously, it was left set to TRUE, which would make a cancelled
timer appear to have not been cancelled. Also, when removing a DPC
from a queue, reset its list pointers, otherwise a cancelled DPC
might mistakenly be treated as still pending.
Lastly, fix the behavior of ntoskrnl_wakeup() when dealing with
objects that have nobody waiting on them: sync event objects get
their signalled state reset to FALSE, but notification objects
should still be set to TRUE.
technically a no-op since uintmax_t is uint64_t on all currently
supported architectures, but we should use an explicit cast instead
of depending on this obscure coincidence.
Oh, one additional change I forgot to mention in the last commit:
NdisOpenFile() was broken in the case for firmware files that were
pre-loaded as modules. When searching for the module in NdisOpenFile(),
we would match against a symbol name, which would contain the string
we were looking for, then save a pointer to the linker file handle.
Later, in NdisMapFile(), we would refer to the filename hung off
this handle when trying to find the starting address symbol. Only
problem is, this filename is different from the embedded symbol
name we're searching for, so the mapping would fail. I found this
problem while testing the AirGo driver, which requires a small
firmware file.
- Remove the old task threads from kern_ndis.c and reimplement them in
subr_ntoskrnl.c, in order to more properly emulate the Windows DPC
API. Each CPU gets its own DPC queue/thread, and each queue can
have low, medium and high importance DPCs. New APIs implemented:
KeSetTargetProcessorDpc(), KeSetImportanceDpc() and KeFlushQueuedDpcs().
(This is the biggest change.)
- Fix a bug in NdisMInitializeTimer(): the k_dpc pointer in the
nmt_timer embedded in the ndis_miniport_timer struct must be set
to point to the DPC, also embedded in the struct. Failing to do
this breaks dequeueing of DPCs submitted via timers, and in turn
breaks cancelling timers.
- Fix a bug in KeCancelTimer(): if the timer is interted in the timer
queue (i.e. the timeout callback is still pending), we have to both
untimeout() the timer _and_ call KeRemoveQueueDpc() to nuke the DPC
that might be pending. Failing to do this breaks cancellation of
periodic timers, which always appear to be inserted in the timer queue.
- Make use of the nmt_nexttimer field in ndis_miniport_timer: keep a
queue of pending timers and cancel them all in ndis_halt_nic(), prior
to calling MiniportHalt(). Also call KeFlushQueuedDpcs() to make sure
any DPCs queued by the timers have expired.
- Modify NdisMAllocateSharedMemory() and NdisMFreeSharedMemory() to keep
track of both the virtual and physical addresses of the shared memory
buffers that get handed out. The AirGo MIMO driver appears to have a bug
in it: for one of the segments is allocates, it returns the wrong
virtual address. This would confuse NdisMFreeSharedMemory() and cause
a crash. Why it doesn't crash Windows too I have no idea (from reading
the documentation for NdisMFreeSharedMemory(), it appears to be a violation
of the API).
- Implement strstr(), strchr() and MmIsAddressValid().
- Implement IoAllocateWorkItem(), IoFreeWorkItem(), IoQueueWorkItem() and
ExQueueWorkItem(). (This is the second biggest change.)
- Make NdisScheduleWorkItem() call ExQueueWorkItem(). (Note that the
ExQueueWorkItem() API is deprecated by Microsoft, but NDIS still uses
it, since NdisScheduleWorkItem() is incompatible with the IoXXXWorkItem()
API.)
- Change if_ndis.c to use the NdisScheduleWorkItem() interface for scheduling
tasks.
With all these changes and fixes, the AirGo MIMO driver for the Belkin
F5D8010 Pre-N card now works. Special thanks to Paul Robinson
(paul dawt robinson at pwermedia dawt net) for the loan of a card
for testing.
here on in, if_ndis.ko will be pre-built as a module, and can be built
into a static kernel (though it's not part of GENERIC). Drivers are
created using the new ndisgen(8) script, which uses ndiscvt(8) under
the covers, along with a few other tools. The result is a driver module
that can be kldloaded into the kernel.
A driver with foo.inf and foo.sys files will be converted into
foo_sys.ko (and foo_sys.o, for those who want/need to make static
kernels). This module contains all of the necessary info from the
.INF file and the driver binary image, converted into an ELF module.
You can kldload this module (or add it to /boot/loader.conf) to have
it loaded automatically. Any required firmware files can be bundled
into the module as well (or converted/loaded separately).
Also, add a workaround for a problem in NdisMSleep(). During system
bootstrap (cold == 1), msleep() always returns 0 without actually
sleeping. The Intel 2200BG driver uses NdisMSleep() to wait for
the NIC's firmware to come to life, and fails to load if NdisMSleep()
doesn't actually delay. As a workaround, if msleep() (and hence
ndis_thsuspend()) returns 0, use a hard DELAY() to sleep instead).
This is not really the right thing to do, but we can't really do much
else. At the very least, this makes the Intel driver happy.
There are probably other drivers that fail in this way during bootstrap.
Unfortunately, the only workaround for those is to avoid pre-loading
them and kldload them once the system is running instead.
layer, but with a twist.
The twist has to do with the fact that Microsoft supports structured
exception handling in kernel mode. On the i386 arch, exception handling
is implemented by hanging an exception registration list off the
Thread Environment Block (TEB), and the TEB is accessed via the %fs
register. The problem is, we use %fs as a pointer to the pcpu stucture,
which means any driver that tries to write through %fs:0 will overwrite
the curthread pointer and make a serious mess of things.
To get around this, Project Evil now creates a special entry in
the GDT on each processor. When we call into Windows code, a context
switch routine will fix up %fs so it points to our new descriptor,
which in turn points to a fake TEB. When the Windows code returns,
or calls out to an external routine, we swap %fs back again. Currently,
Project Evil makes use of GDT slot 7, which is all 0s by default.
I fully expect someone to jump up and say I can't do that, but I
couldn't find any code that makes use of this entry anywhere. Sadly,
this was the only method I could come up with that worked on both
UP and SMP. (Modifying the LDT works on UP, but becomes incredibly
complicated on SMP.) If necessary, the context switching stuff can
be yanked out while preserving the convention calling wrappers.
(Fortunately, it looks like Microsoft uses some special epilog/prolog
code on amd64 to implement exception handling, so the same nastiness
won't be necessary on that arch.)
The advantages are:
- Any driver that uses %fs as though it were a TEB pointer won't
clobber pcpu.
- All the __stdcall/__fastcall/__regparm stuff that's specific to
gcc goes away.
Also, while I'm here, switch NdisGetSystemUpTime() back to using
nanouptime() again. It turns out nanouptime() is way more accurate
than just using ticks(). On slower machines, the Atheros drivers
I tested seem to take a long time to associate due to the loss
in accuracy.
ndis_timercall() in NdisMInitializeTimer(), we can't use the raw
function pointer. This is because ntoskrnl_run_dpc() expects to
invoke a function with Microsoft calling conventions. On i386,
this works because ndis_timercall() is declared with the __stdcall
attribute, but this is a no-op on amd64. To do it correctly, we
have to generate a wrapper for ndis_timercall() and us the wrapper
instead of of the raw function pointer.
Fix this by adding ndis_timercall() to the funcptr table in subr_ndis.c,
and create ndis_findwrap() to extract the wrapped function from the
table in NdisMInitializeTimer() instead of just passing ndis_timercall()
to KeInitializeDpc() directly.
ExAllocatePoolWithTag(), not malloc(), so it should be released
with ExFreePool(), not free(). Fix a couple if instances of
free(fh, ...) that got overlooked.
work on SMP" saga. After several weeks and much gnashing of teeth,
I have finally tracked down all the problems, despite their best
efforts to confound and annoy me.
Problem nunmber one: the Atheros windows driver is _NOT_ a de-serialized
miniport! It used to be that NDIS drivers relied on the NDIS library
itself for all their locking and serialization needs. Transmit packet
queues were all handled internally by NDIS, and all calls to
MiniportXXX() routines were guaranteed to be appropriately serialized.
This proved to be a performance problem however, and Microsoft
introduced de-serialized miniports with the NDIS 5.x spec. Microsoft
still supports serialized miniports, but recommends that all new drivers
written for Windows XP and later be deserialized. Apparently Atheros
wasn't listening when they said this.
This means (among other things) that we have to serialize calls to
MiniportSendPackets(). We also have to serialize calls to MiniportTimer()
that are triggered via the NdisMInitializeTimer() routine. It finally
dawned on me why NdisMInitializeTimer() takes a special
NDIS_MINIPORT_TIMER structure and a pointer to the miniport block:
the timer callback must be serialized, and it's only by saving the
miniport block handle that we can get access to the serialization
lock during the timer callback.
Problem number two: haunted hardware. The thing that was _really_
driving me absolutely bonkers for the longest time is that, for some
reason I couldn't understand, my test machine would occasionally freeze
or more frustratingly, reset completely. That's reset and in *pow!*
back to the BIOS startup. No panic, no crashdump, just a reset. This
appeared to happen most often when MiniportReset() was called. (As
to why MiniportReset() was being called, see problem three below.)
I thought maybe I had created some sort of horrible deadlock
condition in the process of adding the serialization, but after three
weeks, at least 6 different locking implementations and heroic efforts
to debug the spinlock code, the machine still kept resetting. Finally,
I started single stepping through the MiniportReset() routine in
the driver using the kernel debugger, and this ultimately led me to
the source of the problem.
One of the last things the Atheros MiniportReset() routine does is
call NdisReadPciSlotInformation() several times to inspect a portion
of the device's PCI config space. It reads the same chunk of config
space repeatedly, in rapid succession. Presumeably, it's polling
the hardware for some sort of event. The reset occurs partway through
this process. I discovered that when I single-stepped through this
portion of the routine, the reset didn't occur. So I inserted a 1
microsecond delay into the read loop in NdisReadPciSlotInformation().
Suddenly, the reset was gone!!
I'm still very puzzled by the whole thing. What I suspect is happening
is that reading the PCI config space so quickly is causing a severe
PCI bus error. My test system is a Sun w2100z dual Opteron system,
and the NIC is a miniPCI card mounted in a miniPCI-to-PCI carrier card,
plugged into a 100Mhz PCI slot. It's possible that this combination of
hardware causes a bus protocol violation in this scenario which leads
to a fatal machine check. This is pure speculation though. Really all I
know for sure is that inserting the delay makes the problem go away.
(To quote Homer Simpson: "I don't know how it works, but fire makes
it good!")
Problem number three: NdisAllocatePacket() needs to make sure to
initialize the npp_validcounts field in the 'private' section of
the NDIS_PACKET structure. The reason if_ndis was calling the
MiniportReset() routine in the first place is that packet transmits
were sometimes hanging. When sending a packet, an NDIS driver will
call NdisQueryPacket() to learn how many physical buffers the packet
resides in. NdisQueryPacket() is actually a macro, which traverses
the NDIS_BUFFER list attached to the NDIS_PACKET and stashes some
of the results in the 'private' section of the NDIS_PACKET. It also
sets the npp_validcounts field to TRUE To indicate that the results are
now valid. The problem is, now that if_ndis creates a pool of transmit
packets via NdisAllocatePacketPool(), it's important that each time
a new packet is allocated via NdisAllocatePacket() that validcounts
be initialized to FALSE. If it isn't, and a previously transmitted
NDIS_PACKET is pulled out of the pool, it may contain stale data
from a previous transmission which won't get updated by NdisQueryPacket().
This would cause the driver to miscompute the number of fragments
for a given packet, and botch the transmission.
Fixing these three problems seems to make the Atheros driver happy
on SMP, which hopefully means other serialized miniports will be
happy too.
And there was much rejoicing.
Other stuff fixed along the way:
- Modified ndis_thsuspend() to take a mutex as an argument. This
allows KeWaitForSingleObject() and KeWaitForMultipleObjects() to
avoid any possible race conditions with other routines that
use the dispatcher lock.
- Fixed KeCancelTimer() so that it returns the correct value for
'pending' according to the Microsoft documentation
- Modfied NdisGetSystemUpTime() to use ticks and hz rather than
calling nanouptime(). Also added comment that this routine wraps
after 49.7 days.
- Added macros for KeAcquireSpinLock()/KeReleaseSpinLock() to hide
all the MSCALL() goop.
- For x86, KeAcquireSpinLockRaiseToDpc() needs to be a separate
function. This is because it's supposed to be _stdcall on the x86
arch, whereas KeAcquireSpinLock() is supposed to be _fastcall.
On amd64, all routines use the same calling convention so we can
just map KeAcquireSpinLockRaiseToDpc() directly to KfAcquireSpinLock()
and it will work. (The _fastcall attribute is a no-op on amd64.)
- Implement and use IoInitializeDpcRequest() and IoRequestDpc() (they're
just macros) and use them for interrupt handling. This allows us to
move the ndis_intrtask() routine from if_ndis.c to kern_ndis.c.
- Fix the MmInitializeMdl() macro so that is uses sizeof(vm_offset_t)
when computing mdl_size instead of uint32_t, so that it matches the
MmSizeOfMdl() routine.
- Change a could of M_WAITOKs to M_NOWAITs in the unicode routines in
subr_ndis.c.
- Use the dispatcher lock a little more consistently in subr_ntoskrnl.c.
- Get rid of the "wait for link event" hack in ndis_init(). Now that
I fixed NdisReadPciSlotInformation(), it seems I don't need it anymore.
This should fix the witness panic a couple of people have reported.
- Use MSCALL1() when calling the MiniportHangCheck() function in
ndis_ticktask(). I accidentally missed this one when adding the
wrapping for amd64.
both consuming 1K of stack space. This is unfriendly. Allocate the buffers
off the heap instead. It's a little slower, but these aren't performance
critical routines.
Also, add a spinlock to NdisAllocatePacketPool(), NdisAllocatePacket(),
NdisFreePacketPool() and NdisFreePacket(). The pool is maintained as a
linked list. I don't know for a fact that it can be corrupted, but why
take chances.
Ville-Pertti Keinonen (will at exomi dot comohmygodnospampleasekthx)
deserves a big thanks for submitting initial patches to make it
work. I have mangled his contributions appropriately.
The main gotcha with Windows/x86-64 is that Microsoft uses a different
calling convention than everyone else. The standard ABI requires using
6 registers for argument passing, with other arguments on the stack.
Microsoft uses only 4 registers, and requires the caller to leave room
on the stack for the register arguments incase the callee needs to
spill them. Unlike x86, where Microsoft uses a mix of _cdecl, _stdcall
and _fastcall, all routines on Windows/x86-64 uses the same convention.
This unfortunately means that all the functions we export to the
driver require an intermediate translation wrapper. Similarly, we have
to wrap all calls back into the driver binary itself.
The original patches provided macros to wrap every single routine at
compile time, providing a secondary jump table with a customized
wrapper for each exported routine. I decided to use a different approach:
the call wrapper for each function is created from a template at
runtime, and the routine to jump to is patched into the wrapper as
it is created. The subr_pe module has been modified to patch in the
wrapped function instead of the original. (On x86, the wrapping
routine is a no-op.)
There are some minor API differences that had to be accounted for:
- KeAcquireSpinLock() is a real function on amd64, not a macro wrapper
around KfAcquireSpinLock()
- NdisFreeBuffer() is actually IoFreeMdl(). I had to change the whole
NDIS_BUFFER API a bit to accomodate this.
Bugs fixed along the way:
- IoAllocateMdl() always returned NULL
- kern_windrv.c:windrv_unload() wasn't releasing private driver object
extensions correctly (found thanks to memguard)
This has only been tested with the driver for the Broadcom 802.11g
chipset, which was the only Windows/x86-64 driver I could find.
Windows DRIVER_OBJECT and DEVICE_OBJECT mechanism so that we can
simulate driver stacking.
In Windows, each loaded driver image is attached to a DRIVER_OBJECT
structure. Windows uses the registry to match up a given vendor/device
ID combination with a corresponding DRIVER_OBJECT. When a driver image
is first loaded, its DriverEntry() routine is invoked, which sets up
the AddDevice() function pointer in the DRIVER_OBJECT and creates
a dispatch table (based on IRP major codes). When a Windows bus driver
detects a new device, it creates a Physical Device Object (PDO) for
it. This is a DEVICE_OBJECT structure, with semantics analagous to
that of a device_t in FreeBSD. The Windows PNP manager will invoke
the driver's AddDevice() function and pass it pointers to the DRIVER_OBJECT
and the PDO.
The AddDevice() function then creates a new DRIVER_OBJECT structure of
its own. This is known as the Functional Device Object (FDO) and
corresponds roughly to a private softc instance. The driver uses
IoAttachDeviceToDeviceStack() to add this device object to the
driver stack for this PDO. Subsequent drivers (called filter drivers
in Windows-speak) can be loaded which add themselves to the stack.
When someone issues an IRP to a device, it travel along the stack
passing through several possible filter drivers until it reaches
the functional driver (which actually knows how to talk to the hardware)
at which point it will be completed. This is how Windows achieves
driver layering.
Project Evil now simulates most of this. if_ndis now has a modevent
handler which will use MOD_LOAD and MOD_UNLOAD events to drive the
creation and destruction of DRIVER_OBJECTs. (The load event also
does the relocation/dynalinking of the image.) We don't have a registry,
so the DRIVER_OBJECTS are stored in a linked list for now. Eventually,
the list entry will contain the vendor/device ID list extracted from
the .INF file. When ndis_probe() is called and detectes a supported
device, it will create a PDO for the device instance and attach it
to the DRIVER_OBJECT just as in Windows. ndis_attach() will then call
our NdisAddDevice() handler to create the FDO. The NDIS miniport block
is now a device extension hung off the FDO, just as it is in Windows.
The miniport characteristics table is now an extension hung off the
DRIVER_OBJECT as well (the characteristics are the same for all devices
handled by a given driver, so they don't need to be per-instance.)
We also do an IoAttachDeviceToDeviceStack() to put the FDO on the
stack for the PDO. There are a couple of fake bus drivers created
for the PCI and pccard buses. Eventually, there will be one for USB,
which will actually accept USB IRP.s
Things should still work just as before, only now we do things in
the proper order and maintain the correct framework to support passing
IRPs between drivers.
Various changes:
- corrected the comments about IRQL handling in subr_hal.c to more
accurately reflect reality
- update ndiscvt to make the drv_data symbol in ndis_driver_data.h a
global so that if_ndis_pci.o and/or if_ndis_pccard.o can see it.
- Obtain the softc pointer from the miniport block by referencing
the PDO rather than a private pointer of our own (nmb_ifp is no
longer used)
- implement IoAttachDeviceToDeviceStack(), IoDetachDevice(),
IoGetAttachedDevice(), IoAllocateDriverObjectExtension(),
IoGetDriverObjectExtension(), IoCreateDevice(), IoDeleteDevice(),
IoAllocateIrp(), IoReuseIrp(), IoMakeAssociatedIrp(), IoFreeIrp(),
IoInitializeIrp()
- fix a few mistakes in the driver_object and device_object definitions
- add a new module, kern_windrv.c, to handle the driver registration
and relocation/dynalinkign duties (which don't really belong in
kern_ndis.c).
- made ndis_block and ndis_chars in the ndis_softc stucture pointers
and modified all references to it
- fixed NdisMRegisterMiniport() and NdisInitializeWrapper() so they
work correctly with the new driver_object mechanism
- changed ndis_attach() to call NdisAddDevice() instead of ndis_load_driver()
(which is now deprecated)
- used ExAllocatePoolWithTag()/ExFreePool() in lookaside list routines
instead of kludged up alloc/free routines
- added kern_windrv.c to sys/modules/ndis/Makefile and files.i386.
attributes in casts (i.e. foo = (__stdcall sometype)bar). This only
happens in two places where we need to set up function pointers, so
work around the problem with some void pointer magic.
USB device support):
- Convert all of my locally chosen function names to their actual
Windows equivalents, where applicable. This is a big no-op change
since it doesn't affect functionality, but it helps avoid a bit
of confusion (it's now a lot easier to see which functions are
emulated Windows API routines and which are just locally defined).
- Turn ndis_buffer into an mdl, like it should have been. The structure
is the same, but now it belongs to the subr_ntoskrnl module.
- Implement a bunch of MDL handling macros from Windows and use them where
applicable.
- Correct the implementation of IoFreeMdl().
- Properly implement IoAllocateMdl() and MmBuildMdlForNonPagedPool().
- Add the definitions for struct irp and struct driver_object.
- Add IMPORT_FUNC() and IMPORT_FUNC_MAP() macros to make formatting
the module function tables a little cleaner. (Should also help
with AMD64 support later on.)
- Fix if_ndis.c to use KeRaiseIrql() and KeLowerIrql() instead of
the previous calls to hal_raise_irql() and hal_lower_irql() which
have been renamed.
The function renaming generated a lot of churn here, but there should
be very little operational effect.
calls MiniportQueryInformation(), it will return NDIS_STATUS_PENDING.
When this happens, ndis_get_info() will sleep waiting for a completion
event. If two threads call ndis_get_info() and both end up having to
sleep, they will both end up waiting on the same wait channel, which
can cause a panic in sleepq_add() if INVARIANTS are turned on.
Fix this by having ndis_get_info() use a common mutex rather than
using the process mutex with PROC_LOCK(). Also do the same for
ndis_set_info(). Note that Pierre's original patch also made ndis_thsuspend()
use the new mutex, but ndis_thsuspend() shouldn't need this since
it will make each thread that calls it sleep on a unique wait channel.
Also, it occured to me that we probably don't want to enter
MiniportQueryInformation() or MiniportSetInformation() from more
than one thread at any given time, so now we acquire a Windows
spinlock before calling either of them. The Microsoft documentation
says that MiniportQueryInformation() and MiniportSetInformation()
are called at DISPATCH_LEVEL, and previously we would call
KeRaiseIrql() to set the IRQL to DISPATCH_LEVEL before entering
either routine, but this only guarantees mutual exclusion on
uniprocessor machines. To make it SMP safe, we need to use a real
spinlock. For now, I'm abusing the spinlock embedded in the
NDIS_MINIPORT_BLOCK structure for this purpose. (This may need to be
applied to some of the other routines in kern_ndis.c at a later date.)
Export ntoskrnl_init_lock() (KeInitializeSpinlock()) from subr_ntoskrnl.c
since we need to use in in kern_ndis.c, and since it's technically part
of the Windows kernel DDK API along with the other spinlock routines. Use
it in subr_ndis.c too rather than frobbing the spinlock directly.
These normally only manifest if the ndis compat module is statically
compiled into a kernel image by way of 'options NDISAPI'.
Submitted by: Dmitri Nikulin
Approved by: wpaul
PR: kern/71449
MFC after: 1 week
This was tested with a Netgear WG311v2 802.11b/g PCI card. Things
that were fixed:
- This chip has two memory mapped regions, one at PCIR_BAR(0) and the
other at PCIR_BAR(1). This is a little different from the other
chips I've seen with two PCI shared memory regions, since they tend
to have the second BAR ad PCIR_BAR(2). if_ndis_pci.c tests explicitly
for PCIR_BAR(2). This has been changed to simply fill in ndis_res_mem
first and ndis_res_altmem second, if a second shared memory range
exists. Given that NDIS drivers seem to scan for BARs in ascending
order, I think this should be ok.
- Fixed the code that tries to process firmware images that have been
loaded as .ko files. To save a step, I was setting up the address
mapping in ndis_open_file(), but ndis_map_file() flags pre-existing
mappings as an error (to avoid duplicate mappings). Changed this so
that the mapping is now donw in ndis_map_file() as expected.
- Made the typedef for 'driver_entry' explicitly include __stdcall
to silence gcc warning in ndis_load_driver().
NOTE: the Texas Instruments ACX111 driver needs firmware. With my
card, there were 3 .bin files shipped with the driver. You must
either put these files in /compat/ndis or convert them with
ndiscvt -f and kldload them so the driver can use them. Without
the firmware image, the NIC won't work.
- In ntoskrnl_var.h, I had defined compat macros for
ntoskrnl_acquire_spinlock() and ntoskrnl_release_spinlock() but
never used them. This is fortunate since they were stale. Fix them
to work properly. (In Windows/x86 KeAcquireSpinLock() is a macro that
calls KefAcquireSpinLock(), which lives in HAL.dll. To imitate this,
ntoskrnl_acquire_spinlock() is just a macro that calls hal_lock(),
which lives in subr_hal.o.)
- Add macros for ntoskrnl_raise_irql() and ntoskrnl_lower_irql() that
call hal_raise_irql() and hal_lower_irql().
- Use these macros in kern_ndis.c, subr_ndis.c and subr_ntoskrnl.c.
- Along the way, I realised subr_ndis.c:ndis_lock() was not calling
hal_lock() correctly (it was using the FASTCALL2() wrapper when
in reality this routine is FASTCALL1()). Using the
ntoskrnl_acquire_spinlock() fixes this. Not sure if this actually
caused any bugs since hal_lock() would have just ignored what
was in %edx, but it was still bogus.
This hides many of the uses of the FASTCALLx() macros which makes the
code a little cleaner. Should not have any effect on generated object
code, other than the one fix in ndis_lock().
- Give ndiscvt(8) the ability to process a .SYS file directly into
a .o file so that we don't have to emit big messy char arrays into
the ndis_driver_data.h file. This behavior is currently optional, but
may become the default some day.
- Give ndiscvt(8) the ability to turn arbitrary files into .ko files
so that they can be pre-loaded or kldloaded. (Both this and the
previous change involve using objcopy(1)).
- Give NdisOpenFile() the ability to 'read' files out of kernel memory
that have been kldloaded or pre-loaded, and disallow the use of
the normal vn_open() file opening method during bootstrap (when no
filesystems have been mounted yet). Some people have reported that
kldloading if_ndis.ko works fine when the system is running multiuser
but causes a panic when the modile is pre-loaded by /boot/loader. This
happens with drivers that need to use NdisOpenFile() to access
external files (i.e. firmware images). NdisOpenFile() won't work
during kernel bootstrapping because no filesystems have been mounted.
To get around this, you can now do the following:
o Say you have a firmware file called firmware.img
o Do: ndiscvt -f firmware.img -- this creates firmware.img.ko
o Put the firmware.img.ko in /boot/kernel
o add firmware.img_load="YES" in /boot/loader.conf
o add if_ndis_load="YES" and ndis_load="YES" as well
Now the loader will suck the additional file into memory as a .ko. The
phony .ko has two symbols in it: filename_start and filename_end, which
are generated by objcopy(1). ndis_open_file() will traverse each module
in the module list looking for these symbols and, if it finds them, it'll
use them to generate the file mapping address and length values that
the caller of NdisOpenFile() wants.
As a bonus, this will even work if the file has been statically linked
into the kernel itself, since the "kernel" module is searched too.
(ndiscvt(8) will generate both filename.o and filename.ko for you).
- Modify the mechanism used to provide make-pretend FASTCALL support.
Rather than using inline assembly to yank the first two arguments
out of %ecx and %edx, we now use the __regparm__(3) attribute (and
the __stdcall__ attribute) and use some macro magic to re-order
the arguments and provide dummy arguments as needed so that the
arguments passed in registers end up in the right place. Change
taken from DragonflyBSD version of the NDISulator.
actually work.
Make the PCI and PCCARD attachments provide a bus_get_resource_list()
method so that resource listing for PCCARD works. PCCARD does not
have a bus_get_resource_list() method (yet), so I faked up the
resource list management in if_ndis_pccard.c, and added
bus_get_resource_list() methods to both if_ndis_pccard.c and if_ndis_pci.c.
The one in the PCI attechment just hands off to the PCI bus code.
The difference is transparent to the NDIS resource handler code.
Fixed ndis_open_file() so that opening files which live on NFS
filesystems work: pass an actual ucred structure to VOP_GETATTR()
(NFS explodes if the ucred structure is NOCRED).
Make NdisMMapIoSpace() handle mapping of PCMCIA attribute memory
resources correctly.
Turn subr_ndis.c:my_strcasecmp() into ndis_strcasecmp() and export
it so that if_ndis_pccard.c can use it, and junk the other copy
of my_strcasecmp() from if_ndis_pccard.c.
- In subr_ndis.c:ndis_allocate_sharemem(), create the busdma tags
used for shared memory allocations with a lowaddr of 0x3E7FFFFF.
This forces the buffers to be mapped to physical/bus addresses within
the first 1GB of physical memory. It seems that at least one card
(Linksys Instant Wireless PCI V2.7) depends on this behavior. I
don't know if this is a hardware restriction, or if the NDIS
driver for this card is truncating the addresses itself, but using
physical/bus addresses beyong the 1GB limit causes initialization
failures.
- Create am NDIS_INITIALIZED() macro in if_ndisvar.h and use it in
if_ndis.c to test whether the device has been initialized rather
than checking for the presence of the IFF_UP flag in if_flags.
While debugging the previous problem, I noticed that bringing
up the device would always produce failures from ndis_setmulti().
It turns out that the following steps now occur during device
initialization:
- IFF_UP flag is set in if_flags
- ifp->if_ioctl() called with SIOCSIFADDR (which we don't handle)
- ifp->if_ioctl() called with SIOCADDMULTI
- ifp->if_ioctl() called with SIOCADDMULTI (again)
- ifp->if_ioctl() called with SIOCADDMULTI (yet again)
- ifp->if_ioctl() called with SIOCSIFFLAGS
Setting the receive filter and multicast filters can only be done
when the underlying NDIS driver has been initialized, which is done
by ifp->if_init(). However, we don't call ifp->if_init() until
ifp->if_ioctl() is called with SIOCSIFFLAGS and IFF_UP has been
set. It appears that now, the network stack tries to add multicast
addresses to interface's filter before those steps occur. Normally,
ndis_setmulti() would trap this condition by checking for the IFF_UP
flag, but the network code has in fact set this flag already, so
ndis_setmulti() is fooled into thinking the interface has been
initialized when it really hasn't.
It turns out this is usually harmless because the ifp->if_init()
routine (in this case ndis_init()) will set up the multicast
filter when it initializes the hardware anyway, and the underlying
routines (ndis_get_info()/ndis_set_info()) know that the driver/NIC
haven't been initialized yet, but you end up spurious error messages
on the console all the time.
Something tells me this new behavior isn't really correct. I think
the intention was to fix it so that ifp->if_init() is only called
once when we ifconfig an interface up, but the end result seems a
little bogus: the change of the IFF_UP flag should be propagated
down to the driver before calling any other ioctl() that might actually
require the hardware to be up and running.
make the key name matching case-insensitive. There are some drivers
and .inf files that have mismatched cases, e.g. the driver will look
for "AdhocBand" whereas the .inf file specifies a registry key to be
created called "AdHocBand." The mismatch is probably a typo that went
undetected (so much for QA), but since Windows seems to be case-insensitive,
we should be too.
In if_ndis.c, initialize rates and channels correctly so that specify
frequences correctly when trying to set channels in the 5Ghz band, and
so that 802.11b rates show up for some a/b/g cards (which otherwise
appear to have no 802.11b modes).
Also, when setting OID_802_11_CONFIGURATION in ndis_80211_setstate(),
provide default values for the beacon interval, ATIM window and dwelltime.
The Atheros "Aries" driver will crash if you try to select ad-hoc mode
and leave the beacon interval set to 0: it blindly uses this value and
does a division by 0 in the interrupt handler, causing an integer
divide trap.
ntoskrnl_unlocl_dpc().
- hal_raise_irql(), hal_lower_irql() and hal_irql() didn't work right
on SMP (priority inheritance makes things... interesting). For now,
use only two states: DISPATCH_LEVEL (PI_REALTIME) and PASSIVE_LEVEL
(everything else). Tested on a dual PIII box.
- Use ndis_thsuspend() in ndis_sleep() instead of tsleep(). (I added
ndis_thsuspend() and ndis_thresume() to replace kthread_suspend()
and kthread_resume(); the former will preserve a thread's priority
when it wakes up, the latter will not.)
- Change use of tsleep() in ndis_stop_thread() to prevent priority
change on wakeup.
