Commit Graph

18 Commits

Author SHA1 Message Date
John Baldwin
4ac60df584 Add a new 'pmap_invalidate_cache()' to flush the CPU caches via the
wbinvd() instruction.  This includes a new IPI so that all CPU caches on
all CPUs are flushed for the SMP case.

MFC after:	1 month
2006-05-01 21:36:47 +00:00
John Baldwin
215e7c161a Rework how we wire up interrupt sources to CPUs:
- Throw out all of the logical APIC ID stuff.  The Intel docs are somewhat
  ambiguous, but it seems that the "flat" cluster model we are currently
  using is only supported on Pentium and P6 family CPUs.  The other
  "hierarchy" cluster model that is supported on all Intel CPUs with
  local APICs is severely underdocumented.  For example, it's not clear
  if the OS needs to glean the topology of the APIC hierarchy from
  somewhere (neither ACPI nor MP Table include it) and setup the logical
  clusters based on the physical hierarchy or not.  Not only that, but on
  certain Intel chipsets, even though there were 4 CPUs in a logical
  cluster, all the interrupts were only sent to one CPU anyway.
- We now bind interrupts to individual CPUs using physical addressing via
  the local APIC IDs.  This code has also moved out of the ioapic PIC
  driver and into the common interrupt source code so that it can be
  shared with MSI interrupt sources since MSI is addressed to APICs the
  same way that I/O APIC pins are.
- Interrupt source classes grow a new method pic_assign_cpu() to bind an
  interrupt source to a specific local APIC ID.
- The SMP code now tells the interrupt code which CPUs are avaiable to
  handle interrupts in a simpler and more intuitive manner.  For one thing,
  it means we could now choose to not route interrupts to HT cores if we
  wanted to (this code is currently in place in fact, but under an #if 0
  for now).
- For now we simply do static round-robin of IRQs to CPUs when the first
  interrupt handler just as before, with the change that IRQs are now
  bound to individual CPUs rather than groups of up to 4 CPUs.
- Because the IRQ to CPU mapping has now been moved up a layer, it would
  be easier to manage this mapping from higher levels.  For example, we
  could allow drivers to specify a CPU affinity map for their interrupts,
  or we could allow a userland tool to bind IRQs to specific CPUs.

The MFC is tentative, but I want to see if this fixes problems some folks
had with UP APIC kernels on 6.0 on SMP machines (an SMP kernel would work
fine, but a UP APIC kernel (such as GENERIC in RELENG_6) would lose
interrupts).

MFC after:	1 week
2006-02-28 22:24:55 +00:00
John Baldwin
b439e431bf Tweak how the MD code calls the fooclock() methods some. Instead of
passing a pointer to an opaque clockframe structure and requiring the
MD code to supply CLKF_FOO() macros to extract needed values out of the
opaque structure, just pass the needed values directly.  In practice this
means passing the pair (usermode, pc) to hardclock() and profclock() and
passing the boolean (usermode) to hardclock_cpu() and hardclock_process().
Other details:
- Axe clockframe and CLKF_FOO() macros on all architectures.  Basically,
  all the archs were taking a trapframe and converting it into a clockframe
  one way or another.  Now they can just extract the PC and usermode values
  directly out of the trapframe and pass it to fooclock().
- Renamed hardclock_process() to hardclock_cpu() as the latter is more
  accurate.
- On Alpha, we now run profclock() at hz (profhz == hz) rather than at
  the slower stathz.
- On Alpha, for the TurboLaser machines that don't have an 8254
  timecounter, call hardclock() directly.  This removes an extra
  conditional check from every clock interrupt on Alpha on the BSP.
  There is probably room for even further pruning here by changing Alpha
  to use the simplified timecounter we use on x86 with the lapic timer
  since we don't get interrupts from the 8254 on Alpha anyway.
- On x86, clkintr() shouldn't ever be called now unless using_lapic_timer
  is false, so add a KASSERT() to that affect and remove a condition
  to slightly optimize the non-lapic case.
- Change prototypeof  arm_handler_execute() so that it's first arg is a
  trapframe pointer rather than a void pointer for clarity.
- Use KCOUNT macro in profclock() to lookup the kernel profiling bucket.

Tested on:	alpha, amd64, arm, i386, ia64, sparc64
Reviewed by:	bde (mostly)
2005-12-22 22:16:09 +00:00
John Baldwin
333b8de537 MFi386:
- Move PUSH_FRAME and POP_FRAME to asmacros.h and use PUSH_FRAME in
  atpic entry points.
- Move PCPU_* asm macros out of the middle of the asm profiling macros.
- Pass IRQ vector argument as an int rather than void * to reduce diffs
  with i386.
- EOI the lapic in C for the lapic timer handler.
- GC unused Xcpuast function.
- Split IPI_STOP handling code of ipi_nmi_handler() out into a
  cpustop_handler() function and call it from Xcpustop rather than
  duplicating all the logic in assembly.
- Fixup the list of symbols with interrupt frames in ddb traces.
  Xatpic_fastintr* have never existed on amd64, and the lapic timer
  handler and various IPI handlers were missing.
- Use trapframe instead of intrframe for interrupt entry points (on amd64
  the interrupt vector was already a separate argument, so the two frames
  were already identical) and GC intrframe.

Submitted by:	peter (3)
2005-12-08 18:33:30 +00:00
John Baldwin
c7362ff7fb Change the x86 code to allocate IDT vectors on-demand when an interrupt
source is first enabled similar to how intr_event's now allocate ithreads
on-demand.  Previously, we would map IDT vectors 1:1 to IRQs.  Since we
only have 191 available IDT vectors for I/O interrupts, this limited us
to only supporting IRQs 0-190 corresponding to the first 190 I/O APIC
intpins.  On many machines, however, each PCI-X bus has its own APIC even
though it only has 1 or 2 devices, thus, we were reserving between 24 and
32 IRQs just for 1 or 2 devices and thus 24 or 32 IDT vectors.  With this
change, a machine with 100 IRQs but only 5 in use will only use up 5 IDT
vectors.  Also, this change provides an API (apic_alloc_vector() and
apic_free_vector()) that will allow a future MSI interrupt source driver to
request IDT vectors for use by MSI interrupts on x86 machines.

Tested on:	amd64, i386
2005-11-02 20:11:47 +00:00
Stephan Uphoff
6097174e4d Add IPI support for preempting a thread on another CPU.
MFC after:	3 weeks
2005-06-09 18:23:54 +00:00
Peter Wemm
ba5f6b61da MFi386: use the lapic timer for UP systems that are using the apic so that
IRQ0 and mixed mode isn't a problem anymore.  This removes mixed mode
support because nothing is left that uses it.
2005-04-15 18:44:53 +00:00
Peter Wemm
c29f1e2b3b MFi386: Bring over John's local apic timer code 2005-02-28 23:37:35 +00:00
Peter Wemm
b6e89c6d47 JumboMFi386: use bitmapped IPI handler. Update elcr and default mptable
config handler.  Tidy up various local apic initialization.
2005-01-21 06:01:20 +00:00
Peter Wemm
d8ad50b704 MFi386: various io apic cleanups 2004-07-08 01:42:49 +00:00
Peter Wemm
12c1418ccf Kill the LAZYPMAP ifdefs. While they worked, they didn't do anything
to help the AMD cpus (which have a hardware tlb flush filter).  I held
off to see what the 64 bit Intel cpus did, but it doesn't seem to help
much there either.  Oh well, store it in the Attic.
2004-05-16 22:11:50 +00:00
Peter Wemm
463e5aa66e MFi386: numerous interrupt and acpi updates 2004-05-16 20:30:47 +00:00
Peter Wemm
0d2a298904 Initial landing of SMP support for FreeBSD/amd64.
- This is heavily derived from John Baldwin's apic/pci cleanup on i386.
- I have completely rewritten or drastically cleaned up some other parts.
  (in particular, bootstrap)
- This is still a WIP.  It seems that there are some highly bogus bioses
  on nVidia nForce3-150 boards.  I can't stress how broken these boards
  are.  I have a workaround in mind, but right now the Asus SK8N is broken.
  The Gigabyte K8NPro (nVidia based) is also mind-numbingly hosed.
- Most of my testing has been with SCHED_ULE.  SCHED_4BSD works.
- the apic and acpi components are 'standard'.
- If you have an nVidia nForce3-150 board, you are stuck with 'device
  atpic' in addition, because they somehow managed to forget to connect the
  8254 timer to the apic, even though its in the same silicon!  ARGH!
  This directly violates the ACPI spec.
2003-11-17 08:58:16 +00:00
Peter Wemm
40e3826a9f Whitespace nit (sorry, couldn't help it) 2003-11-14 22:21:30 +00:00
John Baldwin
3ab2ba59f4 Shuffle the APIC interrupt vectors around a bit:
- Move the IPI and local APIC interrupt vectors up into the 0xf0 - 0xff
  range.  The pmap lazyfix IPI was reordered down next to the TLB
  shootdowns to avoid conflicting with the spurious interrupt vector.
- Move the base of APIC interrupts up 16 so that the first 16 APIC
  interrupts do not overlap the vectors used by the ATPIC.
- Remove bogus interrupt vector reservations for LINT[01].
- Now that 0xc0 - 0xef are available, use them for device interrupts.
  This increases the number of APIC device interrupts to 191.
- Increase the system-wide number of global interrupts to 191 to catch up
  to more APIC interrupts.

Requested by:	peter (2)
2003-11-14 19:10:13 +00:00
John Baldwin
69487322d8 Fix a typo. 2003-11-13 16:41:07 +00:00
John Baldwin
bd9cd7e3f7 - Move manipulation of td_intr_nesting_level out of assembly interrupt
vector stubs and into the C functions they call.
- Move disabling and EOIing of interrupt sources out of PIC driver entry
  points and into intr_execute_handlers().  Intr_execute_handlers() only
  disables a source for an interrupt if it is a stray interrupt or has
  threaded handlers.  Sources with fast handlers no longer disable (mask)
  the source while executing the handlers.
- Move the setting of clkintr_pending into intr_execute_handlers() and set
  the variable for any interrupt source with a vector of 0.  (Should only
  be true for IRQ 0.)  This fixes clkintr_pending in the NO_MIXED_MODE
  case.
- Implement lapic_eoi() and use it to implement ioapic_eoi_source().
- Rename atpic_sched_ithd() to atpic_handle_intr() since it is used to
  handle all atpic interrupts and not just threaded ones.

Inspired by:	peter's changes to amd64 in p4 (1)
Requested by:	bde (2)
2003-11-12 18:13:57 +00:00
John Baldwin
6f92bdd0c1 New APIC support code:
- The apic interrupt entry points have been rewritten so that each entry
  point can serve 32 different vectors.  When the entry is executed, it
  uses one of the 32-bit ISR registers to determine which vector in its
  assigned range was triggered.  Thus, the apic code can support 159
  different interrupt vectors with only 5 entry points.
- We now always to disable the local APIC to work around an errata in
  certain PPros and then re-enable it again if we decide to use the APICs
  to route interrupts.
- We no longer map IO APICs or local APICs using special page table
  entries.  Instead, we just use pmap_mapdev().  We also no longer
  export the virtual address of the local APIC as a global symbol to
  the rest of the system, but only in local_apic.c.  To aid this, the
  APIC ID of each CPU is exported as a per-CPU variable.
- Interrupt sources are provided for each intpin on each IO APIC.
  Currently, each source is given a unique interrupt vector meaning that
  PCI interrupts are not shared on most machines with an I/O APIC.
  That mapping for interrupt sources to interrupt vectors is up to the
  APIC enumerator driver however.
- We no longer probe to see if we need to use mixed mode to route IRQ 0,
  instead we always use mixed mode to route IRQ 0 for now.  This can be
  disabled via the 'NO_MIXED_MODE' kernel option.
- The npx(4) driver now always probes to see if a built-in FPU is present
  since this test can now be performed with the new APIC code.  However,
  an SMP kernel will panic if there is more than one CPU and a built-in
  FPU is not found.
- PCI interrupts are now properly routed when using APICs to route
  interrupts, so remove the hack to psuedo-route interrupts when the
  intpin register was read.
- The apic.h header was moved to apicreg.h and a new apicvar.h header
  that declares the APIs used by the new APIC code was added.
2003-11-03 21:53:38 +00:00