Commit Graph

72 Commits

Author SHA1 Message Date
Ian Dowse
3c7bcedd06 Remove the Xresume* labels from the i386 interrupt handlers; the
code in ipl.s and icu_ipl.s that used them was removed when the
interrupt thread system was committed. Debuggers also knew about
Xresume* because these labels hide the real names of the interrupt
handlers (Xintr*), and debuggers need to special-case interrupt
handlers to get the interrupt frame.

Both gdb and ddb will now use the Xintr* and Xfastintr* symbols to
detect interrupt frames. Fast interrupt frames were never identified
correctly before, so this fixes the problem of the running stack
frame getting lost in a ddb or gdb trace generated from a fast
interrupt - e.g. when debugging a simple infinite loop in the kernel
using a serial console, the frame containing the loop would never
appear in a gdb or ddb trace.

Reviewed by:	jhb, bde
2001-10-09 19:54:52 +00:00
Julian Elischer
b40ce4165d KSE Milestone 2
Note ALL MODULES MUST BE RECOMPILED
make the kernel aware that there are smaller units of scheduling than the
process. (but only allow one thread per process at this time).
This is functionally equivalent to teh previousl -current except
that there is a thread associated with each process.

Sorry john! (your next MFC will be a doosie!)

Reviewed by: peter@freebsd.org, dillon@freebsd.org

X-MFC after:    ha ha ha ha
2001-09-12 08:38:13 +00:00
John Baldwin
6caa8a1501 Overhaul of the SMP code. Several portions of the SMP kernel support have
been made machine independent and various other adjustments have been made
to support Alpha SMP.

- It splits the per-process portions of hardclock() and statclock() off
  into hardclock_process() and statclock_process() respectively.  hardclock()
  and statclock() call the *_process() functions for the current process so
  that UP systems will run as before.  For SMP systems, it is simply necessary
  to ensure that all other processors execute the *_process() functions when the
  main clock functions are triggered on one CPU by an interrupt.  For the alpha
  4100, clock interrupts are delievered in a staggered broadcast fashion, so
  we simply call hardclock/statclock on the boot CPU and call the *_process()
  functions on the secondaries.  For x86, we call statclock and hardclock as
  usual and then call forward_hardclock/statclock in the MD code to send an IPI
  to cause the AP's to execute forwared_hardclock/statclock which then call the
  *_process() functions.
- forward_signal() and forward_roundrobin() have been reworked to be MI and to
  involve less hackery.  Now the cpu doing the forward sets any flags, etc. and
  sends a very simple IPI_AST to the other cpu(s).  AST IPIs now just basically
  return so that they can execute ast() and don't bother with setting the
  astpending or needresched flags themselves.  This also removes the loop in
  forward_signal() as sched_lock closes the race condition that the loop worked
  around.
- need_resched(), resched_wanted() and clear_resched() have been changed to take
  a process to act on rather than assuming curproc so that they can be used to
  implement forward_roundrobin() as described above.
- Various other SMP variables have been moved to a MI subr_smp.c and a new
  header sys/smp.h declares MI SMP variables and API's.   The IPI API's from
  machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h.
- The globaldata_register() and globaldata_find() functions as well as the
  SLIST of globaldata structures has become MI and moved into subr_smp.c.
  Also, the globaldata list is only available if SMP support is compiled in.

Reviewed by:	jake, peter
Looked over by:	eivind
2001-04-27 19:28:25 +00:00
John Baldwin
ca7ef17c08 Remove the BETTER_CLOCK #ifdef's. The code is on by default and is here
to stay for the foreseeable future.

OK'd by:	peter (the idea)
2001-04-10 21:34:13 +00:00
John Baldwin
ae383d0cc7 Don't enable interrupts before calling sched_ithd for threaded interrupts.
Tested by:	obrien
2001-03-05 04:37:54 +00:00
Jake Burkholder
02318dac2c Remove the leading underscore from all symbols defined in x86 asm
and used in C or vice versa.  The elf compiler uses the same names
for both.  Remove asnames.h with great prejudice; it has served its
purpose.

Note that this does not affect the ability to generate an aout kernel
due to gcc's -mno-underscores option.

moral support from:	peter, jhb
2001-02-25 06:29:04 +00:00
John Baldwin
142ba5f3d7 - Make astpending and need_resched process attributes rather than CPU
attributes.  This is needed for AST's to be properly posted in a preemptive
  kernel.  They are backed by two new flags in p_sflag: PS_ASTPENDING and
  PS_NEEDRESCHED.  They are still accesssed by their old macros:
  aston(), astoff(), etc.  For completeness, an astpending() macro has been
  added to check for a pending AST, and clear_resched() has been added to
  clear need_resched().
- Rename syscall2() on the x86 back to syscall() to be consistent with
  other architectures.
2001-02-10 02:20:34 +00:00
John Baldwin
40bcb55186 Remove unused forward_irq counters. 2001-02-09 14:30:03 +00:00
John Baldwin
cd60bc00a7 - Remove all the #if 0'd code that used to implement IRQ forwarding.
- Remove #if 0'd lazy interrupt mask.
2001-01-24 10:00:07 +00:00
Jake Burkholder
a448b62ac9 Make intr_nesting_level per-process, rather than per-cpu. Setup
interrupt threads to run with it always >= 1, so that malloc can
detect M_WAITOK from "interrupt" context.  This is also necessary
in order to context switch from sched_ithd() directly.

Reviewed By:	peter
2001-01-21 19:25:07 +00:00
Peter Wemm
654c30a008 Remove APIC_INTR_DIAGNOSTIC - this has been disabled for some time now.
Remove some leftovers of removed SMP options.
2001-01-21 07:54:10 +00:00
Peter Wemm
031b6a102e Remove reference to splz_unpend - it is long gone. 2001-01-19 10:51:13 +00:00
Jake Burkholder
7d8e3aa013 Use _lapic+offset to access the local apic from assembly language
files, rather than the symbols in globals.s.  The offsets are
generated by genassym.
2000-12-14 04:16:16 +00:00
Jake Burkholder
6d43764a10 Introduce a new potientially cleaner interface for accessing per-cpu
variables from i386 assembly language.  The syntax is PCPU(member)
where member is the capitalized name of the per-cpu variable, without
the gd_ prefix.  Example: movl %eax,PCPU(CURPROC).  The capitalization
is due to using the offsets generated by genassym rather than the symbols
provided by linking with globals.o.  asmacros.h is the wrong place for
this but it seemed as good a place as any for now.  The old implementation
in asnames.h has not been removed because it is still used to de-mangle
the symbols used by the C variables for the UP case.
2000-12-13 09:23:53 +00:00
Peter Wemm
5ee171d264 Cleanup some leftover lint from the old interrupt system.
Also, while here, run up to 32 interrupt sources on APIC systems.
Normalize INTREN/INTRDIS so they are the same on both UP and SMP systems
rather than sometimes a macro, and sometimes a function.

Reviewed by:  jhb, jakeb
2000-12-04 21:15:14 +00:00
Jake Burkholder
21ad98bca8 Change doreti to take a trapframe instead of an intrframe.
Remove associated pushes of dummy units to convert frame.

Reviewed by:	jhb
2000-12-01 02:09:45 +00:00
John Baldwin
6c56727456 - Change fast interrupts on x86 to push a full interrupt frame and to
return through doreti to handle ast's.  This is necessary for the
  clock interrupts to work properly.
- Change the clock interrupts on the x86 to be fast instead of threaded.
  This is needed because both hardclock() and statclock() need to run in
  the context of the current process, not in a separate thread context.
- Kill the prevproc hack as it is no longer needed.
- We really need Giant when we call psignal(), but we don't want to block
  during the clock interrupt.  Instead, use two p_flag's in the proc struct
  to mark the current process as having a pending SIGVTALRM or a SIGPROF
  and let them be delivered during ast() when hardclock() has finished
  running.
- Remove CLKF_BASEPRI, which was #ifdef'd out on the x86 anyways.  It was
  broken on the x86 if it was turned on since cpl is gone.  It's only use
  was to bogusly run softclock() directly during hardclock() rather than
  scheduling an SWI.
- Remove the COM_LOCK simplelock and replace it with a clock_lock spin
  mutex.  Since the spin mutex already handles disabling/restoring
  interrupts appropriately, this also lets us axe all the *_intr() fu.
- Back out the hacks in the APIC_IO x86 cpu_initclocks() code to use
  temporary fast interrupts for the APIC trial.
- Add two new process flags P_ALRMPEND and P_PROFPEND to mark the pending
  signals in hardclock() that are to be delivered in ast().

Submitted by:	jakeb (making statclock safe in a fast interrupt)
Submitted by:	cp (concept of delaying signals until ast())
2000-10-06 02:20:21 +00:00
John Baldwin
1931cf940a - Heavyweight interrupt threads on the alpha for device I/O interrupts.
- Make softinterrupts (SWI's) almost completely MI, and divorce them
  completely from the x86 hardware interrupt code.
  - The ihandlers array is now gone.  Instead, there is a MI shandlers array
    that just contains SWI handlers.
  - Most of the former machine/ipl.h files have moved to a new sys/ipl.h.
- Stub out all the spl*() functions on all architectures.

Submitted by:	dfr
2000-10-05 23:09:57 +00:00
Jason Evans
0384fff8c5 Major update to the way synchronization is done in the kernel. Highlights
include:

* Mutual exclusion is used instead of spl*().  See mutex(9).  (Note: The
  alpha port is still in transition and currently uses both.)

* Per-CPU idle processes.

* Interrupts are run in their own separate kernel threads and can be
  preempted (i386 only).

Partially contributed by:	BSDi (BSD/OS)
Submissions by (at least):	cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
Matthew Dillon
50ed7dfec0 INTR_TYPE_FAST / FAST_INTR interrupts (currently just serial interrupts)
have their own lock and do not need the MP lock.  The SMP cleanup was
    a little too conservative in MP locking fast interrupts but at least
    it's trivial to fix.  MFC soon.

Submitted by: bde
2000-06-06 15:28:00 +00:00
Bruce Evans
6b4a3ce242 Pack the SWI bits to save some time and space. 2000-05-31 16:36:20 +00:00
Doug Rabson
3e8965ac09 Add SWI_TQ_MASK to imask definition. 2000-05-29 19:40:42 +00:00
David E. O'Brien
fc81cf82e9 1. `movl' is for use with 32-bit operands. Do NOT use it with 16-bit
operands.  `movw' could be used, but instead let the assembler decide
   the right instruction to use.
2. AT&T asm syntax requires a leading '*' in front of the operand for
   indirect calls and jumps.
2000-05-10 01:24:23 +00:00
Matthew Dillon
bd5caafc4d The SMP cleanup commit broke need_resched, this fixes that and also
removed unncessary MPLOCKED and 'lock' prefixes from the interrupt
    nesting level, since (A) the MP lock is held at the time, and (B) since
    the neting level is restored prior to return any interrupted code
    will see a consistent value.
2000-03-29 06:15:43 +00:00
Matthew Dillon
36e9f877df Commit major SMP cleanups and move the BGL (big giant lock) in the
syscall path inward.  A system call may select whether it needs the MP
    lock or not (the default being that it does need it).

    A great deal of conditional SMP code for various deadended experiments
    has been removed.  'cil' and 'cml' have been removed entirely, and the
    locking around the cpl has been removed.  The conditional
    separately-locked fast-interrupt code has been removed, meaning that
    interrupts must hold the CPL now (but they pretty much had to anyway).
    Another reason for doing this is that the original separate-lock for
    interrupts just doesn't apply to the interrupt thread mechanism being
    contemplated.

    Modifications to the cpl may now ONLY occur while holding the MP
    lock.  For example, if an otherwise MP safe syscall needs to mess with
    the cpl, it must hold the MP lock for the duration and must (as usual)
    save/restore the cpl in a nested fashion.

    This is precursor work for the real meat coming later: avoiding having
    to hold the MP lock for common syscalls and I/O's and interrupt threads.
    It is expected that the spl mechanisms and new interrupt threading
    mechanisms will be able to run in tandem, allowing a slow piecemeal
    transition to occur.

    This patch should result in a moderate performance improvement due to
    the considerable amount of code that has been removed from the critical
    path, especially the simplification of the spl*() calls.  The real
    performance gains will come later.

Approved by: jkh
Reviewed by: current, bde (exception.s)
Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
Matthew Dillon
cd906fe3de Optimize two cases in the MP locking code. First, it is not necessary
to use a locked cmpexg when unlocking a lock that we already hold, since
    nobody else can touch the lock while we hold it.  Second, it is not
    necessary to use a locked cmpexg when locking a lock that we already
    hold, for the same reason.  These changes will allow MP locks to be used
    recursively without impacting performance.

    Modify two procedures that are called only by assembly and are already
    NOPROF entries to pass a critical argument in %edx instead of on the
    stack, removing a significant amount of code from the critical path
    as a consequence.

Reviewed by:	Alfred Perlstein <bright@wintelcom.net>, Peter Wemm <peter@netplex.com.au>
1999-11-19 16:49:30 +00:00
Mike Smith
97ecdb1d58 Invoke smp_rendezvous_action() using the a.out compatible asnames.h
technique (bleagh).
1999-09-10 01:17:01 +00:00
Peter Wemm
c3aac50f28 $Id$ -> $FreeBSD$ 1999-08-28 01:08:13 +00:00
Alan Cox
a236cb64a9 Modify the macros IMASK_UNLOCK, CPL_UNLOCK, and REL_FAST_INTR_LOCK
to perform the s_unlock inline.
1999-08-23 19:14:18 +00:00
Mike Smith
91fe3dc1e1 Implement an all-CPU shootdown-style rendezvous facility. This allows
the caller to specify a function to be guarded between an entry and exit
barrier, as well as pre- and post-barrier functions.

The primary use for this function is synchronised update of per-cpu private
data.  The implementation is almost (but not quite) MI; with a better
mechanism for masking per-CPU interrupts it could probably be hoisted.

Reviewed by:	peter (partially)
1999-07-20 06:52:35 +00:00
Bruce Evans
eec2e836e9 Go back to the old (icu.s rev.1.7 1993) way of keeping the AST-pending
bit separate from ipending, since this is simpler and/or necessary for
SMP and may even be better for UP.

Reviewed by:	alc, luoqi, tegge
1999-07-10 15:28:01 +00:00
Alan Cox
789fb7ccdc An SMP-specific change: Add the lock prefix to RMW operations
on ipending.
1999-07-03 06:33:48 +00:00
Tor Egge
a31bad69da Clean up bitrot in interrupt tracing code. 1999-06-16 03:53:52 +00:00
Jonathan Lemon
eb9d435ae7 Unifdef VM86.
Reviewed by:	silence on on -current
1999-06-01 18:20:36 +00:00
Bruce Evans
b50641ef9c Fixed glitches (jumps) of about 1/HZ seconds for the i8254 timecounter.
The old version only worked right when the time was read strictly
more often than every 1/HZ seconds, but we only guarantee reading
it every (1/HZ + epsilon) seconds.  Part of rev.1.126-1.127 attempted
to fix this but didn't succeed.  Detect counter rollover using the
heuristic from the old version of microtime() with additional
complications for supporting calls from fast interrupt handlers.
This works provided i8254 interrupts are not delayed by more than
1/(2*HZ) seconds.

This needs more comments, and cleanups for the SMP case, and more
testing of the SMP case before it is merged into RELENG_3.

Tested by:		jhay
1999-05-28 14:08:59 +00:00
Luoqi Chen
5206bca10a Enable vmspace sharing on SMP. Major changes are,
- %fs register is added to trapframe and saved/restored upon kernel entry/exit.
- Per-cpu pages are no longer mapped at the same virtual address.
- Each cpu now has a separate gdt selector table. A new segment selector
  is added to point to per-cpu pages, per-cpu global variables are now
  accessed through this new selector (%fs). The selectors in gdt table are
  rearranged for cache line optimization.
- fask_vfork is now on as default for both UP and SMP.
- Some aio code cleanup.

Reviewed by:	Alan Cox	<alc@cs.rice.edu>
		John Dyson	<dyson@iquest.net>
		Julian Elischer	<julian@whistel.com>
		Bruce Evans	<bde@zeta.org.au>
		David Greenman	<dg@root.com>
1999-04-28 01:04:33 +00:00
Bruce Evans
6a9df7a421 Generate intrnames[] dynamically. This should be new-bus friendly.
Old version reviewed by: se
1999-04-14 14:26:36 +00:00
Tor Egge
0bf47fca4f Test CF after a btrl operation instead of testing ZF (which is undefined). 1999-04-10 19:19:02 +00:00
Tor Egge
572d053e17 Maintain a mapping from irq number to (ioapic number, int pin) tuple,
and use this when masking/unmasking interrupts.

Maintain a mapping from (iopaic number, int pin) tuple to irq number,
and use this when configuring devices and programming the ioapics.

Previous code assumed that irq number was equal to int pin number, and
that the ioapic number was 0.

Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
Luoqi Chen
02c71890cd Make irq forwarding truely functional. 1998-09-04 23:03:04 +00:00
Bruce Evans
92971f1fd7 Register tty software interrupt handlers at run time using register_swi()
instead of at compile time using ifdefs.

Use _swi_null instead of dummycamisr.  CAM and dpt should call
register_swi() instead of hacking on ihandlers[] directly.
1998-08-11 17:01:32 +00:00
Bruce Evans
18c5a6c435 Implemented dynamic registration of software interrupt handlers. Not
used yet.

Use dummy SWI handlers to avoid some checks for null pointers.
1998-08-11 15:08:13 +00:00
Tor Egge
2f1e70693d Add forwarding of roundrobin to other cpus. This gives a more regular
update of cpu usage as shown by top when one process is cpu bound
(no system calls) while the system is otherwise idle (except for top).

Don't attempt to switch to the BSP in boot().  If the system was idle when
an interrupt caused a panic, this won't work.  Instead, switch to the BSP
in cpu_reset.

Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
Tor Egge
beede91e58 Mask the interrupt before setting the corresponding bit in ipending if
the interrupt is already active.
Don't use lock prefix for operations on ipending.
Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
Tor Egge
5dd528cd4e Remove special handling for resuming clock interrupt when using APIC_IO.
The `generic' vector stubs do the right thing.
1998-03-05 21:45:53 +00:00
Tor Egge
02c1dc3bbc When entering the apic version of slow interrupt handler, level
interrupts are masked, and EOI is sent iff the corresponding ISR bit
is set in the local apic. If the CPU cannot obtain the interrupt
service lock (currently the global kernel lock) the interrupt is
forwarded to the CPU holding that lock.

Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
Tor Egge
3163861c7b Forward the signal if the process runs on a different CPU. This reduces
the signal handling latency for cpu-bound processes that performs very
few system calls.

The IPI for forcing an additional software trap is no longer dependent upon
BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
Justin T. Gibbs
99117bb6ec Addition of splsoftvm and a VM SWI to handle bus dma related callbacks.
This SWI may be useful for other, defered, VM tasks.
1998-01-15 07:34:01 +00:00
Steve Passe
baf41202ca The improvements to clock statistics by Tor Egge
Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h)

apic_vector.s also contains a small change I (smp) made to eliminate
the double level INT problem.  It seems stable, but I haven't the tools
in place to prove it fixes the problem.

Reviewed by:	smp@csn.net
Submitted by:	Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
Justin T. Gibbs
643b3d0cf2 Fix a serious bug I introduced while adding in support for CAM interrupts.
It seems I didn't count my 0's properly when adding the new masks into
icu_vector.s pushing SWI_AST_MASK off the end of the array and screwing
up the indexing for SWI_CLOCK_MASK.

Fix the bug icu_vector.s and also reformat the code in both icu_vector.s and
apic_vector.s so that it will be much harder to make the same mistake in
the future.

Submitted by:	Bruce Evans <bde@zeta.org.au>
1997-09-28 19:30:01 +00:00