Commit Graph

47 Commits

Author SHA1 Message Date
Matthew Dillon
cd906fe3de Optimize two cases in the MP locking code. First, it is not necessary
to use a locked cmpexg when unlocking a lock that we already hold, since
    nobody else can touch the lock while we hold it.  Second, it is not
    necessary to use a locked cmpexg when locking a lock that we already
    hold, for the same reason.  These changes will allow MP locks to be used
    recursively without impacting performance.

    Modify two procedures that are called only by assembly and are already
    NOPROF entries to pass a critical argument in %edx instead of on the
    stack, removing a significant amount of code from the critical path
    as a consequence.

Reviewed by:	Alfred Perlstein <bright@wintelcom.net>, Peter Wemm <peter@netplex.com.au>
1999-11-19 16:49:30 +00:00
Mike Smith
97ecdb1d58 Invoke smp_rendezvous_action() using the a.out compatible asnames.h
technique (bleagh).
1999-09-10 01:17:01 +00:00
Peter Wemm
c3aac50f28 $Id$ -> $FreeBSD$ 1999-08-28 01:08:13 +00:00
Alan Cox
a236cb64a9 Modify the macros IMASK_UNLOCK, CPL_UNLOCK, and REL_FAST_INTR_LOCK
to perform the s_unlock inline.
1999-08-23 19:14:18 +00:00
Mike Smith
91fe3dc1e1 Implement an all-CPU shootdown-style rendezvous facility. This allows
the caller to specify a function to be guarded between an entry and exit
barrier, as well as pre- and post-barrier functions.

The primary use for this function is synchronised update of per-cpu private
data.  The implementation is almost (but not quite) MI; with a better
mechanism for masking per-CPU interrupts it could probably be hoisted.

Reviewed by:	peter (partially)
1999-07-20 06:52:35 +00:00
Bruce Evans
eec2e836e9 Go back to the old (icu.s rev.1.7 1993) way of keeping the AST-pending
bit separate from ipending, since this is simpler and/or necessary for
SMP and may even be better for UP.

Reviewed by:	alc, luoqi, tegge
1999-07-10 15:28:01 +00:00
Alan Cox
789fb7ccdc An SMP-specific change: Add the lock prefix to RMW operations
on ipending.
1999-07-03 06:33:48 +00:00
Tor Egge
a31bad69da Clean up bitrot in interrupt tracing code. 1999-06-16 03:53:52 +00:00
Jonathan Lemon
eb9d435ae7 Unifdef VM86.
Reviewed by:	silence on on -current
1999-06-01 18:20:36 +00:00
Bruce Evans
b50641ef9c Fixed glitches (jumps) of about 1/HZ seconds for the i8254 timecounter.
The old version only worked right when the time was read strictly
more often than every 1/HZ seconds, but we only guarantee reading
it every (1/HZ + epsilon) seconds.  Part of rev.1.126-1.127 attempted
to fix this but didn't succeed.  Detect counter rollover using the
heuristic from the old version of microtime() with additional
complications for supporting calls from fast interrupt handlers.
This works provided i8254 interrupts are not delayed by more than
1/(2*HZ) seconds.

This needs more comments, and cleanups for the SMP case, and more
testing of the SMP case before it is merged into RELENG_3.

Tested by:		jhay
1999-05-28 14:08:59 +00:00
Luoqi Chen
5206bca10a Enable vmspace sharing on SMP. Major changes are,
- %fs register is added to trapframe and saved/restored upon kernel entry/exit.
- Per-cpu pages are no longer mapped at the same virtual address.
- Each cpu now has a separate gdt selector table. A new segment selector
  is added to point to per-cpu pages, per-cpu global variables are now
  accessed through this new selector (%fs). The selectors in gdt table are
  rearranged for cache line optimization.
- fask_vfork is now on as default for both UP and SMP.
- Some aio code cleanup.

Reviewed by:	Alan Cox	<alc@cs.rice.edu>
		John Dyson	<dyson@iquest.net>
		Julian Elischer	<julian@whistel.com>
		Bruce Evans	<bde@zeta.org.au>
		David Greenman	<dg@root.com>
1999-04-28 01:04:33 +00:00
Bruce Evans
6a9df7a421 Generate intrnames[] dynamically. This should be new-bus friendly.
Old version reviewed by: se
1999-04-14 14:26:36 +00:00
Tor Egge
0bf47fca4f Test CF after a btrl operation instead of testing ZF (which is undefined). 1999-04-10 19:19:02 +00:00
Tor Egge
572d053e17 Maintain a mapping from irq number to (ioapic number, int pin) tuple,
and use this when masking/unmasking interrupts.

Maintain a mapping from (iopaic number, int pin) tuple to irq number,
and use this when configuring devices and programming the ioapics.

Previous code assumed that irq number was equal to int pin number, and
that the ioapic number was 0.

Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
Luoqi Chen
02c71890cd Make irq forwarding truely functional. 1998-09-04 23:03:04 +00:00
Bruce Evans
92971f1fd7 Register tty software interrupt handlers at run time using register_swi()
instead of at compile time using ifdefs.

Use _swi_null instead of dummycamisr.  CAM and dpt should call
register_swi() instead of hacking on ihandlers[] directly.
1998-08-11 17:01:32 +00:00
Bruce Evans
18c5a6c435 Implemented dynamic registration of software interrupt handlers. Not
used yet.

Use dummy SWI handlers to avoid some checks for null pointers.
1998-08-11 15:08:13 +00:00
Tor Egge
2f1e70693d Add forwarding of roundrobin to other cpus. This gives a more regular
update of cpu usage as shown by top when one process is cpu bound
(no system calls) while the system is otherwise idle (except for top).

Don't attempt to switch to the BSP in boot().  If the system was idle when
an interrupt caused a panic, this won't work.  Instead, switch to the BSP
in cpu_reset.

Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
Tor Egge
beede91e58 Mask the interrupt before setting the corresponding bit in ipending if
the interrupt is already active.
Don't use lock prefix for operations on ipending.
Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
Tor Egge
5dd528cd4e Remove special handling for resuming clock interrupt when using APIC_IO.
The `generic' vector stubs do the right thing.
1998-03-05 21:45:53 +00:00
Tor Egge
02c1dc3bbc When entering the apic version of slow interrupt handler, level
interrupts are masked, and EOI is sent iff the corresponding ISR bit
is set in the local apic. If the CPU cannot obtain the interrupt
service lock (currently the global kernel lock) the interrupt is
forwarded to the CPU holding that lock.

Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
Tor Egge
3163861c7b Forward the signal if the process runs on a different CPU. This reduces
the signal handling latency for cpu-bound processes that performs very
few system calls.

The IPI for forcing an additional software trap is no longer dependent upon
BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
Justin T. Gibbs
99117bb6ec Addition of splsoftvm and a VM SWI to handle bus dma related callbacks.
This SWI may be useful for other, defered, VM tasks.
1998-01-15 07:34:01 +00:00
Steve Passe
baf41202ca The improvements to clock statistics by Tor Egge
Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h)

apic_vector.s also contains a small change I (smp) made to eliminate
the double level INT problem.  It seems stable, but I haven't the tools
in place to prove it fixes the problem.

Reviewed by:	smp@csn.net
Submitted by:	Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
Justin T. Gibbs
643b3d0cf2 Fix a serious bug I introduced while adding in support for CAM interrupts.
It seems I didn't count my 0's properly when adding the new masks into
icu_vector.s pushing SWI_AST_MASK off the end of the array and screwing
up the indexing for SWI_CLOCK_MASK.

Fix the bug icu_vector.s and also reformat the code in both icu_vector.s and
apic_vector.s so that it will be much harder to make the same mistake in
the future.

Submitted by:	Bruce Evans <bde@zeta.org.au>
1997-09-28 19:30:01 +00:00
Justin T. Gibbs
02a199102d aha1542.c aic6360.c cy.c fd.c ft.c
if_ie.c if_wl.c if_zp.c isa.c isa_device.h
labpc.c mcd.c ncr5380.c scd.c seagate.c si.c
sio.c tw.c ultra14f.c wcd.c wd.c:

	Update for changes in the callout interface.

apic_vector.s icu_vector.s ipl.s ipl_funcs.c:

	Add CAM software/hardware interrupt support.
1997-09-21 21:41:49 +00:00
Steve Passe
20233f27f4 General cleanup of the lock pushdown code. They are grouped and enabled
from machine/smptests.h:

#define PUSHDOWN_LEVEL_1
#define PUSHDOWN_LEVEL_2
#define PUSHDOWN_LEVEL_3
#define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
Steve Passe
78292efeef Another round of lock pushdown.
Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel.
UP kernel expects that this is enough to guarantee exclusive access to
regions of code bracketed by these 2 functions.
Add a simplelock to bracket clock accesses in clock.c: clock_lock.

Help from:	Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
Steve Passe
34162fc615 Support for the new FAST_HI algorithm, enabled.
Preliminary support for the INTR_SIMPLELOCK algorithm, disabled.
Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
Steve Passe
886e789627 The last of the encapsolation of cpl/spl/ipending things into a critical
region protected by the simplelock 'cpl_lock'.

Notes:

 - this code is currently controlled on a section by section basis with
   defines in machine/param.h.  All sections are currently enabled.

 - this code is not as clean as I would like, but that can wait till later.

 - the "giant lock" still surrounds most instances of this "cpl region".
   I still have to do the code that arbitrates setting cpl between the
   top and bottom halves of the kernel.

 - the possibility of deadlock exists, I am committing the code at this
   point so as to exercise it and detect any such cases B4 the "giant lock"
   is removed.
1997-08-24 00:05:37 +00:00
Steve Passe
4a73d99f7e Made PEND_INTS default.
Made NEW_STRATEGY default.
Removed misc. old cruft.

Centralized simple locks into mp_machdep.c
Centralized simple lock macros into param.h

More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
Steve Passe
7b185ef809 Preperation for moving cpl into critical region access.
Several new fine-grained locks.
New FAST_INTR() methods:
 - separate simplelock for FAST_INTR, no more giant lock.
 - FAST_INTR()s no longer checks ipending on way out of ISR.
sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
Steve Passe
e7802310d2 Added trap specific lock calls: get_fpu_lock, etc.
All resolve to the GIANT_LOCK at this time, it is purely a logical partitioning.
1997-08-10 20:59:07 +00:00
Steve Passe
6354231eaa Added 'lock' instruction before 3 places that update ipending.
This may or may not fix the "high IO freezes SMP kernel" problem.
1997-08-09 19:40:28 +00:00
Steve Passe
d1283d9c9d Converted the TEST_LOPRIO code to default.
removed PEND_INTS 1st try
direct call to MPtrylock
1997-07-31 05:42:06 +00:00
Steve Passe
412f3e4d71 Modified the PEND_INTS algorithm to fix the ISA INT loss problem.
Noticed by:	dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
Steve Passe
12084b3cf1 clock.c:
- removed TEST_ALTTIMER.
 - removed APIC_PIN0_TIMER.
 - removed TIMER_ALL.

apic_vector.s:
 - new algorithm where a CPU uses try_mplock instead of get_mplock:
	if successful continue as before.
	if fail set ipending bit, mask INT (to avoid recursion), cleanup & iret.

   This allows the CPU to return to successful work, while the ISR will be run
   by the CPU holding the lock as part of the doreti dance.
1997-07-26 01:53:04 +00:00
Steve Passe
812e4da7a8 New simple_lock code in asm:
- s_lock_init()
 - s_lock()
 - s_lock_try()
 - s_unlock()

Created lock for IO APIC and apic_imen  (SMP version of imen)
 - imen_lock

Code to use imen_lock for access from apic_ipl.s and apic_vector.s.
Moved this code *outside* of mp_lock.

It seems to work!!!
1997-07-23 20:47:19 +00:00
Steve Passe
d2ecb616f2 Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code. 1997-07-18 21:27:53 +00:00
Steve Passe
bb0be67512 Cleanup. 1997-07-15 03:28:53 +00:00
Steve Passe
4e914a7eb7 Tighten up asm code for EOI access. 1997-07-15 02:49:21 +00:00
Steve Passe
c064ef9175 Cleanup old stop_cpus/restart_cpus() cruft.
new code for handling mixed-mode 8259/APIC programming without 'ExtInt'
new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
Steve Passe
b956b5370c First cut at code for handling "spurious INTerrupts".
First cut at code for handling CPU stop/restart.

Notes:
	not working properly yet.
1997-07-06 23:32:38 +00:00
Steve Passe
58db75841d apic_vector.s:
- added Xcpustop IPI code to support stop_cpus()/restart_cpus().
   it is off by default, enable via smptests.h:TEST_CPUSTOP

intr_machdep.h:
 - moved +ICULEN to lower level.
 - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Peter Wemm
b3196e4b9f Preliminary support for per-cpu data pages.
This eliminates a lot of #ifdef SMP type code.  Things like _curproc reside
in a data page that is unique on each cpu, eliminating the expensive macros
like:    #define curproc (SMPcurproc[cpunumber()])

There are some unresolved bootstrap and address space sharing issues at
present, but Steve is waiting on this for other work.  There is still some
strictly temporary code present that isn't exactly pretty.

This is part of a larger change that has run into some bumps, this part is
standalone so it should be safe.  The temporary code goes away when the
full idle cpu support is finished.

Reviewed by: fsmp, dyson
1997-06-22 16:04:22 +00:00
Peter Wemm
0589fe3b6e The SWI_NET_MASK and SWI_TTY_MASK handlers are now back adjacent to the
top of the hardware interrupt handlers.  Apparently this is slightly
faster with the bit scanning instruction that looks these up - this set of
changes reverts the original change.

Reviewed by: bde
1997-05-31 08:59:51 +00:00
Steve Passe
81c1993f87 Split vector.s into UP and SMP specific files:
- vector.s		<- stub called by i386/exception.s
 - icu_vector.s		<- UP
 - apic_vector.s	<- SMP

Split icu.s into UP and SMP specific files:
 - ipl.s		<- stub called by i386/exception.s (formerly icu.s)
 - icu_ipl.s		<- UP
 - apic_ipl.s		<- SMP

This was done in preparation for massive changes to the SMP INTerrupt
mechanisms.  More fine tuning, such as merging ipl.s into exception.s,
may be appropriate.
1997-05-26 17:58:27 +00:00