d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
4dbc7c2826
freebsd-dev/sys/i386/isa/intr_machdep.h

252 lines
8.8 KiB
C
Raw Normal View History

Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
/*-
* Copyright (c) 1991 The Regents of the University of California.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
* $FreeBSD$
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
*/
#ifndef _I386_ISA_INTR_MACHDEP_H_
#define _I386_ISA_INTR_MACHDEP_H_
/*
* Low level interrupt code.
*/
Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL" is an application space macro and the applications are supposed to be free to use it as they please (but cannot). This is consistant with the other BSD's who made this change quite some time ago. More commits to come.
1999-12-29 04:46:21 +00:00
#ifdef _KERNEL
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
Ifdefed SMP-only declarations.
1998-02-13 06:59:22 +00:00
#if defined(SMP) || defined(APIC_IO)
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
/*
* XXX FIXME: rethink location for all IPI vectors.
*/
/*
APIC TPR priority vector levels:
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
0xff (255) +-------------+
| | 15 (IPIs: Xspuriousint)
0xf0 (240) +-------------+
| | 14
0xe0 (224) +-------------+
| | 13
0xd0 (208) +-------------+
| | 12
0xc0 (192) +-------------+
| | 11
0xb0 (176) +-------------+
| | 10 (IPIs: Xcpustop)
0xa0 (160) +-------------+
| | 9 (IPIs: Xinvltlb)
0x90 (144) +-------------+
| | 8 (linux/BSD syscall, IGNORE FAST HW INTS)
0x80 (128) +-------------+
| | 7 (FAST_INTR 16-23)
0x70 (112) +-------------+
| | 6 (FAST_INTR 0-15)
0x60 (96) +-------------+
| | 5 (IGNORE HW INTS)
0x50 (80) +-------------+
| | 4 (2nd IO APIC)
0x40 (64) +------+------+
| | | 3 (upper APIC hardware INTs: PCI)
0x30 (48) +------+------+
| | 2 (start of hardware INTs: ISA)
0x20 (32) +-------------+
| | 1 (exceptions, traps, etc.)
0x10 (16) +-------------+
| | 0 (exceptions, traps, etc.)
0x00 (0) +-------------+
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
*/
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
/* IDT vector base for regular (aka. slow) and fast interrupts */
#define TPR_SLOW_INTS 0x20
#define TPR_FAST_INTS 0x60
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
/* XXX note that the AST interrupt is at 0x50 */
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
/* blocking values for local APIC Task Priority Register */
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
#define TPR_BLOCK_HWI 0x4f /* hardware INTs */
#define TPR_IGNORE_HWI 0x5f /* ignore INTs */
#define TPR_BLOCK_FHWI 0x7f /* hardware FAST INTs */
#define TPR_IGNORE_FHWI 0x8f /* ignore FAST INTs */
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
#define TPR_BLOCK_XINVLTLB 0x9f /* */
#define TPR_BLOCK_XCPUSTOP 0xaf /* */
#define TPR_BLOCK_ALL 0xff /* all INTs */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
#ifdef TEST_TEST1
/* put a 'fake' HWI in top of APIC prio 0x3x, 32 + 31 = 63 = 0x3f */
#define XTEST1_OFFSET (ICU_OFFSET + 31)
#endif /** TEST_TEST1 */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
/* TLB shootdowns */
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
#define XINVLTLB_OFFSET (ICU_OFFSET + 112) /* 0x90 */
#define XINVLPG_OFFSET (ICU_OFFSET + 113) /* 0x91 */
#define XINVLRNG_OFFSET (ICU_OFFSET + 114) /* 0x92 */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
/* inter-cpu clock handling */
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
#define XHARDCLOCK_OFFSET (ICU_OFFSET + 120) /* 0x98 */
#define XSTATCLOCK_OFFSET (ICU_OFFSET + 121) /* 0x99 */
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
Implement an all-CPU shootdown-style rendezvous facility. This allows the caller to specify a function to be guarded between an entry and exit barrier, as well as pre- and post-barrier functions. The primary use for this function is synchronised update of per-cpu private data. The implementation is almost (but not quite) MI; with a better mechanism for masking per-CPU interrupts it could probably be hoisted. Reviewed by: peter (partially)
1999-07-20 06:52:35 +00:00
/* inter-CPU rendezvous */
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
#define XRENDEZVOUS_OFFSET (ICU_OFFSET + 122) /* 0x9A */
Implement an all-CPU shootdown-style rendezvous facility. This allows the caller to specify a function to be guarded between an entry and exit barrier, as well as pre- and post-barrier functions. The primary use for this function is synchronised update of per-cpu private data. The implementation is almost (but not quite) MI; with a better mechanism for masking per-CPU interrupts it could probably be hoisted. Reviewed by: peter (partially)
1999-07-20 06:52:35 +00:00
Commit a partial lazy thread switch mechanism for i386. it isn't as lazy as it could be and can do with some more cleanup. Currently its under options LAZY_SWITCH. What this does is avoid %cr3 reloads for short context switches that do not involve another user process. ie: we can take an interrupt, switch to a kthread and return to the user without explicitly flushing the tlb. However, this isn't as exciting as it could be, the interrupt overhead is still high and too much blocks on Giant still. There are some debug sysctls, for stats and for an on/off switch. The main problem with doing this has been "what if the process that you're running on exits while we're borrowing its address space?" - in this case we use an IPI to give it a kick when we're about to reclaim the pmap. Its not compiled in unless you add the LAZY_SWITCH option. I want to fix a few more things and get some more feedback before turning it on by default. This is NOT a replacement for Bosko's lazy interrupt stuff. This was more meant for the kthread case, while his was for interrupts. Mine helps a little for interrupts, but his helps a lot more. The stats are enabled with options SWTCH_OPTIM_STATS - this has been a pseudo-option for years, I just added a bunch of stuff to it. One non-trivial change was to select a new thread before calling cpu_switch() in the first place. This allows us to catch the silly case of doing a cpu_switch() to the current process. This happens uncomfortably often. This simplifies a bit of the asm code in cpu_switch (no longer have to call choosethread() in the middle). This has been implemented on i386 and (thanks to jake) sparc64. The others will come soon. This is actually seperate to the lazy switch stuff. Glanced at by: jake, jhb
2003-04-02 23:53:30 +00:00
/* lazy pmap release */
#define XLAZYPMAP_OFFSET (ICU_OFFSET + 123) /* 0x9B */
Forward the signal if the process runs on a different CPU. This reduces the signal handling latency for cpu-bound processes that performs very few system calls. The IPI for forcing an additional software trap is no longer dependent upon BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
/* IPI to generate an additional software trap at the target CPU */
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
/* XXX in the middle of the interrupt range, overlapping IRQ48 */
#define XCPUAST_OFFSET (ICU_OFFSET + 48) /* 0x50 */
Forward the signal if the process runs on a different CPU. This reduces the signal handling latency for cpu-bound processes that performs very few system calls. The IPI for forcing an additional software trap is no longer dependent upon BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
/* IPI to signal CPUs to stop and wait for another CPU to restart them */
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
#define XCPUSTOP_OFFSET (ICU_OFFSET + 128) /* 0xA0 */
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
/*
* Note: this vector MUST be xxxx1111, 32 + 223 = 255 = 0xff:
*/
#define XSPURIOUSINT_OFFSET (ICU_OFFSET + 223)
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
Ifdefed SMP-only declarations.
1998-02-13 06:59:22 +00:00
#endif /* SMP || APIC_IO */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
Introduce a standard name for the lock protecting an interrupt controller and it's associated state variables: icu_lock with the name "icu". This renames the imen_mtx for x86 SMP, but also uses the lock to protect access to the 8259 PIC on x86 UP. This also adds an appropriate lock to the various Alpha chipsets which fixes problems with Alpha SMP machines dropping interrupts with an SMP kernel.
2001-12-20 23:48:31 +00:00
#ifdef LOCORE
/*
* Protects the IO APIC, 8259 PIC, imen, and apic_imen
*/
#define ICU_LOCK MTX_LOCK_SPIN(icu_lock, 0)
#define ICU_UNLOCK MTX_UNLOCK_SPIN(icu_lock)
#else /* LOCORE */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
/*
* Type of the first (asm) part of an interrupt handler.
*/
Remove __P.
2002-03-20 07:51:46 +00:00
typedef void inthand_t(u_int cs, u_int ef, u_int esp, u_int ss);
Remove a couple of __P() stragglers.
2002-06-29 02:32:34 +00:00
typedef void unpendhand_t(void);
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
#define IDTVEC(name) __CONCAT(X,name)
extern u_long *intr_countp[]; /* pointers into intrcnt[] */
- Remove the inthand2_t type and use the equivalent driver_intr_t type from newbus for referencing device interrupt handlers. - Move the 'struct intrec' type which describes interrupt sources into sys/interrupt.h instead of making it just be a x86 structure. - Don't create 'ithd' and 'intrec' typedefs, instead, just use 'struct ithd' and 'struct intrec' - Move the code to translate new-bus interrupt flags into an interrupt thread priority out of the x86 nexus code and into a MI ithread_priority() function in sys/kern/kern_intr.c. - Remove now-uneeded x86-specific headers from sys/dev/ata/ata-all.c and sys/pci/pci_compat.c.
2000-09-13 18:33:25 +00:00
extern driver_intr_t *intr_handler[]; /* C entry points of intr handlers */
extern struct ithd *ithds[];
Changed the type of an isa/general interrupt handler to take a `void *' arg. Fixed or hid most of the resulting type mismatches. Handlers can now be updated locally (except for reworking their global declarations in isa_device.h).
1998-06-18 15:32:09 +00:00
extern void *intr_unit[]; /* cookies to pass to intr handlers */
Introduce a standard name for the lock protecting an interrupt controller and it's associated state variables: icu_lock with the name "icu". This renames the imen_mtx for x86 SMP, but also uses the lock to protect access to the 8259 PIC on x86 UP. This also adds an appropriate lock to the various Alpha chipsets which fixes problems with Alpha SMP machines dropping interrupts with an SMP kernel.
2001-12-20 23:48:31 +00:00
extern struct mtx icu_lock;
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
inthand_t
IDTVEC(fastintr0), IDTVEC(fastintr1),
IDTVEC(fastintr2), IDTVEC(fastintr3),
IDTVEC(fastintr4), IDTVEC(fastintr5),
IDTVEC(fastintr6), IDTVEC(fastintr7),
IDTVEC(fastintr8), IDTVEC(fastintr9),
IDTVEC(fastintr10), IDTVEC(fastintr11),
IDTVEC(fastintr12), IDTVEC(fastintr13),
IDTVEC(fastintr14), IDTVEC(fastintr15);
inthand_t
IDTVEC(intr0), IDTVEC(intr1), IDTVEC(intr2), IDTVEC(intr3),
IDTVEC(intr4), IDTVEC(intr5), IDTVEC(intr6), IDTVEC(intr7),
IDTVEC(intr8), IDTVEC(intr9), IDTVEC(intr10), IDTVEC(intr11),
IDTVEC(intr12), IDTVEC(intr13), IDTVEC(intr14), IDTVEC(intr15);
Compromise for critical*()/cpu_critical*() recommit. Cleanup the interrupt disablement assumptions in kern_fork.c by adding another API call, cpu_critical_fork_exit(). Cleanup the td_savecrit field by moving it from MI to MD. Temporarily move cpu_critical*() from <arch>/include/cpufunc.h to <arch>/<arch>/critical.c (stage-2 will clean this up). Implement interrupt deferral for i386 that allows interrupts to remain enabled inside critical sections. This also fixes an IPI interlock bug, and requires uses of icu_lock to be enclosed in a true interrupt disablement. This is the stage-1 commit. Stage-2 will occur after stage-1 has stabilized, and will move cpu_critical*() into its own header file(s) + other things. This commit may break non-i386 architectures in trivial ways. This should be temporary. Reviewed by: core Approved by: core
2002-03-27 05:39:23 +00:00
unpendhand_t
IDTVEC(fastunpend0), IDTVEC(fastunpend1), IDTVEC(fastunpend2),
IDTVEC(fastunpend3), IDTVEC(fastunpend4), IDTVEC(fastunpend5),
IDTVEC(fastunpend6), IDTVEC(fastunpend7), IDTVEC(fastunpend8),
IDTVEC(fastunpend9), IDTVEC(fastunpend10), IDTVEC(fastunpend11),
IDTVEC(fastunpend12), IDTVEC(fastunpend13), IDTVEC(fastunpend14),
IDTVEC(fastunpend15), IDTVEC(fastunpend16), IDTVEC(fastunpend17),
IDTVEC(fastunpend18), IDTVEC(fastunpend19), IDTVEC(fastunpend20),
IDTVEC(fastunpend21), IDTVEC(fastunpend22), IDTVEC(fastunpend23),
IDTVEC(fastunpend24), IDTVEC(fastunpend25), IDTVEC(fastunpend26),
IDTVEC(fastunpend27), IDTVEC(fastunpend28), IDTVEC(fastunpend29),
IDTVEC(fastunpend30), IDTVEC(fastunpend31);
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
Ifdefed SMP-only declarations.
1998-02-13 06:59:22 +00:00
#if defined(SMP) || defined(APIC_IO)
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
inthand_t
IDTVEC(fastintr16), IDTVEC(fastintr17),
IDTVEC(fastintr18), IDTVEC(fastintr19),
IDTVEC(fastintr20), IDTVEC(fastintr21),
Cleanup some leftover lint from the old interrupt system. Also, while here, run up to 32 interrupt sources on APIC systems. Normalize INTREN/INTRDIS so they are the same on both UP and SMP systems rather than sometimes a macro, and sometimes a function. Reviewed by: jhb, jakeb
2000-12-04 21:15:14 +00:00
IDTVEC(fastintr22), IDTVEC(fastintr23),
IDTVEC(fastintr24), IDTVEC(fastintr25),
IDTVEC(fastintr26), IDTVEC(fastintr27),
IDTVEC(fastintr28), IDTVEC(fastintr29),
IDTVEC(fastintr30), IDTVEC(fastintr31);
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
inthand_t
IDTVEC(intr16), IDTVEC(intr17), IDTVEC(intr18), IDTVEC(intr19),
Cleanup some leftover lint from the old interrupt system. Also, while here, run up to 32 interrupt sources on APIC systems. Normalize INTREN/INTRDIS so they are the same on both UP and SMP systems rather than sometimes a macro, and sometimes a function. Reviewed by: jhb, jakeb
2000-12-04 21:15:14 +00:00
IDTVEC(intr20), IDTVEC(intr21), IDTVEC(intr22), IDTVEC(intr23),
IDTVEC(intr24), IDTVEC(intr25), IDTVEC(intr26), IDTVEC(intr27),
IDTVEC(intr28), IDTVEC(intr29), IDTVEC(intr30), IDTVEC(intr31);
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
inthand_t
Revive backed out pmap related changes from Feb 2002. The highlights are: - It actually works this time, honest! - Fine grained TLB shootdowns for SMP on i386. IPI's are very expensive, so try and optimize things where possible. - Introduce ranged shootdowns that can be done as a single IPI. - PG_G support for i386 - Specific-cpu targeted shootdowns. For example, there is no sense in globally purging the TLB cache for where we are stealing a page from the local unshared process on the local cpu. Use pm_active to track this. - Add some instrumentation for the tlb shootdown code. - Rip out SMP code from <machine/cpufunc.h> - Try and fix some very bogus PG_G and PG_PS interactions that were bad enough to cause vm86 bios calls to break. vm86 depended on our existing bugs and this was the cause of the VESA panics last time. - Fix the silly one-line error that caused the 'panic: bad pte' last time. - Fix a couple of other silly one-line errors that should have caused more pain than they did. Some more work is needed: - pmap_{zero,copy}_page[_idle]. These can be done without IPI's if we have a hook in cpu_switch. - The IPI handlers need some cleanup. I have a bogus %ds load that can be avoided. - APTD handling is rather bogus and appears to be a large source of global TLB IPI shootdowns for no really good reason. I see speedups of between 1.5% and ~4% on buildworlds in a while 1 loop. I expect to see a bigger difference when there is significant pageout activity or the system otherwise has memory shortages. I have backed out a few optimizations that I had been using over the last few days in order to be a little more conservative. I'll revisit these again over the next few days as the dust settles. New option: DISABLE_PG_G - In case I missed something.
2002-07-12 07:56:11 +00:00
Xinvltlb, /* TLB shootdowns - global */
Xinvlpg, /* TLB shootdowns - 1 page */
Xinvlrng, /* TLB shootdowns - page range */
Overhaul of the SMP code. Several portions of the SMP kernel support have been made machine independent and various other adjustments have been made to support Alpha SMP. - It splits the per-process portions of hardclock() and statclock() off into hardclock_process() and statclock_process() respectively. hardclock() and statclock() call the *_process() functions for the current process so that UP systems will run as before. For SMP systems, it is simply necessary to ensure that all other processors execute the *_process() functions when the main clock functions are triggered on one CPU by an interrupt. For the alpha 4100, clock interrupts are delievered in a staggered broadcast fashion, so we simply call hardclock/statclock on the boot CPU and call the *_process() functions on the secondaries. For x86, we call statclock and hardclock as usual and then call forward_hardclock/statclock in the MD code to send an IPI to cause the AP's to execute forwared_hardclock/statclock which then call the *_process() functions. - forward_signal() and forward_roundrobin() have been reworked to be MI and to involve less hackery. Now the cpu doing the forward sets any flags, etc. and sends a very simple IPI_AST to the other cpu(s). AST IPIs now just basically return so that they can execute ast() and don't bother with setting the astpending or needresched flags themselves. This also removes the loop in forward_signal() as sched_lock closes the race condition that the loop worked around. - need_resched(), resched_wanted() and clear_resched() have been changed to take a process to act on rather than assuming curproc so that they can be used to implement forward_roundrobin() as described above. - Various other SMP variables have been moved to a MI subr_smp.c and a new header sys/smp.h declares MI SMP variables and API's. The IPI API's from machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h. - The globaldata_register() and globaldata_find() functions as well as the SLIST of globaldata structures has become MI and moved into subr_smp.c. Also, the globaldata list is only available if SMP support is compiled in. Reviewed by: jake, peter Looked over by: eivind
2001-04-27 19:28:25 +00:00
Xhardclock, /* Forward hardclock() */
Xstatclock, /* Forward statclock() */
Forward the signal if the process runs on a different CPU. This reduces the signal handling latency for cpu-bound processes that performs very few system calls. The IPI for forcing an additional software trap is no longer dependent upon BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
Xcpuast, /* Additional software trap on other cpu */
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
Xcpustop, /* CPU stops & waits for another CPU to restart it */
Implement an all-CPU shootdown-style rendezvous facility. This allows the caller to specify a function to be guarded between an entry and exit barrier, as well as pre- and post-barrier functions. The primary use for this function is synchronised update of per-cpu private data. The implementation is almost (but not quite) MI; with a better mechanism for masking per-CPU interrupts it could probably be hoisted. Reviewed by: peter (partially)
1999-07-20 06:52:35 +00:00
Xspuriousint, /* handle APIC "spurious INTs" */
Commit a partial lazy thread switch mechanism for i386. it isn't as lazy as it could be and can do with some more cleanup. Currently its under options LAZY_SWITCH. What this does is avoid %cr3 reloads for short context switches that do not involve another user process. ie: we can take an interrupt, switch to a kthread and return to the user without explicitly flushing the tlb. However, this isn't as exciting as it could be, the interrupt overhead is still high and too much blocks on Giant still. There are some debug sysctls, for stats and for an on/off switch. The main problem with doing this has been "what if the process that you're running on exits while we're borrowing its address space?" - in this case we use an IPI to give it a kick when we're about to reclaim the pmap. Its not compiled in unless you add the LAZY_SWITCH option. I want to fix a few more things and get some more feedback before turning it on by default. This is NOT a replacement for Bosko's lazy interrupt stuff. This was more meant for the kthread case, while his was for interrupts. Mine helps a little for interrupts, but his helps a lot more. The stats are enabled with options SWTCH_OPTIM_STATS - this has been a pseudo-option for years, I just added a bunch of stuff to it. One non-trivial change was to select a new thread before calling cpu_switch() in the first place. This allows us to catch the silly case of doing a cpu_switch() to the current process. This happens uncomfortably often. This simplifies a bit of the asm code in cpu_switch (no longer have to call choosethread() in the middle). This has been implemented on i386 and (thanks to jake) sparc64. The others will come soon. This is actually seperate to the lazy switch stuff. Glanced at by: jake, jhb
2003-04-02 23:53:30 +00:00
Xrendezvous, /* handle CPU rendezvous */
Xlazypmap; /* handle lazy pmap release */
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code.
1997-07-18 21:27:53 +00:00
#ifdef TEST_TEST1
Added a hook for a "spurious INTerrupt handler".
1997-07-06 22:11:26 +00:00
inthand_t
SMP or APIC_IO: - Increased NIDT to 256. - Moved IPI vectors up above the linux compat vector. - Removed runtime setup of RTC vector.
1997-07-19 02:28:30 +00:00
Xtest1; /* 'fake' HWI at top of APIC prio 0x3x, 32+31 = 0x3f */
Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code.
1997-07-18 21:27:53 +00:00
#endif /** TEST_TEST1 */
Ifdefed SMP-only declarations.
1998-02-13 06:59:22 +00:00
#endif /* SMP || APIC_IO */
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
#ifdef APIC_IO
/*
* This is to accommodate "mixed-mode" programming for
* motherboards that don't connect the 8254 to the IO APIC.
*/
#define AUTO_EOI_1 1
#endif
#define NR_INTRNAMES (1 + ICU_LEN + 2 * ICU_LEN)
Remove __P.
2002-03-20 07:51:46 +00:00
void isa_defaultirq(void);
int isa_nmi(int cd);
int icu_setup(int intr, driver_intr_t *func, void *arg, int flags);
int icu_unset(int intr, driver_intr_t *handler);
Add ACPI S2-S4BIOS Suspend/Resume code. Some problems may remain. Reviewed by:iwasaki
2001-07-20 06:07:34 +00:00
void icu_reinit(void);
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
Mark two functions as private.
2000-04-29 07:48:37 +00:00
/*
* WARNING: These are internal functions and not to be used by device drivers!
* They are subject to change without notice.
*/
Use the MI ithread helper functions in the x86 interrupt code.
2001-02-09 17:47:44 +00:00
int inthand_add(const char *name, int irq, driver_intr_t handler, void *arg,
enum intr_type flags, void **cookiep);
int inthand_remove(void *cookie);
void sched_ithd(void *dummy);
Compromise for critical*()/cpu_critical*() recommit. Cleanup the interrupt disablement assumptions in kern_fork.c by adding another API call, cpu_critical_fork_exit(). Cleanup the td_savecrit field by moving it from MI to MD. Temporarily move cpu_critical*() from <arch>/include/cpufunc.h to <arch>/<arch>/critical.c (stage-2 will clean this up). Implement interrupt deferral for i386 that allows interrupts to remain enabled inside critical sections. This also fixes an IPI interlock bug, and requires uses of icu_lock to be enclosed in a true interrupt disablement. This is the stage-1 commit. Stage-2 will occur after stage-1 has stabilized, and will move cpu_critical*() into its own header file(s) + other things. This commit may break non-i386 architectures in trivial ways. This should be temporary. Reviewed by: core Approved by: core
2002-03-27 05:39:23 +00:00
void call_fast_unpend(int irq);
Stage 1 of a cleanup of the i386 interrupt registration mechanism. Interrupts under the new scheme are managed by the i386 nexus with the awareness of the resource manager. There is further room for optimizing the interfaces still. All the users of register_intr()/intr_create() should be gone, with the exception of pcic and i386/isa/clock.c.
1999-04-21 07:26:30 +00:00
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
#endif /* LOCORE */
Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL" is an application space macro and the applications are supposed to be free to use it as they please (but cannot). This is consistant with the other BSD's who made this change quite some time ago. More commits to come.
1999-12-29 04:46:21 +00:00
#endif /* _KERNEL */
Move interrupt handling code from isa.c to a new file. This should make isa.c (slightly) more portable and will make my life developing the really portable version much easier. Reviewed by: peter, fsmp
1997-06-02 08:19:06 +00:00
#endif /* !_I386_ISA_INTR_MACHDEP_H_ */
Reference in New Issue Copy Permalink