d/freebsd-nq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1857b6fe17
freebsd-nq/sys/i386/isa/apic_vector.s

1043 lines
25 KiB
ArmAsm
Raw Normal View History

Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/*
* from: vector.s, 386BSD 0.1 unknown origin
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
* $Id: apic_vector.s,v 1.33 1998/09/04 23:03:04 luoqi Exp $
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
*/
Added trap specific lock calls: get_fpu_lock, etc. All resolve to the GIANT_LOCK at this time, it is purely a logical partitioning.
1997-08-10 20:59:07 +00:00
#include <machine/apic.h>
Modified the PEND_INTS algorithm to fix the ISA INT loss problem. Noticed by: dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
#include <machine/smp.h>
Added trap specific lock calls: get_fpu_lock, etc. All resolve to the GIANT_LOCK at this time, it is purely a logical partitioning.
1997-08-10 20:59:07 +00:00
Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code.
1997-07-18 21:27:53 +00:00
#include "i386/isa/intr_machdep.h"
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Added trap specific lock calls: get_fpu_lock, etc. All resolve to the GIANT_LOCK at this time, it is purely a logical partitioning.
1997-08-10 20:59:07 +00:00
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
#ifdef FAST_SIMPLELOCK
#define GET_FAST_INTR_LOCK \
pushl $_fast_intr_lock ; /* address of lock */ \
call _s_lock ; /* MP-safe */ \
addl $4,%esp
#define REL_FAST_INTR_LOCK \
pushl $_fast_intr_lock ; /* address of lock */ \
call _s_unlock ; /* MP-safe */ \
addl $4,%esp
#else /* FAST_SIMPLELOCK */
#define GET_FAST_INTR_LOCK \
call _get_isrlock
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
#define REL_FAST_INTR_LOCK \
pushl $_mp_lock ; /* GIANT_LOCK */ \
call _MPrellock ; \
add $4, %esp
#endif /* FAST_SIMPLELOCK */
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/* convert an absolute IRQ# into a bitmask */
#define IRQ_BIT(irq_num) (1 << (irq_num))
/* make an index into the IO APIC from the IRQ# */
#define REDTBL_IDX(irq_num) (0x10 + ((irq_num) * 2))
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/*
* Macros for interrupt interrupt entry, call to handler, and exit.
*/
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
#ifdef FAST_WITHOUTCPL
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
/*
*/
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
#define FAST_INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
IDTVEC(vec_name) ; \
pushl %eax ; /* save only call-used registers */ \
pushl %ecx ; \
pushl %edx ; \
pushl %ds ; \
MAYBE_PUSHL_ES ; \
movl $KDSEL,%eax ; \
movl %ax,%ds ; \
MAYBE_MOVW_AX_ES ; \
FAKE_MCOUNT((4+ACTUALLY_PUSHED)*4(%esp)) ; \
pushl _intr_unit + (irq_num) * 4 ; \
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
GET_FAST_INTR_LOCK ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
call *_intr_handler + (irq_num) * 4 ; /* do the work ASAP */ \
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
REL_FAST_INTR_LOCK ; \
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
addl $4, %esp ; \
Tighten up asm code for EOI access.
1997-07-15 02:49:21 +00:00
movl $0, lapic_eoi ; \
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
lock ; \
incl _cnt+V_INTR ; /* book-keeping can wait */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; \
incl (%eax) ; \
MEXITCOUNT ; \
MAYBE_POPL_ES ; \
popl %ds ; \
popl %edx ; \
popl %ecx ; \
popl %eax ; \
iret
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
#else /* FAST_WITHOUTCPL */
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
#define FAST_INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
IDTVEC(vec_name) ; \
pushl %eax ; /* save only call-used registers */ \
pushl %ecx ; \
pushl %edx ; \
pushl %ds ; \
MAYBE_PUSHL_ES ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
movl $KDSEL, %eax ; \
movl %ax, %ds ; \
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
MAYBE_MOVW_AX_ES ; \
FAKE_MCOUNT((4+ACTUALLY_PUSHED)*4(%esp)) ; \
GET_FAST_INTR_LOCK ; \
pushl _intr_unit + (irq_num) * 4 ; \
call *_intr_handler + (irq_num) * 4 ; /* do the work ASAP */ \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
addl $4, %esp ; \
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
movl $0, lapic_eoi ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
lock ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
incl _cnt+V_INTR ; /* book-keeping can wait */ \
movl _intr_countp + (irq_num) * 4,%eax ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
lock ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
incl (%eax) ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
movl _cpl, %eax ; /* unmasking pending HWIs or SWIs? */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
notl %eax ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
andl _ipending, %eax ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
jne 2f ; /* yes, maybe handle them */ \
1: ; \
MEXITCOUNT ; \
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
REL_FAST_INTR_LOCK ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
MAYBE_POPL_ES ; \
popl %ds ; \
popl %edx ; \
popl %ecx ; \
popl %eax ; \
iret ; \
; \
ALIGN_TEXT ; \
2: ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
cmpb $3, _intr_nesting_level ; /* enough stack? */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
jae 1b ; /* no, return */ \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
movl _cpl, %eax ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/* XXX next line is probably unnecessary now. */ \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
movl $HWI_MASK|SWI_MASK, _cpl ; /* limit nesting ... */ \
lock ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
incb _intr_nesting_level ; /* ... really limit it ... */ \
sti ; /* to do this as early as possible */ \
MAYBE_POPL_ES ; /* discard most of thin frame ... */ \
popl %ecx ; /* ... original %ds ... */ \
popl %edx ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
xchgl %eax, 4(%esp) ; /* orig %eax; save cpl */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
pushal ; /* build fat frame (grrr) ... */ \
pushl %ecx ; /* ... actually %ds ... */ \
pushl %es ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
movl $KDSEL, %eax ; \
movl %ax, %es ; \
movl (2+8+0)*4(%esp), %ecx ; /* %ecx from thin frame ... */ \
movl %ecx, (2+6)*4(%esp) ; /* ... to fat frame ... */ \
movl (2+8+1)*4(%esp), %eax ; /* ... cpl from thin frame */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
pushl %eax ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
subl $4, %esp ; /* junk for unit number */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
MEXITCOUNT ; \
jmp _doreti
Preperation for moving cpl into critical region access. Several new fine-grained locks. New FAST_INTR() methods: - separate simplelock for FAST_INTR, no more giant lock. - FAST_INTR()s no longer checks ipending on way out of ISR. sio made MP-safe (I hope).
1997-08-20 05:25:48 +00:00
#endif /** FAST_WITHOUTCPL */
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
/*
*
*/
#define PUSH_FRAME \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
pushl $0 ; /* dummy error code */ \
pushl $0 ; /* dummy trap type */ \
pushal ; \
pushl %ds ; /* save data and extra segments ... */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
pushl %es
#define POP_FRAME \
popl %es ; \
popl %ds ; \
popal ; \
addl $4+4,%esp
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
#define IOAPICADDR(irq_num) CNAME(int_to_apicintpin) + 16 * (irq_num) + 8
#define REDIRIDX(irq_num) CNAME(int_to_apicintpin) + 16 * (irq_num) + 12
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
#define MASK_IRQ(irq_num) \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
IMASK_LOCK ; /* into critical reg */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
testl $IRQ_BIT(irq_num), _apic_imen ; \
jne 7f ; /* masked, don't mask */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
orl $IRQ_BIT(irq_num), _apic_imen ; /* set the mask bit */ \
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
movl IOAPICADDR(irq_num), %ecx ; /* ioapic addr */ \
movl REDIRIDX(irq_num), %eax ; /* get the index */ \
movl %eax, (%ecx) ; /* write the index */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
movl IOAPIC_WINDOW(%ecx), %eax ; /* current value */ \
orl $IOART_INTMASK, %eax ; /* set the mask */ \
movl %eax, IOAPIC_WINDOW(%ecx) ; /* new value */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
7: ; /* already masked */ \
IMASK_UNLOCK
/*
* Test to see whether we are handling an edge or level triggered INT.
* Level-triggered INTs must still be masked as we don't clear the source,
* and the EOI cycle would cause redundant INTs to occur.
*/
#define MASK_LEVEL_IRQ(irq_num) \
testl $IRQ_BIT(irq_num), _apic_pin_trigger ; \
jz 9f ; /* edge, don't mask */ \
MASK_IRQ(irq_num) ; \
9:
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
#ifdef APIC_INTR_REORDER
#define EOI_IRQ(irq_num) \
movl _apic_isrbit_location + 8 * (irq_num), %eax ; \
movl (%eax), %eax ; \
testl _apic_isrbit_location + 4 + 8 * (irq_num), %eax ; \
jz 9f ; /* not active */ \
movl $0, lapic_eoi ; \
APIC_ITRACE(apic_itrace_eoi, irq_num, APIC_ITRACE_EOI) ; \
9:
#else
#define EOI_IRQ(irq_num) \
testl $IRQ_BIT(irq_num), lapic_isr1; \
jz 9f ; /* not active */ \
movl $0, lapic_eoi; \
APIC_ITRACE(apic_itrace_eoi, irq_num, APIC_ITRACE_EOI) ; \
9:
#endif
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
/*
* Test to see if the source is currntly masked, clear if so.
*/
#define UNMASK_IRQ(irq_num) \
IMASK_LOCK ; /* into critical reg */ \
testl $IRQ_BIT(irq_num), _apic_imen ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
je 7f ; /* bit clear, not masked */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
andl $~IRQ_BIT(irq_num), _apic_imen ;/* clear mask bit */ \
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
movl IOAPICADDR(irq_num),%ecx ; /* ioapic addr */ \
movl REDIRIDX(irq_num), %eax ; /* get the index */ \
movl %eax,(%ecx) ; /* write the index */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
movl IOAPIC_WINDOW(%ecx),%eax ; /* current value */ \
andl $~IOART_INTMASK,%eax ; /* clear the mask */ \
movl %eax,IOAPIC_WINDOW(%ecx) ; /* new value */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
7: ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
IMASK_UNLOCK
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
#ifdef INTR_SIMPLELOCK
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#define ENLOCK
#define DELOCK
#define LATELOCK call _get_isrlock
#else
#define ENLOCK \
ISR_TRYLOCK ; /* XXX this is going away... */ \
testl %eax, %eax ; /* did we get it? */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
jz 3f
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#define DELOCK ISR_RELLOCK
#define LATELOCK
#endif
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
#ifdef APIC_INTR_DIAGNOSTIC
#ifdef APIC_INTR_DIAGNOSTIC_IRQ
log_intr_event:
pushf
cli
pushl $CNAME(apic_itrace_debuglock)
call _s_lock_np
addl $4, %esp
movl CNAME(apic_itrace_debugbuffer_idx), %ecx
andl $32767, %ecx
movl _cpuid, %eax
shll $8, %eax
orl 8(%esp), %eax
movw %ax, CNAME(apic_itrace_debugbuffer)(,%ecx,2)
incl %ecx
andl $32767, %ecx
movl %ecx, CNAME(apic_itrace_debugbuffer_idx)
pushl $CNAME(apic_itrace_debuglock)
call _s_unlock_np
addl $4, %esp
popf
ret
#define APIC_ITRACE(name, irq_num, id) \
lock ; /* MP-safe */ \
incl CNAME(name) + (irq_num) * 4 ; \
pushl %eax ; \
pushl %ecx ; \
pushl %edx ; \
movl $(irq_num), %eax ; \
cmpl $APIC_INTR_DIAGNOSTIC_IRQ, %eax ; \
jne 7f ; \
pushl $id ; \
call log_intr_event ; \
addl $4, %esp ; \
7: ; \
popl %edx ; \
popl %ecx ; \
popl %eax
#else
#define APIC_ITRACE(name, irq_num, id) \
lock ; /* MP-safe */ \
incl CNAME(name) + (irq_num) * 4
#endif
#define APIC_ITRACE_ENTER 1
#define APIC_ITRACE_EOI 2
#define APIC_ITRACE_TRYISRLOCK 3
#define APIC_ITRACE_GOTISRLOCK 4
#define APIC_ITRACE_ENTER2 5
#define APIC_ITRACE_LEAVE 6
#define APIC_ITRACE_UNMASK 7
#define APIC_ITRACE_ACTIVE 8
#define APIC_ITRACE_MASKED 9
#define APIC_ITRACE_NOISRLOCK 10
#define APIC_ITRACE_MASKED2 11
#define APIC_ITRACE_SPLZ 12
#define APIC_ITRACE_DORETI 13
#else
#define APIC_ITRACE(name, irq_num, id)
#endif
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#ifdef CPL_AND_CML
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
#define INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/* _XintrNN: entry point used by IDT/HWIs & splz_unpend via _vec[]. */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
IDTVEC(vec_name) ; \
PUSH_FRAME ; \
movl $KDSEL, %eax ; /* reload with kernel's data segment */ \
movl %ax, %ds ; \
movl %ax, %es ; \
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_enter, irq_num, APIC_ITRACE_ENTER) ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
lock ; /* MP-safe */ \
btsl $(irq_num), iactive ; /* lazy masking */ \
jc 1f ; /* already active */ \
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
MASK_LEVEL_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
0: ; \
APIC_ITRACE(apic_itrace_tryisrlock, irq_num, APIC_ITRACE_TRYISRLOCK) ;\
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
ENLOCK ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_gotisrlock, irq_num, APIC_ITRACE_GOTISRLOCK) ;\
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_LOCK ; /* MP-safe */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
testl $IRQ_BIT(irq_num), _cpl ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
jne 2f ; /* this INT masked */ \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
testl $IRQ_BIT(irq_num), _cml ; \
jne 2f ; /* this INT masked */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
orl $IRQ_BIT(irq_num), _cil ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_UNLOCK ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
incb _intr_nesting_level ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
; \
/* entry point used by doreti_unpend for HWIs. */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
__CONCAT(Xresume,irq_num): ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
FAKE_MCOUNT(12*4(%esp)) ; /* XXX avoid dbl cnt */ \
lock ; incl _cnt+V_INTR ; /* tally interrupts */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; incl (%eax) ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
; \
AVCPL_LOCK ; /* MP-safe */ \
movl _cml, %eax ; \
pushl %eax ; \
orl _intr_mask + (irq_num) * 4, %eax ; \
movl %eax, _cml ; \
AVCPL_UNLOCK ; \
; \
pushl _intr_unit + (irq_num) * 4 ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
incl _inside_intr ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_enter2, irq_num, APIC_ITRACE_ENTER2) ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
sti ; \
call *_intr_handler + (irq_num) * 4 ; \
cli ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_leave, irq_num, APIC_ITRACE_LEAVE) ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
decl _inside_intr ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
; \
lock ; andl $~IRQ_BIT(irq_num), iactive ; \
lock ; andl $~IRQ_BIT(irq_num), _cil ; \
UNMASK_IRQ(irq_num) ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_unmask, irq_num, APIC_ITRACE_UNMASK) ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
sti ; /* doreti repeats cli/sti */ \
MEXITCOUNT ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
LATELOCK ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
jmp _doreti ; \
; \
ALIGN_TEXT ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
1: ; /* active */ \
APIC_ITRACE(apic_itrace_active, irq_num, APIC_ITRACE_ACTIVE) ; \
Mask the interrupt before setting the corresponding bit in ipending if the interrupt is already active. Don't use lock prefix for operations on ipending. Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
MASK_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
AVCPL_UNLOCK ; \
Mask the interrupt before setting the corresponding bit in ipending if the interrupt is already active. Don't use lock prefix for operations on ipending. Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
lock ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
btsl $(irq_num), iactive ; /* still active */ \
jnc 0b ; /* retry */ \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
POP_FRAME ; \
iret ; \
; \
ALIGN_TEXT ; \
2: ; /* masked by cpl|cml */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_masked, irq_num, APIC_ITRACE_MASKED) ; \
orl $IRQ_BIT(irq_num), _ipending ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
AVCPL_UNLOCK ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
DELOCK ; /* XXX this is going away... */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
POP_FRAME ; \
iret ; \
ALIGN_TEXT ; \
3: ; /* other cpu has isr lock */ \
APIC_ITRACE(apic_itrace_noisrlock, irq_num, APIC_ITRACE_NOISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 4f ; /* this INT masked */ \
testl $IRQ_BIT(irq_num), _cml ; \
jne 4f ; /* this INT masked */ \
orl $IRQ_BIT(irq_num), _cil ; \
AVCPL_UNLOCK ; \
call forward_irq ; /* forward irq to lock holder */ \
POP_FRAME ; /* and return */ \
iret ; \
ALIGN_TEXT ; \
4: ; /* blocked */ \
APIC_ITRACE(apic_itrace_masked2, irq_num, APIC_ITRACE_MASKED2) ;\
AVCPL_UNLOCK ; \
POP_FRAME ; /* and return */ \
iret
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#else /* CPL_AND_CML */
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
#define INTR(irq_num, vec_name) \
.text ; \
SUPERALIGN_TEXT ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/* _XintrNN: entry point used by IDT/HWIs & splz_unpend via _vec[]. */ \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
IDTVEC(vec_name) ; \
PUSH_FRAME ; \
movl $KDSEL, %eax ; /* reload with kernel's data segment */ \
movl %ax, %ds ; \
movl %ax, %es ; \
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_enter, irq_num, APIC_ITRACE_ENTER) ; \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
lock ; /* MP-safe */ \
btsl $(irq_num), iactive ; /* lazy masking */ \
jc 1f ; /* already active */ \
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
MASK_LEVEL_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
0: ; \
APIC_ITRACE(apic_itrace_tryisrlock, irq_num, APIC_ITRACE_TRYISRLOCK) ;\
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
ISR_TRYLOCK ; /* XXX this is going away... */ \
testl %eax, %eax ; /* did we get it? */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
jz 3f ; /* no */ \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_gotisrlock, irq_num, APIC_ITRACE_GOTISRLOCK) ;\
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
AVCPL_LOCK ; /* MP-safe */ \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 2f ; /* this INT masked */ \
AVCPL_UNLOCK ; \
; \
incb _intr_nesting_level ; \
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
; \
/* entry point used by doreti_unpend for HWIs. */ \
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
__CONCAT(Xresume,irq_num): ; \
FAKE_MCOUNT(12*4(%esp)) ; /* XXX avoid dbl cnt */ \
lock ; incl _cnt+V_INTR ; /* tally interrupts */ \
movl _intr_countp + (irq_num) * 4, %eax ; \
lock ; incl (%eax) ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_LOCK ; /* MP-safe */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
movl _cpl, %eax ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
pushl %eax ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
orl _intr_mask + (irq_num) * 4, %eax ; \
movl %eax, _cpl ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
andl $~IRQ_BIT(irq_num), _ipending ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_UNLOCK ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
pushl _intr_unit + (irq_num) * 4 ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_enter2, irq_num, APIC_ITRACE_ENTER2) ; \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
sti ; \
call *_intr_handler + (irq_num) * 4 ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
cli ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_leave, irq_num, APIC_ITRACE_LEAVE) ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
; \
lock ; andl $~IRQ_BIT(irq_num), iactive ; \
UNMASK_IRQ(irq_num) ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
APIC_ITRACE(apic_itrace_unmask, irq_num, APIC_ITRACE_UNMASK) ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
sti ; /* doreti repeats cli/sti */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
MEXITCOUNT ; \
jmp _doreti ; \
; \
ALIGN_TEXT ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
1: ; /* active */ \
APIC_ITRACE(apic_itrace_active, irq_num, APIC_ITRACE_ACTIVE) ; \
Mask the interrupt before setting the corresponding bit in ipending if the interrupt is already active. Don't use lock prefix for operations on ipending. Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
MASK_IRQ(irq_num) ; \
EOI_IRQ(irq_num) ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_LOCK ; /* MP-safe */ \
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
orl $IRQ_BIT(irq_num), _ipending ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_UNLOCK ; \
Mask the interrupt before setting the corresponding bit in ipending if the interrupt is already active. Don't use lock prefix for operations on ipending. Always use lock prefix for operations on iactive.
1998-04-22 22:49:29 +00:00
lock ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
btsl $(irq_num), iactive ; /* still active */ \
jnc 0b ; /* retry */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
POP_FRAME ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
iret ; /* XXX: iactive bit might be 0 now */ \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
ALIGN_TEXT ; \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
2: ; /* masked by cpl, leave iactive set */ \
APIC_ITRACE(apic_itrace_masked, irq_num, APIC_ITRACE_MASKED) ; \
orl $IRQ_BIT(irq_num), _ipending ; \
The last of the encapsolation of cpl/spl/ipending things into a critical region protected by the simplelock 'cpl_lock'. Notes: - this code is currently controlled on a section by section basis with defines in machine/param.h. All sections are currently enabled. - this code is not as clean as I would like, but that can wait till later. - the "giant lock" still surrounds most instances of this "cpl region". I still have to do the code that arbitrates setting cpl between the top and bottom halves of the kernel. - the possibility of deadlock exists, I am committing the code at this point so as to exercise it and detect any such cases B4 the "giant lock" is removed.
1997-08-24 00:05:37 +00:00
AVCPL_UNLOCK ; \
Made PEND_INTS default. Made NEW_STRATEGY default. Removed misc. old cruft. Centralized simple locks into mp_machdep.c Centralized simple lock macros into param.h More cleanup in the direction of making splxx()/cpl MP-safe.
1997-08-21 05:08:25 +00:00
ISR_RELLOCK ; /* XXX this is going away... */ \
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
POP_FRAME ; \
iret ; \
ALIGN_TEXT ; \
3: ; /* other cpu has isr lock */ \
APIC_ITRACE(apic_itrace_noisrlock, irq_num, APIC_ITRACE_NOISRLOCK) ;\
AVCPL_LOCK ; /* MP-safe */ \
orl $IRQ_BIT(irq_num), _ipending ; \
testl $IRQ_BIT(irq_num), _cpl ; \
jne 4f ; /* this INT masked */ \
AVCPL_UNLOCK ; \
call forward_irq ; /* forward irq to lock holder */ \
POP_FRAME ; /* and return */ \
iret ; \
ALIGN_TEXT ; \
4: ; /* blocked */ \
APIC_ITRACE(apic_itrace_masked2, irq_num, APIC_ITRACE_MASKED2) ;\
AVCPL_UNLOCK ; \
POP_FRAME ; /* and return */ \
iret
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#endif /* CPL_AND_CML */
Support for the new FAST_HI algorithm, enabled. Preliminary support for the INTR_SIMPLELOCK algorithm, disabled. Note that this code is NOT ready.
1997-08-29 18:45:23 +00:00
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
/*
* Handle "spurious INTerrupts".
* Notes:
* This is different than the "spurious INTerrupt" generated by an
* 8259 PIC for missing INTs. See the APIC documentation for details.
* This routine should NOT do an 'EOI' cycle.
*/
.text
SUPERALIGN_TEXT
.globl _Xspuriousint
_Xspuriousint:
/* No EOI cycle used here */
iret
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
/*
* Handle TLB shootdowns.
*/
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
.text
SUPERALIGN_TEXT
.globl _Xinvltlb
_Xinvltlb:
pushl %eax
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
#ifdef COUNT_XINVLTLB_HITS
ss
movl _cpuid, %eax
ss
incl _xhits(,%eax,4)
#endif /* COUNT_XINVLTLB_HITS */
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
movl %cr3, %eax /* invalidate the TLB */
movl %eax, %cr3
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
ss /* stack segment, avoid %ds load */
Tighten up asm code for EOI access.
1997-07-15 02:49:21 +00:00
movl $0, lapic_eoi /* End Of Interrupt to APIC */
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
popl %eax
iret
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
#ifdef BETTER_CLOCK
/*
* Executed by a CPU when it receives an Xcpucheckstate IPI from another CPU,
*
* - Stores current cpu state in checkstate_cpustate[cpuid]
* 0 == user, 1 == sys, 2 == intr
* - Stores current process in checkstate_curproc[cpuid]
*
* - Signals its receipt by setting bit cpuid in checkstate_probed_cpus.
*
* stack: 0 -> ds, 4 -> ebx, 8 -> eax, 12 -> eip, 16 -> cs, 20 -> eflags
*/
.text
SUPERALIGN_TEXT
.globl _Xcpucheckstate
.globl _checkstate_cpustate
.globl _checkstate_curproc
.globl _checkstate_pc
_Xcpucheckstate:
pushl %eax
pushl %ebx
pushl %ds /* save current data segment */
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl $0, lapic_eoi /* End Of Interrupt to APIC */
movl $0, %ebx
movl 16(%esp), %eax
andl $3, %eax
cmpl $3, %eax
je 1f
#ifdef VM86
testl $PSL_VM, 20(%esp)
jne 1f
#endif
incl %ebx /* system or interrupt */
#ifdef CPL_AND_CML
cmpl $0, _inside_intr
je 1f
incl %ebx /* interrupt */
#endif
1:
movl _cpuid, %eax
movl %ebx, _checkstate_cpustate(,%eax,4)
movl _curproc, %ebx
movl %ebx, _checkstate_curproc(,%eax,4)
movl 12(%esp), %ebx
movl %ebx, _checkstate_pc(,%eax,4)
lock /* checkstate_probed_cpus |= (1<<id) */
btsl %eax, _checkstate_probed_cpus
popl %ds /* restore previous data segment */
popl %ebx
popl %eax
iret
Forward the signal if the process runs on a different CPU. This reduces the signal handling latency for cpu-bound processes that performs very few system calls. The IPI for forcing an additional software trap is no longer dependent upon BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
#endif /* BETTER_CLOCK */
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
/*
* Executed by a CPU when it receives an Xcpuast IPI from another CPU,
*
* - Signals its receipt by clearing bit cpuid in checkstate_need_ast.
*
* - We need a better method of triggering asts on other cpus.
*/
.text
SUPERALIGN_TEXT
.globl _Xcpuast
_Xcpuast:
PUSH_FRAME
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl %ax, %es
movl _cpuid, %eax
lock /* checkstate_need_ast &= ~(1<<id) */
btrl %eax, _checkstate_need_ast
movl $0, lapic_eoi /* End Of Interrupt to APIC */
lock
btsl %eax, _checkstate_pending_ast
jc 1f
FAKE_MCOUNT(12*4(%esp))
/*
* Giant locks do not come cheap.
* A lot of cycles are going to be wasted here.
*/
call _get_isrlock
AVCPL_LOCK
#ifdef CPL_AND_CML
movl _cml, %eax
#else
movl _cpl, %eax
#endif
pushl %eax
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
orl $SWI_AST_PENDING, _ipending
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
AVCPL_UNLOCK
lock
incb _intr_nesting_level
sti
pushl $0
movl _cpuid, %eax
lock
btrl %eax, _checkstate_pending_ast
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
lock
btrl %eax, CNAME(resched_cpus)
jz 2f
movl $1, CNAME(want_resched)
lock
incl CNAME(want_resched_cnt)
2:
lock
incl CNAME(cpuast_cnt)
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
MEXITCOUNT
jmp _doreti
1:
/* We are already in the process of delivering an ast for this CPU */
POP_FRAME
iret
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
/*
* Executed by a CPU when it receives an XFORWARD_IRQ IPI.
*/
.text
SUPERALIGN_TEXT
.globl _Xforward_irq
_Xforward_irq:
PUSH_FRAME
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
movl %ax, %es
movl $0, lapic_eoi /* End Of Interrupt to APIC */
FAKE_MCOUNT(12*4(%esp))
ISR_TRYLOCK
testl %eax,%eax /* Did we get the lock ? */
jz 1f /* No */
lock
incl CNAME(forward_irq_hitcnt)
cmpb $4, _intr_nesting_level
jae 2f
AVCPL_LOCK
#ifdef CPL_AND_CML
movl _cml, %eax
#else
movl _cpl, %eax
#endif
pushl %eax
AVCPL_UNLOCK
lock
incb _intr_nesting_level
sti
pushl $0
MEXITCOUNT
jmp _doreti /* Handle forwarded interrupt */
1:
lock
incl CNAME(forward_irq_misscnt)
call forward_irq /* Oops, we've lost the isr lock */
MEXITCOUNT
POP_FRAME
iret
2:
lock
incl CNAME(forward_irq_toodeepcnt)
3:
ISR_RELLOCK
MEXITCOUNT
POP_FRAME
iret
/*
*
*/
forward_irq:
MCOUNT
cmpl $0,_invltlb_ok
jz 4f
cmpl $0, CNAME(forward_irq_enabled)
jz 4f
movl _mp_lock,%eax
cmpl $FREE_LOCK,%eax
jne 1f
movl $0, %eax /* Pick CPU #0 if noone has lock */
1:
shrl $24,%eax
movl _cpu_num_to_apic_id(,%eax,4),%ecx
shll $24,%ecx
movl lapic_icr_hi, %eax
andl $~APIC_ID_MASK, %eax
orl %ecx, %eax
movl %eax, lapic_icr_hi
2:
movl lapic_icr_lo, %eax
andl $APIC_DELSTAT_MASK,%eax
jnz 2b
movl lapic_icr_lo, %eax
andl $APIC_RESV2_MASK, %eax
orl $(APIC_DEST_DESTFLD|APIC_DELMODE_FIXED|XFORWARD_IRQ_OFFSET), %eax
movl %eax, lapic_icr_lo
3:
movl lapic_icr_lo, %eax
andl $APIC_DELSTAT_MASK,%eax
jnz 3b
4:
ret
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
/*
* Executed by a CPU when it receives an Xcpustop IPI from another CPU,
*
* - Signals its receipt.
* - Waits for permission to restart.
* - Signals its restart.
*/
.text
SUPERALIGN_TEXT
.globl _Xcpustop
_Xcpustop:
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
pushl %ebp
movl %esp, %ebp
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
pushl %eax
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
pushl %ecx
pushl %edx
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
pushl %ds /* save current data segment */
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
pushl %es
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
movl $KDSEL, %eax
movl %ax, %ds /* use KERNEL data segment */
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
movl $0, lapic_eoi /* End Of Interrupt to APIC */
movl _cpuid, %eax
imull $PCB_SIZE, %eax
leal CNAME(stoppcbs)(%eax), %eax
pushl %eax
call CNAME(savectx) /* Save process context */
addl $4, %esp
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
movl _cpuid, %eax
Cleanup old stop_cpus/restart_cpus() cruft. new code for handling mixed-mode 8259/APIC programming without 'ExtInt' new code to control other CPUs: stop_cpus()/restart_cpus()/_Xstopcpu
1997-07-13 01:18:51 +00:00
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
lock
btsl %eax, _stopped_cpus /* stopped_cpus |= (1<<id) */
1:
btl %eax, _started_cpus /* while (!(started_cpus & (1<<id))) */
jnc 1b
lock
btrl %eax, _started_cpus /* started_cpus &= ~(1<<id) */
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
lock
btrl %eax, _stopped_cpus /* stopped_cpus &= ~(1<<id) */
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
test %eax, %eax
jnz 2f
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
movl CNAME(cpustop_restartfunc), %eax
test %eax, %eax
jz 2f
movl $0, CNAME(cpustop_restartfunc) /* One-shot */
call %eax
2:
popl %es
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
popl %ds /* restore previous data segment */
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
popl %edx
popl %ecx
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
popl %eax
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
movl %ebp, %esp
popl %ebp
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
iret
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
MCOUNT_LABEL(bintr)
FAST_INTR(0,fastintr0)
FAST_INTR(1,fastintr1)
FAST_INTR(2,fastintr2)
FAST_INTR(3,fastintr3)
FAST_INTR(4,fastintr4)
FAST_INTR(5,fastintr5)
FAST_INTR(6,fastintr6)
FAST_INTR(7,fastintr7)
FAST_INTR(8,fastintr8)
FAST_INTR(9,fastintr9)
FAST_INTR(10,fastintr10)
FAST_INTR(11,fastintr11)
FAST_INTR(12,fastintr12)
FAST_INTR(13,fastintr13)
FAST_INTR(14,fastintr14)
FAST_INTR(15,fastintr15)
FAST_INTR(16,fastintr16)
FAST_INTR(17,fastintr17)
FAST_INTR(18,fastintr18)
FAST_INTR(19,fastintr19)
FAST_INTR(20,fastintr20)
FAST_INTR(21,fastintr21)
FAST_INTR(22,fastintr22)
FAST_INTR(23,fastintr23)
INTR(0,intr0)
INTR(1,intr1)
INTR(2,intr2)
INTR(3,intr3)
INTR(4,intr4)
INTR(5,intr5)
INTR(6,intr6)
INTR(7,intr7)
INTR(8,intr8)
INTR(9,intr9)
INTR(10,intr10)
INTR(11,intr11)
INTR(12,intr12)
INTR(13,intr13)
INTR(14,intr14)
INTR(15,intr15)
INTR(16,intr16)
INTR(17,intr17)
INTR(18,intr18)
INTR(19,intr19)
INTR(20,intr20)
INTR(21,intr21)
INTR(22,intr22)
INTR(23,intr23)
MCOUNT_LABEL(eintr)
.data
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/*
* Addresses of interrupt handlers.
* XresumeNN: Resumption addresses for HWIs.
*/
aha1542.c aic6360.c cy.c fd.c ft.c if_ie.c if_wl.c if_zp.c isa.c isa_device.h labpc.c mcd.c ncr5380.c scd.c seagate.c si.c sio.c tw.c ultra14f.c wcd.c wd.c: Update for changes in the callout interface. apic_vector.s icu_vector.s ipl.s ipl_funcs.c: Add CAM software/hardware interrupt support.
1997-09-21 21:41:49 +00:00
.globl _ihandlers
_ihandlers:
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/*
* used by:
* ipl.s: doreti_unpend
*/
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
.long Xresume0, Xresume1, Xresume2, Xresume3
.long Xresume4, Xresume5, Xresume6, Xresume7
.long Xresume8, Xresume9, Xresume10, Xresume11
.long Xresume12, Xresume13, Xresume14, Xresume15
.long Xresume16, Xresume17, Xresume18, Xresume19
.long Xresume20, Xresume21, Xresume22, Xresume23
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/*
* used by:
* ipl.s: doreti_unpend
* apic_ipl.s: splz_unpend
*/
Register tty software interrupt handlers at run time using register_swi() instead of at compile time using ifdefs. Use _swi_null instead of dummycamisr. CAM and dpt should call register_swi() instead of hacking on ihandlers[] directly.
1998-08-11 17:01:32 +00:00
.long _swi_null, swi_net, _swi_null, _swi_null
Implemented dynamic registration of software interrupt handlers. Not used yet. Use dummy SWI handlers to avoid some checks for null pointers.
1998-08-11 15:08:13 +00:00
.long _swi_vm, _swi_null, _softclock, swi_ast
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
imasks: /* masks for interrupt handlers */
.space NHWI*4 /* padding; HWI masks are elsewhere */
Implemented dynamic registration of software interrupt handlers. Not used yet. Use dummy SWI handlers to avoid some checks for null pointers.
1998-08-11 15:08:13 +00:00
.long SWI_TTY_MASK, SWI_NET_MASK, SWI_CAMNET_MASK, SWI_CAMBIO_MASK
.long SWI_VM_MASK, 0, SWI_CLOCK_MASK, SWI_AST_MASK
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/* active flag for lazy masking */
iactive:
.long 0
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
#ifdef COUNT_XINVLTLB_HITS
.globl _xhits
_xhits:
Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code.
1997-07-18 21:27:53 +00:00
.space (NCPU * 4), 0
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
#endif /* COUNT_XINVLTLB_HITS */
Split TEST_CPUSTOP code into CPUSTOP_ON_DDBBREAK and mainline code.
1997-07-18 21:27:53 +00:00
/* variables used by stop_cpus()/restart_cpus()/Xcpustop */
.globl _stopped_cpus, _started_cpus
apic_vector.s: - added Xcpustop IPI code to support stop_cpus()/restart_cpus(). it is off by default, enable via smptests.h:TEST_CPUSTOP intr_machdep.h: - moved +ICULEN to lower level. - added entry for Xcpustop.
1997-06-27 23:48:05 +00:00
_stopped_cpus:
.long 0
_started_cpus:
.long 0
First cut at code for handling "spurious INTerrupts". First cut at code for handling CPU stop/restart. Notes: not working properly yet.
1997-07-06 23:32:38 +00:00
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
#ifdef BETTER_CLOCK
.globl _checkstate_probed_cpus
_checkstate_probed_cpus:
.long 0
Forward the signal if the process runs on a different CPU. This reduces the signal handling latency for cpu-bound processes that performs very few system calls. The IPI for forcing an additional software trap is no longer dependent upon BETTER_CLOCK being defined.
1998-03-03 20:55:26 +00:00
#endif /* BETTER_CLOCK */
.globl _checkstate_need_ast
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
_checkstate_need_ast:
.long 0
_checkstate_pending_ast:
.long 0
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
.globl CNAME(forward_irq_misscnt)
.globl CNAME(forward_irq_toodeepcnt)
.globl CNAME(forward_irq_hitcnt)
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
.globl CNAME(resched_cpus)
.globl CNAME(want_resched_cnt)
.globl CNAME(cpuast_cnt)
.globl CNAME(cpustop_restartfunc)
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
CNAME(forward_irq_misscnt):
.long 0
CNAME(forward_irq_hitcnt):
.long 0
CNAME(forward_irq_toodeepcnt):
.long 0
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
CNAME(resched_cpus):
.long 0
CNAME(want_resched_cnt):
.long 0
CNAME(cpuast_cnt):
.long 0
CNAME(cpustop_restartfunc):
.long 0
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
The improvements to clock statistics by Tor Egge Wrappered and enabled by the define BETTER_CLOCK (on by default in smpyests.h) apic_vector.s also contains a small change I (smp) made to eliminate the double level INT problem. It seems stable, but I haven't the tools in place to prove it fixes the problem. Reviewed by: smp@csn.net Submitted by: Tor Egge <Tor.Egge@idi.ntnu.no>
1997-12-08 22:59:39 +00:00
Modified the PEND_INTS algorithm to fix the ISA INT loss problem. Noticed by: dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
.globl _apic_pin_trigger
_apic_pin_trigger:
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
.long 0
Modified the PEND_INTS algorithm to fix the ISA INT loss problem. Noticed by: dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
Split vector.s into UP and SMP specific files: - vector.s <- stub called by i386/exception.s - icu_vector.s <- UP - apic_vector.s <- SMP Split icu.s into UP and SMP specific files: - ipl.s <- stub called by i386/exception.s (formerly icu.s) - icu_ipl.s <- UP - apic_ipl.s <- SMP This was done in preparation for massive changes to the SMP INTerrupt mechanisms. More fine tuning, such as merging ipl.s into exception.s, may be appropriate.
1997-05-26 17:58:27 +00:00
/*
* Interrupt counters and names. The format of these and the label names
* must agree with what vmstat expects. The tables are indexed by device
* ids so that we don't have to move the names around as devices are
* attached.
*/
#include "vector.h"
.globl _intrcnt, _eintrcnt
_intrcnt:
.space (NR_DEVICES + ICU_LEN) * 4
_eintrcnt:
.globl _intrnames, _eintrnames
_intrnames:
.ascii DEVICE_NAMES
.asciz "stray irq0"
.asciz "stray irq1"
.asciz "stray irq2"
.asciz "stray irq3"
.asciz "stray irq4"
.asciz "stray irq5"
.asciz "stray irq6"
.asciz "stray irq7"
.asciz "stray irq8"
.asciz "stray irq9"
.asciz "stray irq10"
.asciz "stray irq11"
.asciz "stray irq12"
.asciz "stray irq13"
.asciz "stray irq14"
.asciz "stray irq15"
.asciz "stray irq16"
.asciz "stray irq17"
.asciz "stray irq18"
.asciz "stray irq19"
.asciz "stray irq20"
.asciz "stray irq21"
.asciz "stray irq22"
.asciz "stray irq23"
_eintrnames:
.text
Reference in New Issue Copy Permalink