d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
3b0f6960df
freebsd-dev/sys/amd64/include/smp.h

150 lines
4.1 KiB
C
Raw Normal View History

Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/*
* ----------------------------------------------------------------------------
* "THE BEER-WARE LICENSE" (Revision 42):
* <phk@FreeBSD.org> wrote this file. As long as you retain this notice you
* can do whatever you want with this stuff. If we meet some day, and you think
* this stuff is worth it, you can buy me a beer in return. Poul-Henning Kamp
* ----------------------------------------------------------------------------
*
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
* $FreeBSD$
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
*
*/
#ifndef _MACHINE_SMP_H_
#define _MACHINE_SMP_H_
Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL" is an application space macro and the applications are supposed to be free to use it as they please (but cannot). This is consistant with the other BSD's who made this change quite some time ago. More commits to come.
1999-12-29 04:46:21 +00:00
#ifdef _KERNEL
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
Make LINT compile.
2000-09-16 18:55:05 +00:00
#if defined(SMP) && defined(I386_CPU) && !defined(COMPILING_LINT)
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
#error SMP not supported with I386_CPU
#endif
Force user to config SMP kernel with "options APIC_IO". Reviewed by: Peter Wemm <peter@spinner.DIALix.COM>
1997-05-06 21:29:57 +00:00
#if defined(SMP) && !defined(APIC_IO)
# error APIC_IO required for SMP, add "options APIC_IO" to your config file.
Preliminaries for stop_cpus()/restart_cpus(). Both are turned off by default. Added macro for displaying POST codes from kernel.
1997-06-27 23:12:31 +00:00
#endif /* SMP && !APIC_IO */
Force user to config SMP kernel with "options APIC_IO". Reviewed by: Peter Wemm <peter@spinner.DIALix.COM>
1997-05-06 21:29:57 +00:00
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
#if defined(SMP) || defined(APIC_IO)
Modified the PEND_INTS algorithm to fix the ISA INT loss problem. Noticed by: dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
#ifndef LOCORE
Preliminaries for stop_cpus()/restart_cpus(). Both are turned off by default. Added macro for displaying POST codes from kernel.
1997-06-27 23:12:31 +00:00
/*
* For sending values to POST displays.
Additional debugging functions and macros. "spurious INTerrupt" support.
1997-07-06 23:40:15 +00:00
* XXX FIXME: where does this really belong, isa.h/isa.c perhaps?
Preliminaries for stop_cpus()/restart_cpus(). Both are turned off by default. Added macro for displaying POST codes from kernel.
1997-06-27 23:12:31 +00:00
*/
Additional debugging functions and macros. "spurious INTerrupt" support.
1997-07-06 23:40:15 +00:00
extern int current_postcode; /** XXX currently in mp_machdep.c */
#define POSTCODE(X) current_postcode = (X), \
outb(0x80, current_postcode)
#define POSTCODE_LO(X) current_postcode &= 0xf0, \
current_postcode |= ((X) & 0x0f), \
outb(0x80, current_postcode)
#define POSTCODE_HI(X) current_postcode &= 0x0f, \
current_postcode |= (((X) << 4) & 0xf0), \
outb(0x80, current_postcode)
Preliminaries for stop_cpus()/restart_cpus(). Both are turned off by default. Added macro for displaying POST codes from kernel.
1997-06-27 23:12:31 +00:00
Overhaul of the SMP code. Several portions of the SMP kernel support have been made machine independent and various other adjustments have been made to support Alpha SMP. - It splits the per-process portions of hardclock() and statclock() off into hardclock_process() and statclock_process() respectively. hardclock() and statclock() call the *_process() functions for the current process so that UP systems will run as before. For SMP systems, it is simply necessary to ensure that all other processors execute the *_process() functions when the main clock functions are triggered on one CPU by an interrupt. For the alpha 4100, clock interrupts are delievered in a staggered broadcast fashion, so we simply call hardclock/statclock on the boot CPU and call the *_process() functions on the secondaries. For x86, we call statclock and hardclock as usual and then call forward_hardclock/statclock in the MD code to send an IPI to cause the AP's to execute forwared_hardclock/statclock which then call the *_process() functions. - forward_signal() and forward_roundrobin() have been reworked to be MI and to involve less hackery. Now the cpu doing the forward sets any flags, etc. and sends a very simple IPI_AST to the other cpu(s). AST IPIs now just basically return so that they can execute ast() and don't bother with setting the astpending or needresched flags themselves. This also removes the loop in forward_signal() as sched_lock closes the race condition that the loop worked around. - need_resched(), resched_wanted() and clear_resched() have been changed to take a process to act on rather than assuming curproc so that they can be used to implement forward_roundrobin() as described above. - Various other SMP variables have been moved to a MI subr_smp.c and a new header sys/smp.h declares MI SMP variables and API's. The IPI API's from machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h. - The globaldata_register() and globaldata_find() functions as well as the SLIST of globaldata structures has become MI and moved into subr_smp.c. Also, the globaldata list is only available if SMP support is compiled in. Reviewed by: jake, peter Looked over by: eivind
2001-04-27 19:28:25 +00:00
#include <sys/bus.h> /* XXX */
apic.h now has structure definitions for both the local APIC and io APIC. apic.h has defines like: #define lapic__id lapic->id Once private pages and "known virtual addr" mapping of the APICs is ready all 'lapic__XXX' will be changed to 'lapic.XXX', and the defines will be removed. Changes to smp.h for lapic_t lapic && ioapic_t ioapic pointers, currently equal to apic_base && io_apic_base, will stand alone with the private page mapping.
1997-05-29 05:57:43 +00:00
#include <machine/apic.h>
Overhaul of the SMP code. Several portions of the SMP kernel support have been made machine independent and various other adjustments have been made to support Alpha SMP. - It splits the per-process portions of hardclock() and statclock() off into hardclock_process() and statclock_process() respectively. hardclock() and statclock() call the *_process() functions for the current process so that UP systems will run as before. For SMP systems, it is simply necessary to ensure that all other processors execute the *_process() functions when the main clock functions are triggered on one CPU by an interrupt. For the alpha 4100, clock interrupts are delievered in a staggered broadcast fashion, so we simply call hardclock/statclock on the boot CPU and call the *_process() functions on the secondaries. For x86, we call statclock and hardclock as usual and then call forward_hardclock/statclock in the MD code to send an IPI to cause the AP's to execute forwared_hardclock/statclock which then call the *_process() functions. - forward_signal() and forward_roundrobin() have been reworked to be MI and to involve less hackery. Now the cpu doing the forward sets any flags, etc. and sends a very simple IPI_AST to the other cpu(s). AST IPIs now just basically return so that they can execute ast() and don't bother with setting the astpending or needresched flags themselves. This also removes the loop in forward_signal() as sched_lock closes the race condition that the loop worked around. - need_resched(), resched_wanted() and clear_resched() have been changed to take a process to act on rather than assuming curproc so that they can be used to implement forward_roundrobin() as described above. - Various other SMP variables have been moved to a MI subr_smp.c and a new header sys/smp.h declares MI SMP variables and API's. The IPI API's from machine/ipl.h have moved to machine/smp.h which is included by sys/smp.h. - The globaldata_register() and globaldata_find() functions as well as the SLIST of globaldata structures has become MI and moved into subr_smp.c. Also, the globaldata list is only available if SMP support is compiled in. Reviewed by: jake, peter Looked over by: eivind
2001-04-27 19:28:25 +00:00
#include <machine/frame.h>
#include <i386/isa/icu.h>
#include <i386/isa/intr_machdep.h>
/*
* Interprocessor interrupts for SMP.
*/
#define IPI_INVLTLB XINVLTLB_OFFSET
#define IPI_RENDEZVOUS XRENDEZVOUS_OFFSET
#define IPI_AST XCPUAST_OFFSET
#define IPI_STOP XCPUSTOP_OFFSET
#define IPI_HARDCLOCK XHARDCLOCK_OFFSET
#define IPI_STATCLOCK XSTATCLOCK_OFFSET
apic.h now has structure definitions for both the local APIC and io APIC. apic.h has defines like: #define lapic__id lapic->id Once private pages and "known virtual addr" mapping of the APICs is ready all 'lapic__XXX' will be changed to 'lapic.XXX', and the defines will be removed. Changes to smp.h for lapic_t lapic && ioapic_t ioapic pointers, currently equal to apic_base && io_apic_base, will stand alone with the private page mapping.
1997-05-29 05:57:43 +00:00
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/* global data in mpboot.s */
extern int bootMP_size;
/* functions in mpboot.s */
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 15:09:35 +00:00
void bootMP(void);
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/* global data in mp_machdep.c */
A clean fix for the spl "deadlock before smp_active" problem. Added a new variable, 'bsp_apic_ready', which is set as soon as the bootstrap CPU has initialized its local APIC. Conditionalize the GENSPLR functions to call ss_lock ONLY after bsp_apic_ready is TRUE; This should prevent any problems with races between the time the 1st AP becomes ready and the time smp_active is set.
1997-08-24 20:33:32 +00:00
extern int bsp_apic_ready;
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
extern int mp_naps;
extern int mp_nbusses;
extern int mp_napics;
extern int mp_picmode;
extern int boot_cpu_id;
extern vm_offset_t cpu_apic_address;
extern vm_offset_t io_apic_address[];
extern u_int32_t cpu_apic_versions[];
Remove the NCPU, NAPIC, NBUS, NINTR config options. Make NAPIC, NBUS, NINTR dynamic and set NCPU to a maximum of 16 under SMP. Reviewed by: peter
2000-09-22 23:40:10 +00:00
extern u_int32_t *io_apic_versions;
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
extern int cpu_num_to_apic_id[];
extern int io_num_to_apic_id[];
extern int apic_id_to_logical[];
Cleanup some leftover lint from the old interrupt system. Also, while here, run up to 32 interrupt sources on APIC systems. Normalize INTREN/INTRDIS so they are the same on both UP and SMP systems rather than sometimes a macro, and sometimes a function. Reviewed by: jhb, jakeb
2000-12-04 21:15:14 +00:00
#define APIC_INTMAPSIZE 32
Maintain a mapping from irq number to (ioapic number, int pin) tuple, and use this when masking/unmasking interrupts. Maintain a mapping from (iopaic number, int pin) tuple to irq number, and use this when configuring devices and programming the ioapics. Previous code assumed that irq number was equal to int pin number, and that the ioapic number was 0. Don't let an AP enter _cpu_switch before all local apics are initialized.
1998-09-06 22:41:42 +00:00
struct apic_intmapinfo {
int ioapic;
int int_pin;
volatile void *apic_address;
int redirindex;
};
extern struct apic_intmapinfo int_to_apicintpin[];
Add forwarding of roundrobin to other cpus. This gives a more regular update of cpu usage as shown by top when one process is cpu bound (no system calls) while the system is otherwise idle (except for top). Don't attempt to switch to the BSP in boot(). If the system was idle when an interrupt caused a panic, this won't work. Instead, switch to the BSP in cpu_reset. Remove some spurious forward_statclock/forward_hardclock warnings.
1998-05-17 22:12:14 +00:00
extern struct pcb stoppcbs[];
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/* functions in mp_machdep.c */
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 15:09:35 +00:00
void i386_mp_probe(void);
u_int mp_bootaddress(u_int);
u_int isa_apic_mask(u_int);
int isa_apic_irq(int);
int pci_apic_irq(int, int, int);
int apic_irq(int, int);
int next_apic_irq(int);
int undirect_isa_irq(int);
int undirect_pci_irq(int);
int apic_bus_type(int);
int apic_src_bus_id(int, int);
int apic_src_bus_irq(int, int);
int apic_int_type(int, int);
int apic_trigger(int, int);
int apic_polarity(int, int);
void assign_apic_irq(int apic, int intpin, int irq);
void revoke_apic_irq(int irq);
void bsp_apic_configure(void);
void init_secondary(void);
void smp_invltlb(void);
void forward_statclock(void);
void forwarded_statclock(struct trapframe frame);
void forward_hardclock(void);
void forwarded_hardclock(struct trapframe frame);
void ipi_selected(u_int cpus, u_int ipi);
void ipi_all(u_int ipi);
void ipi_all_but_self(u_int ipi);
void ipi_self(u_int ipi);
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
#ifdef APIC_INTR_REORDER
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 15:09:35 +00:00
void set_lapic_isrloc(int, int);
When entering the apic version of slow interrupt handler, level interrupts are masked, and EOI is sent iff the corresponding ISR bit is set in the local apic. If the CPU cannot obtain the interrupt service lock (currently the global kernel lock) the interrupt is forwarded to the CPU holding that lock. Clock interrupts now have higher priority than other slow interrupts.
1998-03-03 22:56:30 +00:00
#endif /* APIC_INTR_REORDER */
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/* global data in mpapic.c */
Preliminary support for per-cpu data pages. This eliminates a lot of #ifdef SMP type code. Things like _curproc reside in a data page that is unique on each cpu, eliminating the expensive macros like: #define curproc (SMPcurproc[cpunumber()]) There are some unresolved bootstrap and address space sharing issues at present, but Steve is waiting on this for other work. There is still some strictly temporary code present that isn't exactly pretty. This is part of a larger change that has run into some bumps, this part is standalone so it should be safe. The temporary code goes away when the full idle cpu support is finished. Reviewed by: fsmp, dyson
1997-06-22 16:04:22 +00:00
extern volatile lapic_t lapic;
Remove the NCPU, NAPIC, NBUS, NINTR config options. Make NAPIC, NBUS, NINTR dynamic and set NCPU to a maximum of 16 under SMP. Reviewed by: peter
2000-09-22 23:40:10 +00:00
extern volatile ioapic_t **ioapic;
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
/* functions in mpapic.c */
Fixed some style bugs in the removal of __P(()). The main ones were not removing tabs before "__P((", and not outdenting continuation lines to preserve non-KNF lining up of code with parentheses. Switch to KNF formatting and/or rewrap the whole prototype in some cases.
2002-03-23 15:09:35 +00:00
void apic_dump(char*);
void apic_initialize(void);
void imen_dump(void);
int apic_ipi(int, int, int);
int selected_apic_ipi(u_int, int, int);
int io_apic_setup(int);
void io_apic_setup_intpin(int, int);
void io_apic_set_id(int, int);
int io_apic_get_id(int);
int ext_int_setup(int, int);
void set_apic_timer(int);
int read_apic_timer(void);
void u_sleep(int);
u_int io_apic_read(int, int);
void io_apic_write(int, int, u_int);
Move io_apic_{read,write} from apic_ipl.s (where they do not belong) into mpapic.c. This gives us the benefit of C type checking. These functions are not called in any critical paths and are not used by the interrupt routines.
2000-12-06 01:04:02 +00:00
Modified the PEND_INTS algorithm to fix the ISA INT loss problem. Noticed by: dave adkins <adkin003@gold.tc.umn.edu> and others.
1997-07-28 03:59:54 +00:00
#endif /* !LOCORE */
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
#endif /* SMP && !APIC_IO */
The SMP cleanup commit broke UP compiles. Make UP compiles work again.
2000-03-28 18:06:49 +00:00
Change #ifdef KERNEL to #ifdef _KERNEL in the public headers. "KERNEL" is an application space macro and the applications are supposed to be free to use it as they please (but cannot). This is consistant with the other BSD's who made this change quite some time ago. More commits to come.
1999-12-29 04:46:21 +00:00
#endif /* _KERNEL */
Man the liferafts! Here comes the long awaited SMP -> -current merge! There are various options documented in i386/conf/LINT, there is more to come over the next few days. The kernel should run pretty much "as before" without the options to activate SMP mode. There are a handful of known "loose ends" that need to be fixed, but have been put off since the SMP kernel is in a moderately good condition at the moment. This commit is the result of the tinkering and testing over the last 14 months by many people. A special thanks to Steve Passe for implementing the APIC code!
1997-04-26 11:46:25 +00:00
#endif /* _MACHINE_SMP_H_ */
Reference in New Issue Copy Permalink