d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
b40a504727
freebsd-dev/sys/sparc64/include/smp.h

308 lines
6.7 KiB
C
Raw Normal View History

Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
/*-
* Copyright (c) 2001 Jake Burkholder.
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
The author isn't a [UC] Regents. Correct the copyright language.
2001-08-09 02:09:34 +00:00
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
The author isn't a [UC] Regents. Correct the copyright language.
2001-08-09 02:09:34 +00:00
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $FreeBSD$
*/
#ifndef _MACHINE_SMP_H_
#define _MACHINE_SMP_H_
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#ifdef SMP
- USIII-based machines can consist of CPUs running at different frequencies (and having different cache sizes) so use the STICK (System TICK) timer, which was introduced due to this and is driven by the same frequency across all CPUs, instead of the TICK timer, whose frequency varies with the CPU clock, to drive hardclock. We try to use the STICK counter with all CPUs that are USIII or beyond, even when not necessary due to identical CPUs, as we can can also avoid the workaround for the BlackBird erratum #1 there. Unfortunately, using the STICK counter currently causes a hang with USIIIi MP machines for reasons unknown, so we still use the TICK timer there (which is okay as they can only consist of identical CPUs). - Given that we only (try to) synchronize the (S)TICK timers of APs with the BSP during startup, we could end up spinning forever in DELAY(9) if that function is migrated to another CPU while we're spinning due to clock drift afterwards, so pin to the CPU in order to avoid migration. Unfortunately, pinning doesn't work at the point DELAY(9) is required by the low-level console drivers, yet, so switch to a function pointer, which is updated accordingly, for implementing DELAY(9). For USIII and beyond, this would also allow to easily use the STICK counter instead of the TICK one here, there's no benefit in doing so however. While at it, use cpu_spinwait(9) for spinning in the delay- functions. This currently is a NOP though. - Don't set the TICK timer of the BSP to 0 during at startup as there's no need to do so. - Implement cpu_est_clockrate(). - Unfortunately, USIIIi-based machines don't provide a timecounter device besides the STICK and TICK counters (well, in theory the Tomatillo bridges have a performance counter that can be (ab)used as timecounter by configuring it to count bus cycles, though unlike the performance counter of Schizo bridges, the Tomatillo one is broken and counts Sun knows what in this mode). This means that we've to use a (S)TICK counter for timecounting, which has the old problem of not being in sync across CPUs, so provide an additional timecounter function which binds itself to the BSP but has an adequate low priority.
2008-09-03 17:39:19 +00:00
#define CPU_TICKSYNC 1
#define CPU_STICKSYNC 2
#define CPU_INIT 3
#define CPU_BOOTSTRAP 4
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#ifndef LOCORE
Include intr_machdep.h only for !LOCORE.
2002-02-23 18:41:34 +00:00
#include <machine/intr_machdep.h>
Include machine/pcb.hto turn extern struct pcb stoppcbs[]; construct into the valid C.
2007-05-19 05:01:43 +00:00
#include <machine/pcb.h>
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
#include <machine/tte.h>
Include intr_machdep.h only for !LOCORE.
2002-02-23 18:41:34 +00:00
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
#define IDR_BUSY 0x0000000000000001ULL
#define IDR_NACK 0x0000000000000002ULL
#define IDR_CHEETAH_ALL_BUSY 0x5555555555555555ULL
#define IDR_CHEETAH_ALL_NACK (~IDR_CHEETAH_ALL_BUSY)
#define IDR_CHEETAH_MAX_BN_PAIRS 32
#define IDR_JALAPENO_MAX_BN_PAIRS 4
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
#define IDC_ITID_SHIFT 14
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
#define IDC_BN_SHIFT 24
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#define IPI_AST PIL_AST
#define IPI_RENDEZVOUS PIL_RENDEZVOUS
- Add support for IPI_PREEMPT. [1] - Add my copyright to mp_machdep.c for having implemented support for USIII and up and some fixes. Obtained from: sun4v (modulo style(9) bugs) [1]
2008-04-09 21:14:01 +00:00
#define IPI_PREEMPT PIL_PREEMPT
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#define IPI_STOP PIL_STOP
MFC r196196: * Completely remove the option STOP_NMI from the kernel. This option has proven to have a good effect when entering KDB by using a NMI, but it completely violates all the good rules about interrupts disabled while holding a spinlock in other occasions. This can be the cause of deadlocks on events where a normal IPI_STOP is expected. * Add an new IPI called IPI_STOP_HARD on all the supported architectures. This IPI is responsible for sending a stop message among CPUs using a privileged channel when disponible. In other cases it just does match a normal IPI_STOP. Right now the IPI_STOP_HARD functionality uses a NMI on ia32 and amd64 architectures, while on the other has a normal IPI_STOP effect. It is responsibility of maintainers to eventually implement an hard stop when necessary and possible. * Use the new IPI facility in order to implement a new userend SMP kernel function called stop_cpus_hard(). That is specular to stop_cpu() but it does use the privileged channel for the stopping facility. * Let KDB use the newly introduced function stop_cpus_hard() and leave stop_cpus() for all the other cases * Disable interrupts on CPU0 when starting the process of APs suspension. * Style cleanup and comments adding This patch should fix the reboot/shutdown deadlocks many users are constantly reporting on mailing lists. Please don't forget to update your config file with the STOP_NMI option removal Reviewed by: jhb Tested by: pho, bz, rink Approved by: re (kib)
2009-08-13 17:54:11 +00:00
#define IPI_STOP_HARD PIL_STOP
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
We seem to have occasions where sending an IPI takes significantly longer than 'normal'. The cause is still being tracked down but in the meantime there are machines where raising IPI_RETRIES does help - it's not just a case of the machine staying locked up longer and then panic-ing anyway. Several helpful folks on sparc64@ tried a patch that helped figure out what to raise this number to. Discussed on: sparc64@ MFC after: 3 days
2004-09-29 21:39:36 +00:00
#define IPI_RETRIES 5000
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
struct cpu_start_args {
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
u_int csa_count;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
u_int csa_mid;
u_int csa_state;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
vm_offset_t csa_pcpu;
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
u_long csa_tick;
- USIII-based machines can consist of CPUs running at different frequencies (and having different cache sizes) so use the STICK (System TICK) timer, which was introduced due to this and is driven by the same frequency across all CPUs, instead of the TICK timer, whose frequency varies with the CPU clock, to drive hardclock. We try to use the STICK counter with all CPUs that are USIII or beyond, even when not necessary due to identical CPUs, as we can can also avoid the workaround for the BlackBird erratum #1 there. Unfortunately, using the STICK counter currently causes a hang with USIIIi MP machines for reasons unknown, so we still use the TICK timer there (which is okay as they can only consist of identical CPUs). - Given that we only (try to) synchronize the (S)TICK timers of APs with the BSP during startup, we could end up spinning forever in DELAY(9) if that function is migrated to another CPU while we're spinning due to clock drift afterwards, so pin to the CPU in order to avoid migration. Unfortunately, pinning doesn't work at the point DELAY(9) is required by the low-level console drivers, yet, so switch to a function pointer, which is updated accordingly, for implementing DELAY(9). For USIII and beyond, this would also allow to easily use the STICK counter instead of the TICK one here, there's no benefit in doing so however. While at it, use cpu_spinwait(9) for spinning in the delay- functions. This currently is a NOP though. - Don't set the TICK timer of the BSP to 0 during at startup as there's no need to do so. - Implement cpu_est_clockrate(). - Unfortunately, USIIIi-based machines don't provide a timecounter device besides the STICK and TICK counters (well, in theory the Tomatillo bridges have a performance counter that can be (ab)used as timecounter by configuring it to count bus cycles, though unlike the performance counter of Schizo bridges, the Tomatillo one is broken and counts Sun knows what in this mode). This means that we've to use a (S)TICK counter for timecounting, which has the old problem of not being in sync across CPUs, so provide an additional timecounter function which binds itself to the BSP but has an adequate low priority.
2008-09-03 17:39:19 +00:00
u_long csa_stick;
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
u_long csa_ver;
struct tte csa_ttes[PCPU_PAGES];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
};
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
struct ipi_cache_args {
u_int ica_mask;
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
vm_paddr_t ica_pa;
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
};
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
struct ipi_tlb_args {
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
u_int ita_mask;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
struct pmap *ita_pmap;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
u_long ita_start;
u_long ita_end;
};
#define ita_va ita_start
struct pcpu;
Add stoppcbs[] arrays on Alpha and sparc64 and have each CPU save its current context in the IPI_STOP handler so that we can get accurate stack traces of threads on other CPUs on these two archs like we do now on i386 and amd64. Tested on: alpha, sparc64
2005-11-03 21:08:20 +00:00
extern struct pcb stoppcbs[];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
void cpu_mp_bootstrap(struct pcpu *pc);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
void cpu_mp_shutdown(void);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
typedef void cpu_ipi_selected_t(u_int, u_long, u_long, u_long);
extern cpu_ipi_selected_t *cpu_ipi_selected;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
void mp_init(void);
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
extern struct mtx ipi_mtx;
extern struct ipi_cache_args ipi_cache_args;
extern struct ipi_tlb_args ipi_tlb_args;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
extern char *mp_tramp_code;
extern u_long mp_tramp_code_len;
extern u_long mp_tramp_tlb_slots;
extern u_long mp_tramp_func;
extern void mp_startup(void);
- Remove unused cache flushing routines. These will not necessary work on future UltraSPARC cpus for which the data cache is not direct mapped. - Move UltraSPARC I and II (spitfire, blackbird, sapphire, sabre) specific functions to spitfire.c, and add cheetah.c for UltraSPARC III specific functions. Initially just cache flushing, but there are a few other functions that will need to move here. - Add an ipi handler for data cache flushing on UltraSPARC III. - Use function pointers to select the right cache flushing functions based on cpu_impl. With this it is possible to boot single user from an mfs root on UltraSPARC III systems, including spinning up secondary processors. There is currently no support for the host to pci bridge, and no documentation for it is publically available. Thanks to Oleg Derevenetz for providing access to a system with UltraSPARC III+ cpus.
2003-03-19 06:55:37 +00:00
extern char tl_ipi_cheetah_dcache_page_inval[];
extern char tl_ipi_spitfire_dcache_page_inval[];
extern char tl_ipi_spitfire_icache_page_inval[];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
extern char tl_ipi_level[];
extern char tl_ipi_tlb_context_demap[];
extern char tl_ipi_tlb_page_demap[];
extern char tl_ipi_tlb_range_demap[];
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
static __inline void
ipi_all_but_self(u_int ipi)
{
cpu_ipi_selected(PCPU_GET(other_cpus), 0, (u_long)tl_ipi_level, ipi);
}
static __inline void
ipi_selected(u_int cpus, u_int ipi)
{
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_level, ipi);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
#if defined(_MACHINE_PMAP_H_) && defined(_SYS_MUTEX_H_)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_dcache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
struct ipi_cache_args *ica;
if (smp_cpus == 1)
return (NULL);
ica = &ipi_cache_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_lock_spin(&ipi_mtx);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_mask = all_cpus;
ica->ica_pa = pa;
- Remove unused cache flushing routines. These will not necessary work on future UltraSPARC cpus for which the data cache is not direct mapped. - Move UltraSPARC I and II (spitfire, blackbird, sapphire, sabre) specific functions to spitfire.c, and add cheetah.c for UltraSPARC III specific functions. Initially just cache flushing, but there are a few other functions that will need to move here. - Add an ipi handler for data cache flushing on UltraSPARC III. - Use function pointers to select the right cache flushing functions based on cpu_impl. With this it is possible to boot single user from an mfs root on UltraSPARC III systems, including spinning up secondary processors. There is currently no support for the host to pci bridge, and no documentation for it is publically available. Thanks to Oleg Derevenetz for providing access to a system with UltraSPARC III+ cpus.
2003-03-19 06:55:37 +00:00
cpu_ipi_selected(PCPU_GET(other_cpus), 0, (u_long)func, (u_long)ica);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (&ica->ica_mask);
}
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_icache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
struct ipi_cache_args *ica;
if (smp_cpus == 1)
return (NULL);
ica = &ipi_cache_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_lock_spin(&ipi_mtx);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_mask = all_cpus;
ica->ica_pa = pa;
- Remove unused cache flushing routines. These will not necessary work on future UltraSPARC cpus for which the data cache is not direct mapped. - Move UltraSPARC I and II (spitfire, blackbird, sapphire, sabre) specific functions to spitfire.c, and add cheetah.c for UltraSPARC III specific functions. Initially just cache flushing, but there are a few other functions that will need to move here. - Add an ipi handler for data cache flushing on UltraSPARC III. - Use function pointers to select the right cache flushing functions based on cpu_impl. With this it is possible to boot single user from an mfs root on UltraSPARC III systems, including spinning up secondary processors. There is currently no support for the host to pci bridge, and no documentation for it is publically available. Thanks to Oleg Derevenetz for providing access to a system with UltraSPARC III+ cpus.
2003-03-19 06:55:37 +00:00
cpu_ipi_selected(PCPU_GET(other_cpus), 0, (u_long)func, (u_long)ica);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (&ica->ica_mask);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_context_demap(struct pmap *pm)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
u_int cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
if ((cpus = (pm->pm_active & PCPU_GET(other_cpus))) == 0)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
ita = &ipi_tlb_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_lock_spin(&ipi_mtx);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
ita->ita_mask = cpus | PCPU_GET(cpumask);
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_context_demap,
(u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void *
Remove the tlb argument to tlb_page_demap (itlb or dtlb), in order to better match the pmap_invalidate api.
2002-07-26 15:54:04 +00:00
ipi_tlb_page_demap(struct pmap *pm, vm_offset_t va)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
u_int cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
if ((cpus = (pm->pm_active & PCPU_GET(other_cpus))) == 0)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
ita = &ipi_tlb_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_lock_spin(&ipi_mtx);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
ita->ita_mask = cpus | PCPU_GET(cpumask);
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita->ita_va = va;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_page_demap, (u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_range_demap(struct pmap *pm, vm_offset_t start, vm_offset_t end)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
u_int cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
if ((cpus = (pm->pm_active & PCPU_GET(other_cpus))) == 0)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
ita = &ipi_tlb_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_lock_spin(&ipi_mtx);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
ita->ita_mask = cpus | PCPU_GET(cpumask);
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita->ita_start = start;
ita->ita_end = end;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_range_demap, (u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void
ipi_wait(void *cookie)
{
Declare as volatile the memory location referenced by a pointer rather than the pointer's value.
2005-03-06 20:57:08 +00:00
volatile u_int *mask;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
if ((mask = cookie) != NULL) {
atomic_clear_int(mask, PCPU_GET(cpumask));
while (*mask != 0)
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
mtx_unlock_spin(&ipi_mtx);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
}
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
#endif /* _MACHINE_PMAP_H_ && _SYS_MUTEX_H_ */
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#endif /* !LOCORE */
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#else
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#ifndef LOCORE
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_dcache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (NULL);
}
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_icache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (NULL);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_context_demap(struct pmap *pm)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void *
Remove the tlb argument to tlb_page_demap (itlb or dtlb), in order to better match the pmap_invalidate api.
2002-07-26 15:54:04 +00:00
ipi_tlb_page_demap(struct pmap *pm, vm_offset_t va)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_range_demap(struct pmap *pm, vm_offset_t start, vm_offset_t end)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void
ipi_wait(void *cookie)
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
static __inline void
tl_ipi_cheetah_dcache_page_inval(void)
{
}
static __inline void
tl_ipi_spitfire_dcache_page_inval(void)
{
}
static __inline void
tl_ipi_spitfire_icache_page_inval(void)
{
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#endif /* !LOCORE */
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#endif /* SMP */
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
#endif /* !_MACHINE_SMP_H_ */
Reference in New Issue Copy Permalink