d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
1a125d6d85
freebsd-dev/sys/sparc64/include/smp.h

383 lines
8.1 KiB
C
Raw Normal View History

Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
/*-
* Copyright (c) 2001 Jake Burkholder.
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
* Copyright (c) 2007 - 2011 Marius Strobl <marius@FreeBSD.org>
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
The author isn't a [UC] Regents. Correct the copyright language.
2001-08-09 02:09:34 +00:00
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
The author isn't a [UC] Regents. Correct the copyright language.
2001-08-09 02:09:34 +00:00
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $FreeBSD$
*/
#ifndef _MACHINE_SMP_H_
#define _MACHINE_SMP_H_
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#ifdef SMP
- USIII-based machines can consist of CPUs running at different frequencies (and having different cache sizes) so use the STICK (System TICK) timer, which was introduced due to this and is driven by the same frequency across all CPUs, instead of the TICK timer, whose frequency varies with the CPU clock, to drive hardclock. We try to use the STICK counter with all CPUs that are USIII or beyond, even when not necessary due to identical CPUs, as we can can also avoid the workaround for the BlackBird erratum #1 there. Unfortunately, using the STICK counter currently causes a hang with USIIIi MP machines for reasons unknown, so we still use the TICK timer there (which is okay as they can only consist of identical CPUs). - Given that we only (try to) synchronize the (S)TICK timers of APs with the BSP during startup, we could end up spinning forever in DELAY(9) if that function is migrated to another CPU while we're spinning due to clock drift afterwards, so pin to the CPU in order to avoid migration. Unfortunately, pinning doesn't work at the point DELAY(9) is required by the low-level console drivers, yet, so switch to a function pointer, which is updated accordingly, for implementing DELAY(9). For USIII and beyond, this would also allow to easily use the STICK counter instead of the TICK one here, there's no benefit in doing so however. While at it, use cpu_spinwait(9) for spinning in the delay- functions. This currently is a NOP though. - Don't set the TICK timer of the BSP to 0 during at startup as there's no need to do so. - Implement cpu_est_clockrate(). - Unfortunately, USIIIi-based machines don't provide a timecounter device besides the STICK and TICK counters (well, in theory the Tomatillo bridges have a performance counter that can be (ab)used as timecounter by configuring it to count bus cycles, though unlike the performance counter of Schizo bridges, the Tomatillo one is broken and counts Sun knows what in this mode). This means that we've to use a (S)TICK counter for timecounting, which has the old problem of not being in sync across CPUs, so provide an additional timecounter function which binds itself to the BSP but has an adequate low priority.
2008-09-03 17:39:19 +00:00
#define CPU_TICKSYNC 1
#define CPU_STICKSYNC 2
#define CPU_INIT 3
#define CPU_BOOTSTRAP 4
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#ifndef LOCORE
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
#include <sys/cpuset.h>
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
#include <sys/proc.h>
#include <sys/sched.h>
Revert the part of r239864 which removed obtaining the SMP mutex around reading registers from other CPUs. As it turns out, the hardware doesn't really like concurrent IPI'ing causing adverse effects. Also the thought deadlock when using this spin lock here and the targeted CPU(s) are also holding or in case of nested locks can't actually happen. This is due to the fact that on sparc64, spinlock_enter() only raises the PIL but doesn't disable interrupts completely. Thus direct cross calls as used for the register reading (and all other MD IPI needs) still will be executed by the targeted CPU(s) in that case. MFC after: 3 days
2013-01-23 22:52:20 +00:00
#include <sys/smp.h>
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
Include intr_machdep.h only for !LOCORE.
2002-02-23 18:41:34 +00:00
#include <machine/intr_machdep.h>
Include machine/pcb.hto turn extern struct pcb stoppcbs[]; construct into the valid C.
2007-05-19 05:01:43 +00:00
#include <machine/pcb.h>
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
#include <machine/tte.h>
Include intr_machdep.h only for !LOCORE.
2002-02-23 18:41:34 +00:00
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
#define IDR_BUSY 0x0000000000000001ULL
#define IDR_NACK 0x0000000000000002ULL
#define IDR_CHEETAH_ALL_BUSY 0x5555555555555555ULL
#define IDR_CHEETAH_ALL_NACK (~IDR_CHEETAH_ALL_BUSY)
#define IDR_CHEETAH_MAX_BN_PAIRS 32
#define IDR_JALAPENO_MAX_BN_PAIRS 4
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
#define IDC_ITID_SHIFT 14
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
#define IDC_BN_SHIFT 24
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#define IPI_AST PIL_AST
#define IPI_RENDEZVOUS PIL_RENDEZVOUS
- Add support for IPI_PREEMPT. [1] - Add my copyright to mp_machdep.c for having implemented support for USIII and up and some fixes. Obtained from: sun4v (modulo style(9) bugs) [1]
2008-04-09 21:14:01 +00:00
#define IPI_PREEMPT PIL_PREEMPT
Adapt sparc64 and sun4v timer code for the new event timers infrastructure. Reviewed by: marius@
2010-07-29 12:08:46 +00:00
#define IPI_HARDCLOCK PIL_HARDCLOCK
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#define IPI_STOP PIL_STOP
* Completely Remove the option STOP_NMI from the kernel. This option has proven to have a good effect when entering KDB by using a NMI, but it completely violates all the good rules about interrupts disabled while holding a spinlock in other occasions. This can be the cause of deadlocks on events where a normal IPI_STOP is expected. * Adds an new IPI called IPI_STOP_HARD on all the supported architectures. This IPI is responsible for sending a stop message among CPUs using a privileged channel when disponible. In other cases it just does match a normal IPI_STOP. Right now the IPI_STOP_HARD functionality uses a NMI on ia32 and amd64 architectures, while on the other has a normal IPI_STOP effect. It is responsibility of maintainers to eventually implement an hard stop when necessary and possible. * Use the new IPI facility in order to implement a new userend SMP kernel function called stop_cpus_hard(). That is specular to stop_cpu() but it does use the privileged channel for the stopping facility. * Let KDB use the newly introduced function stop_cpus_hard() and leave stop_cpus() for all the other cases * Disable interrupts on CPU0 when starting the process of APs suspension. * Style cleanup and comments adding This patch should fix the reboot/shutdown deadlocks many users are constantly reporting on mailing lists. Please don't forget to update your config file with the STOP_NMI option removal Reviewed by: jhb Tested by: pho, bz, rink Approved by: re (kib)
2009-08-13 17:09:45 +00:00
#define IPI_STOP_HARD PIL_STOP
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
We seem to have occasions where sending an IPI takes significantly longer than 'normal'. The cause is still being tracked down but in the meantime there are machines where raising IPI_RETRIES does help - it's not just a case of the machine staying locked up longer and then panic-ing anyway. Several helpful folks on sparc64@ tried a patch that helped figure out what to raise this number to. Discussed on: sparc64@ MFC after: 3 days
2004-09-29 21:39:36 +00:00
#define IPI_RETRIES 5000
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
struct cpu_start_args {
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
u_int csa_count;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
u_int csa_mid;
u_int csa_state;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
vm_offset_t csa_pcpu;
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
u_long csa_tick;
- USIII-based machines can consist of CPUs running at different frequencies (and having different cache sizes) so use the STICK (System TICK) timer, which was introduced due to this and is driven by the same frequency across all CPUs, instead of the TICK timer, whose frequency varies with the CPU clock, to drive hardclock. We try to use the STICK counter with all CPUs that are USIII or beyond, even when not necessary due to identical CPUs, as we can can also avoid the workaround for the BlackBird erratum #1 there. Unfortunately, using the STICK counter currently causes a hang with USIIIi MP machines for reasons unknown, so we still use the TICK timer there (which is okay as they can only consist of identical CPUs). - Given that we only (try to) synchronize the (S)TICK timers of APs with the BSP during startup, we could end up spinning forever in DELAY(9) if that function is migrated to another CPU while we're spinning due to clock drift afterwards, so pin to the CPU in order to avoid migration. Unfortunately, pinning doesn't work at the point DELAY(9) is required by the low-level console drivers, yet, so switch to a function pointer, which is updated accordingly, for implementing DELAY(9). For USIII and beyond, this would also allow to easily use the STICK counter instead of the TICK one here, there's no benefit in doing so however. While at it, use cpu_spinwait(9) for spinning in the delay- functions. This currently is a NOP though. - Don't set the TICK timer of the BSP to 0 during at startup as there's no need to do so. - Implement cpu_est_clockrate(). - Unfortunately, USIIIi-based machines don't provide a timecounter device besides the STICK and TICK counters (well, in theory the Tomatillo bridges have a performance counter that can be (ab)used as timecounter by configuring it to count bus cycles, though unlike the performance counter of Schizo bridges, the Tomatillo one is broken and counts Sun knows what in this mode). This means that we've to use a (S)TICK counter for timecounting, which has the old problem of not being in sync across CPUs, so provide an additional timecounter function which binds itself to the BSP but has an adequate low priority.
2008-09-03 17:39:19 +00:00
u_long csa_stick;
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
u_long csa_ver;
struct tte csa_ttes[PCPU_PAGES];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
};
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
struct ipi_cache_args {
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t ica_mask;
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
vm_paddr_t ica_pa;
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
};
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
struct ipi_rd_args {
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t ira_mask;
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
register_t *ira_val;
};
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
struct ipi_tlb_args {
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t ita_mask;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
struct pmap *ita_pmap;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
u_long ita_start;
u_long ita_end;
};
#define ita_va ita_start
struct pcpu;
Add stoppcbs[] arrays on Alpha and sparc64 and have each CPU save its current context in the IPI_STOP handler so that we can get accurate stack traces of threads on other CPUs on these two archs like we do now on i386 and amd64. Tested on: alpha, sparc64
2005-11-03 21:08:20 +00:00
extern struct pcb stoppcbs[];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
void cpu_mp_bootstrap(struct pcpu *pc);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
void cpu_mp_shutdown(void);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
typedef void cpu_ipi_selected_t(cpuset_t, u_long, u_long, u_long);
- Add support for sending IPIs with USIII and greater sun4u CPUs. These CPUs use an enhanced layout of the interrupt vector dispatch and dispatch status registers in order to allow sending IPIs to multiple targets simultaneously. Thus support for these CPUs was put in a newly added cheetah_ipi_selected(). This is intended to be pointed to by cpu_ipi_selected, which now is a function pointer, in order to avoid cpu_impl checks once booted. Alternatively it can point to spitfire_ipi_selected(), which was renamed from cpu_ipi_selected(). Consequently cpu_ipi_send() was also renamed to spitfire_ipi_send() (there's no need for a cheetah equivalent of this so far). Initialization of the cpu_ipi_selected pointer and other requirements is done in mp_init(), which was renamed from mp_tramp_alloc(), as cpu_mp_start() isn't called on UP systems while cpu_ipi_selected() is. As a side-effect this allows to make mp_tramp static to sys/sparc64/sparc64/mp_machdep.c. For the sake of avoiding #ifdef SMP and for keeping the history in place cheetah_ipi_selected() and spitfire_ipi_{selected,send}() where not put into/moved to sys/sparc64/sparc64/{cheetah,spitfire}.c - Add some CTASSERTs and KASSERTs ensuring that MAXCPU doesn't exceed the data types we use to store the CPU bit fields or the number of USIII and greater CPUs supported by the current cheetah_ipi_selected() implementation (which for JBus-CPUs is only 4; that should be fine though as according to OpenSolaris there are no sun4u machines with more than 4 JBus-CPUs). - In cpu_mp_start() don't enumerate and start more than MAXCPU CPUs as we can't handle more than that. - In cpu_mp_start() check for upa-portid vs. portid depending on cpu_impl for consistency with nexus(4). - In spitfire_ipi_selected() add KASSERTs ensuring that a CPU isn't told to IPI itself as sun4u CPUs just can't do that. - In spitfire_ipi_send() do a MEMBAR #Sync after writing the interrupt vector data as we want to make sure the payload was actually written before we trigger the dispatch. - In spitfire_ipi_send() also verify IDR_BUSY when checking whether the dispatch was successful as it has to be cleared for this to be the case. - Remove some redundant variables.
2007-06-16 23:26:00 +00:00
extern cpu_ipi_selected_t *cpu_ipi_selected;
- Introduce a cpu_ipi_single() function pointer in order to send IPIs to single CPUs more efficiently with Cheetah(-class) and Jalapeno CPUs. Besides being used to implement the ipi_cpu() introduced in r210939, cpu_ipi_single() will also be used internally by the sparc64 MD code. - Factor out the Jalapeno support from the Cheetah IPI send functions in order to be able to more easily and efficiently implement support for more than 32 target CPUs as well as a workaround for Cheetah+ erratum 25 for the latter.
2010-08-08 00:09:22 +00:00
typedef void cpu_ipi_single_t(u_int, u_long, u_long, u_long);
extern cpu_ipi_single_t *cpu_ipi_single;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Some machines can not only consist of CPUs running at different speeds but also of different types, f.e. Sun Fire V890 can be equipped with a mix of UltraSPARC IV and IV+ CPUs, requiring different MMU initialization and different workarounds for model specific errata. Therefore move the CPU implementation number from a global variable to the per-CPU data. Functions which are called before the latter is available are passed the implementation number as a parameter now.
2010-02-20 23:24:19 +00:00
void mp_init(u_int cpu_impl);
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
extern struct ipi_cache_args ipi_cache_args;
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
extern struct ipi_rd_args ipi_rd_args;
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
extern struct ipi_tlb_args ipi_tlb_args;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
extern char *mp_tramp_code;
extern u_long mp_tramp_code_len;
extern u_long mp_tramp_tlb_slots;
extern u_long mp_tramp_func;
extern void mp_startup(void);
- Remove unused cache flushing routines. These will not necessary work on future UltraSPARC cpus for which the data cache is not direct mapped. - Move UltraSPARC I and II (spitfire, blackbird, sapphire, sabre) specific functions to spitfire.c, and add cheetah.c for UltraSPARC III specific functions. Initially just cache flushing, but there are a few other functions that will need to move here. - Add an ipi handler for data cache flushing on UltraSPARC III. - Use function pointers to select the right cache flushing functions based on cpu_impl. With this it is possible to boot single user from an mfs root on UltraSPARC III systems, including spinning up secondary processors. There is currently no support for the host to pci bridge, and no documentation for it is publically available. Thanks to Oleg Derevenetz for providing access to a system with UltraSPARC III+ cpus.
2003-03-19 06:55:37 +00:00
extern char tl_ipi_cheetah_dcache_page_inval[];
extern char tl_ipi_spitfire_dcache_page_inval[];
extern char tl_ipi_spitfire_icache_page_inval[];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
extern char tl_ipi_level[];
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
extern char tl_ipi_stick_rd[];
extern char tl_ipi_tick_rd[];
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
extern char tl_ipi_tlb_context_demap[];
extern char tl_ipi_tlb_page_demap[];
extern char tl_ipi_tlb_range_demap[];
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
static __inline void
ipi_all_but_self(u_int ipi)
{
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
cpuset_t cpus;
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
cpus = all_cpus;
CPU_CLR(PCPU_GET(cpuid), &cpus);
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_level, ipi);
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
}
static __inline void
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
ipi_selected(cpuset_t cpus, u_int ipi)
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
{
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_level, ipi);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Add a new ipi_cpu() function to the MI IPI API that can be used to send an IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that constructed a mask for a single CPU with calls to ipi_cpu() instead. This will matter more in the future when we transition from cpumask_t to cpuset_t for CPU masks in which case building a CPU mask is more expensive. Submitted by: peter, sbruno Reviewed by: rookie Obtained from: Yahoo! (x86) MFC after: 1 month
2010-08-06 15:36:59 +00:00
static __inline void
ipi_cpu(int cpu, u_int ipi)
{
- Introduce a cpu_ipi_single() function pointer in order to send IPIs to single CPUs more efficiently with Cheetah(-class) and Jalapeno CPUs. Besides being used to implement the ipi_cpu() introduced in r210939, cpu_ipi_single() will also be used internally by the sparc64 MD code. - Factor out the Jalapeno support from the Cheetah IPI send functions in order to be able to more easily and efficiently implement support for more than 32 target CPUs as well as a workaround for Cheetah+ erratum 25 for the latter.
2010-08-08 00:09:22 +00:00
cpu_ipi_single(cpu, 0, (u_long)tl_ipi_level, ipi);
Add a new ipi_cpu() function to the MI IPI API that can be used to send an IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that constructed a mask for a single CPU with calls to ipi_cpu() instead. This will matter more in the future when we transition from cpumask_t to cpuset_t for CPU masks in which case building a CPU mask is more expensive. Submitted by: peter, sbruno Reviewed by: rookie Obtained from: Yahoo! (x86) MFC after: 1 month
2010-08-06 15:36:59 +00:00
}
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
#if defined(_MACHINE_PMAP_H_) && defined(_SYS_MUTEX_H_)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_dcache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
struct ipi_cache_args *ica;
if (smp_cpus == 1)
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_pin();
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica = &ipi_cache_args;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_mask = all_cpus;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_CLR(PCPU_GET(cpuid), &ica->ica_mask);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_pa = pa;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
cpu_ipi_selected(ica->ica_mask, 0, (u_long)func, (u_long)ica);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (&ica->ica_mask);
}
static __inline void *
Use vm_paddr_t for physical addresses.
2003-04-08 06:35:09 +00:00
ipi_icache_page_inval(void *func, vm_paddr_t pa)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
struct ipi_cache_args *ica;
if (smp_cpus == 1)
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_pin();
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica = &ipi_cache_args;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_mask = all_cpus;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_CLR(PCPU_GET(cpuid), &ica->ica_mask);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
ica->ica_pa = pa;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
cpu_ipi_selected(ica->ica_mask, 0, (u_long)func, (u_long)ica);
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (&ica->ica_mask);
}
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
static __inline void *
ipi_rd(u_int cpu, void *func, u_long *val)
{
struct ipi_rd_args *ira;
if (smp_cpus == 1)
return (NULL);
sched_pin();
ira = &ipi_rd_args;
Revert the part of r239864 which removed obtaining the SMP mutex around reading registers from other CPUs. As it turns out, the hardware doesn't really like concurrent IPI'ing causing adverse effects. Also the thought deadlock when using this spin lock here and the targeted CPU(s) are also holding or in case of nested locks can't actually happen. This is due to the fact that on sparc64, spinlock_enter() only raises the PIL but doesn't disable interrupts completely. Thus direct cross calls as used for the register reading (and all other MD IPI needs) still will be executed by the targeted CPU(s) in that case. MFC after: 3 days
2013-01-23 22:52:20 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_SETOF(cpu, &ira->ira_mask);
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
ira->ira_val = val;
cpu_ipi_single(cpu, 0, (u_long)func, (u_long)ira);
return (&ira->ira_mask);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_context_demap(struct pmap *pm)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_pin();
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpus = pm->pm_active;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_AND(&cpus, &all_cpus);
CPU_CLR(PCPU_GET(cpuid), &cpus);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
if (CPU_EMPTY(&cpus)) {
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_unpin();
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita = &ipi_tlb_args;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
ita->ita_mask = cpus;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_context_demap,
(u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void *
Remove the tlb argument to tlb_page_demap (itlb or dtlb), in order to better match the pmap_invalidate api.
2002-07-26 15:54:04 +00:00
ipi_tlb_page_demap(struct pmap *pm, vm_offset_t va)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_pin();
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpus = pm->pm_active;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_AND(&cpus, &all_cpus);
CPU_CLR(PCPU_GET(cpuid), &cpus);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
if (CPU_EMPTY(&cpus)) {
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_unpin();
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita = &ipi_tlb_args;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
ita->ita_mask = cpus;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita->ita_va = va;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_page_demap, (u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void *
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ipi_tlb_range_demap(struct pmap *pm, vm_offset_t start, vm_offset_t end)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
struct ipi_tlb_args *ita;
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpuset_t cpus;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
if (smp_cpus == 1)
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_pin();
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
cpus = pm->pm_active;
Don't include curcpu in the mask which is used as the IPI cookie as we have to ignore it when sending the IPI anyway. Actually I can't think of a good reason why this ever was done that way in the first place as it's not even usefull for debugging. While at it replace the use of pc_other_cpus as it's slated for deorbit.
2011-06-15 22:41:55 +00:00
CPU_AND(&cpus, &all_cpus);
CPU_CLR(PCPU_GET(cpuid), &cpus);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
if (CPU_EMPTY(&cpus)) {
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
sched_unpin();
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita = &ipi_tlb_args;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_lock_spin(&smp_ipi_mtx);
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
ita->ita_mask = cpus;
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
ita->ita_pmap = pm;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
ita->ita_start = start;
ita->ita_end = end;
- Introduce a cpu_ipi_single() function pointer in order to send IPIs to single CPUs more efficiently with Cheetah(-class) and Jalapeno CPUs. Besides being used to implement the ipi_cpu() introduced in r210939, cpu_ipi_single() will also be used internally by the sparc64 MD code. - Factor out the Jalapeno support from the Cheetah IPI send functions in order to be able to more easily and efficiently implement support for more than 32 target CPUs as well as a workaround for Cheetah+ erratum 25 for the latter.
2010-08-08 00:09:22 +00:00
cpu_ipi_selected(cpus, 0, (u_long)tl_ipi_tlb_range_demap,
(u_long)ita);
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
return (&ita->ita_mask);
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
}
static __inline void
ipi_wait(void *cookie)
{
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
volatile cpuset_t *mask;
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
Make IPI_WAIT use a bit mask of the cpus that a pmap is active on and only wait for those cpus, instead of all of them by using a count. Oops. Make the pointer to the mask that the primary cpu spins on volatile, so gcc doesn't optimize out an important load. Oops again. Activate tlb shootdown ipi synchronization now that it works. We have all involved cpus wait until all the others are done. This may not be necessary, it is mostly for sanity. Make the trigger level interrupt ipi handler work. Submitted by: tmm
2002-03-13 03:43:00 +00:00
if ((mask = cookie) != NULL) {
Add sparc64 support. Compiled (and helped) by: pluknet
2011-05-06 21:53:29 +00:00
while (!CPU_EMPTY(mask))
Add support for starting secondary cpus in kernel, as opposed to relying on the loader to do it. Improve smp startup code to be less racy and to defer certain things until the right time. This almost boots single user on my dual ultra 60, it is still very fragile: SMP: AP CPU #1 Launched! Enter full pathname of shell or RETURN for /bin/sh: # ls Debugger("trapsig") Stopped at Debugger+0x1c: ta %xcc, 1 db> heh No such command db>
2002-03-04 07:12:36 +00:00
;
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
mtx_unlock_spin(&smp_ipi_mtx);
sched_unpin();
}
}
- Add a spin lock to single thread cache invalidation and tlb flush ipis, which allows ipis to be sent outside of Giant. - Remove the ap boot mutex, which is unused.
2002-12-22 20:50:23 +00:00
#endif /* _MACHINE_PMAP_H_ && _SYS_MUTEX_H_ */
Implement delivery of tlb shootdown ipis. This is currently more fine grained than the other implementations; we have complete control over the tlb, so we only demap specific pages. We take advantage of the ranged tlb flush api to send one ipi for a range of pages, and due to the pm_active optimization we rarely send ipis for demaps from user pmaps. Remove now unused routines to load the tlb; this is only done once outside of the tlb fault handlers. Minor cleanups to the smp startup code. This boots multi user with both cpus active on a dual ultra 60 and on a dual ultra 2.
2002-03-07 06:01:40 +00:00
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#endif /* !LOCORE */
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#else
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#ifndef LOCORE
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
static __inline void *
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
ipi_dcache_page_inval(void *func __unused, vm_paddr_t pa __unused)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (NULL);
}
static __inline void *
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
ipi_icache_page_inval(void *func __unused, vm_paddr_t pa __unused)
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add SMP aware cache flushing functions, which operate on a single physical page. These send IPIs if necessary in order to keep the caches in sync on all cpus.
2002-05-20 16:30:47 +00:00
return (NULL);
}
- As it is not possible for sched_bind(9) to context switch with td_critnest > 1 when not already running on the desired CPU read the TICK counter of the BSP via a direct cross trap request in that case instead. - Treat the STICK based timecounter the same way as the TICK based one regarding its quality and obtaining the counter value from the BSP. Like the TICK timers the STICK ones also are only synchronized during their startup (which might not result in good synchronicity in the first place) but not afterwards and might drift over time, causing problems when the time is read from different CPUs (see r135972).
2010-08-08 14:00:21 +00:00
static __inline void *
ipi_rd(u_int cpu __unused, void *func __unused, u_long *val __unused)
{
return (NULL);
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
static __inline void *
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
ipi_tlb_context_demap(struct pmap *pm __unused)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void *
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
ipi_tlb_page_demap(struct pmap *pm __unused, vm_offset_t va __unused)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void *
- Pin the IPI cache and TLB demap functions in order to prevent migration between determining the other CPUs and calling cpu_ipi_selected(), which apart from generally doing the wrong thing can lead to a panic when a CPU is told to IPI itself (which sun4u doesn't support). Reported and tested by: Nathaniel W Filardo - Add __unused where appropriate. MFC after: 3 days
2010-07-04 12:43:12 +00:00
ipi_tlb_range_demap(struct pmap *pm __unused, vm_offset_t start __unused,
__unused vm_offset_t end)
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
{
- Staticize cpu_ipi_send() and cpu_mp_unleash() as these aren't referenced outside of mp_machdep.c - Replace a magic 14 with the newly added IDC_ITID_SHIFT macro. - Remove the global mp_boot_mid variable as it's not really necessary and just replacing it with PCPU_GET(mid) doesn't have any impact on performance once booted. - Replace PCPU_GET(cpuid) with the curcpu shortcut. - Replace hardcoded function names in panic strings etc with __func__ so they don't need to be updated when renaming the function. - Use register_t instead of u_long for variables used to hold the return value of intr_disable() so we don't need to apply any knowledge about the actual width of that value here. - Improve the wording of some comments. - Fix several style(9) bugs.
2007-05-20 14:49:01 +00:00
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
return (NULL);
}
static __inline void
- Unlike cache invalidation and TLB demapping IPIs, reading registers from other CPUs doesn't require locking so get rid of it. As the latter is used for the timecounter on certain machine models, using a spin lock in this case can lead to a deadlock with the upcoming callout(9) rework. - Merge r134227/r167250 from x86: Avoid cross-IPI SMP deadlock by using the smp_ipi_mtx spin lock not only for smp_rendezvous_cpus() but also for the MD cache invalidation and TLB demapping IPIs. - Mark some unused function arguments as such. MFC after: 1 week
2012-08-29 16:56:50 +00:00
ipi_wait(void *cookie __unused)
{
}
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
static __inline void
tl_ipi_cheetah_dcache_page_inval(void)
{
}
static __inline void
tl_ipi_spitfire_dcache_page_inval(void)
{
}
static __inline void
tl_ipi_spitfire_icache_page_inval(void)
{
}
Add initial smp support. This gets as far as allowing the secondary cpu(s) into the kernel, and sync-ing them up to "kernel" mode so we can send them ipis, which also work. Thanks to John Baldwin for providing me with access to the hardware that made this possible. Parts obtained from: bsd/os
2002-01-08 05:50:26 +00:00
#endif /* !LOCORE */
- Newer firmware versions no longer provide SUNW,stop-self so just disable interrupts and loop forever with these. - Hide all MP-related bits in <machine/smp.h> underneath #ifdef SMP. - Inline ipi_all_but_self(9) and ipi_selected(9). We don't expose any additional bits but save a few cycles by doing so. - Remove ipi_all(9), which actually only called panic(9). It can't be implemented natively anyway and having it removed at least causes MI users to fail already fail when linking.
2008-09-18 13:56:30 +00:00
#endif /* SMP */
Add skeleton machine dependent headers and c files for a port of freebsd to a new architecture. This is the base of the sparc64 port, but contains limited machine dependent code, and can be used a base for ports. Included are: - standard machine dependent headers, tweaked for a 64 bit, big endian architecture, including empty versions of all the machine dependent structures - a machine independent atomic.h, which can be used until a port has support for interrupts and the operations really need to be atomic - stub versions of all the machine dependent functions, which panic when called and print out the name of the function that needs to be implemented. functions which are normally in assembly files are not included, but this should reduce the number of different undefined references on the first few compiles from hundreds to 5 or 6 Given minimal startup code and console support it should be trivial to make this compile and run the first few sysinits on almost any architecture. Requested by: alfred, imp, jhb
2001-07-31 05:45:16 +00:00
#endif /* !_MACHINE_SMP_H_ */
Reference in New Issue Copy Permalink