d/freebsd-skq
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
84b2995b2f
freebsd-skq/sys/kern/subr_trap.c

210 lines
6.2 KiB
C
Raw Normal View History

Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*-
trap.c: Vastly improved trap.c from me. This rewritten version has a variety of features, amoung them: higher performance and much higher code quality. support.s, cpufunc.h: No longer use gs override to enforce range limits - compare directly against VM_MAXUSER_ADDRESS instead. The old way caused problems in preserving the gs selector...and this method is just as fast or faster.
1994-06-06 14:54:41 +00:00
* Copyright (C) 1994, David Greenman
* Copyright (c) 1990, 1993
* The Regents of the University of California. All rights reserved.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*
* This code is derived from software contributed to Berkeley by
* the University of Utah, and William Jolitz.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* 3. All advertising materials mentioning features or use of this software
* must display the following acknowledgement:
* This product includes software developed by the University of
* California, Berkeley and its contributors.
* 4. Neither the name of the University nor the names of its contributors
* may be used to endorse or promote products derived from this software
* without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE REGENTS AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE REGENTS OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
genassym.c: Remove NKMEMCLUSTERS, it is no longer define or used. locores.s: Fix comment on PTDpde and APTDpde to be pde instead of pte Add new equation for calculating location of Sysmap Remove Bill's old #ifdef garbage for counting up memory, that stuff will never be made to work and was just cluttering up the file. Add code that places the PTD, page table pages, and kernel stack below the 640k ISA hole if there is room for it, otherwise put this stuff all at 1MB. This fixes the 28K bogusity in the boot blocks, that can now go away! Fix the caclulation of where first is to be dependent on NKPDE so that we can skip over the above mentioned areas. The 28K thing is now 44K in size due to the increase in kernel virtual memory space, but since we no longer have to worry about that this is no big deal. Use if NNPX > 0 instead of ifdef NPX for floating point code. machdep.c Change the calculation of for the buffer cache to be 20% of all memory above 2MB and add back the upper limit of 2/5's of the VM_KMEM_SIZE so that we do not eat ALL of the kernel memory space on large memory machines, note that this will not even come into effect unless you have more than 32MB. The current buffer cache limit is 6.7MB due to this caclulation. It seems that we where erroniously allocating bufpages pages for buffer_map. buffer_map is UNUSED in this implementation of the buffer cache, but since the map is referenced in several if statements a quick fix was to simply allocate 1 vm page (but no real memory) to it. pmap.h Remove rcsid, don't want them in the kernel files! Removed some cruft inside an #ifdef DEBUGx that caused compiler errors if you where compiling this for debug. Use the #defines for PD_SHIFT and PG_SHIFT in place of constants. trap.c: Remove patch kit header and rcsid, fix $Id$. Now include "npx.h" and use NNPX for controlling the floating point code. Remove a now completly invalid check for a maximum virtual address, the virtual address now ends at 0xFFFFFFFF so there is no more MAX!! (Thanks David, I completly missed that one!) vm_machdep.c Remove patch kit header and rcsid, fix $Id$. Now include "npx.h" and use NNPX for controlling the floating point code. Replace several 0xFE00000 constants with KERNBASE
1993-10-15 10:34:29 +00:00
* from: @(#)trap.c 7.4 (Berkeley) 5/13/91
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
* $FreeBSD$
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
#ifdef __i386__
Use #ifdef DEV_NPX from opt_npx.h instead of #if NNPX > 0 from npx.h
2001-01-19 13:19:02 +00:00
#include "opt_npx.h"
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
#endif
Converted two options over to the new scheme: USER_LDT and KTRACE.
1996-01-03 21:42:35 +00:00
The big 4.4BSD Lite to FreeBSD 2.0.0 (Development) patch. Reviewed by: Rodney W. Grimes Submitted by: John Dyson and David Greenman
1994-05-25 09:21:21 +00:00
#include <sys/param.h>
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
#include <sys/bus.h>
The big 4.4BSD Lite to FreeBSD 2.0.0 (Development) patch. Reviewed by: Rodney W. Grimes Submitted by: John Dyson and David Greenman
1994-05-25 09:21:21 +00:00
#include <sys/kernel.h>
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
#include <sys/lock.h>
Catch up to moving headers: - machine/ipl.h -> sys/ipl.h - machine/mutex.h -> sys/mutex.h
2000-10-20 07:58:15 +00:00
#include <sys/mutex.h>
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
#include <sys/proc.h>
Fixed some #include messes. Hid the check of the user %cs in syscall() under `#ifdef DIAGNOSTIC'.
1997-11-24 13:25:37 +00:00
#include <sys/resourcevar.h>
#include <sys/signalvar.h>
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
#include <sys/systm.h>
Untangled the vm.h include file spaghetti.
1995-12-07 12:48:31 +00:00
#include <sys/vmmeter.h>
The big 4.4BSD Lite to FreeBSD 2.0.0 (Development) patch. Reviewed by: Rodney W. Grimes Submitted by: John Dyson and David Greenman
1994-05-25 09:21:21 +00:00
#include <machine/cpu.h>
Fixed some #include messes. Hid the check of the user %cs in syscall() under `#ifdef DIAGNOSTIC'.
1997-11-24 13:25:37 +00:00
#include <machine/pcb.h>
- Add code to detect if a system call returns with locks other than Giant held and panic if so (conditional on witness). - Change witness_list to return the number of locks held so this is easier. - Add kern/syscalls.c to the kernel build if witness is defined so that the panic message can contain the name of the offending system call. - Add assertions that Giant and sched_lock are not held when returning from a system call, which were missing for alpha and ia64.
2000-12-12 01:14:32 +00:00
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
/*
* Define the code needed before returning to user mode, for
* trap and syscall.
Remove the MPSAFE keyword from the parser for syscalls.master. Instead introduce the [M] prefix to existing keywords. e.g. MSTD is the MP SAFE version of STD. This is prepatory for a massive Giant lock pushdown. The old MPSAFE keyword made syscalls.master too messy. Begin comments MP-Safe procedures with the comment: /* * MPSAFE */ This comments means that the procedure may be called without Giant held (The procedure itself may still need to obtain Giant temporarily to do its thing). sv_prepsyscall() is now MP SAFE and assumed to be MP SAFE sv_transtrap() is now MP SAFE and assumed to be MP SAFE ktrsyscall() and ktrsysret() are now MP SAFE (Giant Pushdown) trapsignal() is now MP SAFE (Giant Pushdown) Places which used to do the if (mtx_owned(&Giant)) mtx_unlock(&Giant) test in syscall[2]() in */*/trap.c now do not. Instead they explicitly unlock Giant if they previously obtained it, and then assert that it is no longer held to catch broken system calls. Rebuild syscall tables.
2001-08-30 18:50:57 +00:00
*
* MPSAFE
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
*/
- Kill the have_giant parameter to userret() along with all instances of that name as a variable. Use mtx_owned(&Giant) where appropriate instead. - Proc locking. - P_FOO -> PS_FOO. - Update comments about enable interrupts during trap and why this may be bad if we trap while holding a spin mutex. - Don't bother resetting p to curproc in syscall() in case we are the child returning from fork. The child hasn't returned from fork through syscall in a while. - Remove fork_return() as it has been superseded by the MI version.
2001-01-24 09:53:49 +00:00
void
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
userret(td, frame, oticks)
struct thread *td;
trap.c: Vastly improved trap.c from me. This rewritten version has a variety of features, amoung them: higher performance and much higher code quality. support.s, cpufunc.h: No longer use gs override to enforce range limits - compare directly against VM_MAXUSER_ADDRESS instead. The old way caused problems in preserving the gs selector...and this method is just as fast or faster.
1994-06-06 14:54:41 +00:00
struct trapframe *frame;
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
u_int oticks;
trap.c: Vastly improved trap.c from me. This rewritten version has a variety of features, amoung them: higher performance and much higher code quality. support.s, cpufunc.h: No longer use gs override to enforce range limits - compare directly against VM_MAXUSER_ADDRESS instead. The old way caused problems in preserving the gs selector...and this method is just as fast or faster.
1994-06-06 14:54:41 +00:00
{
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct proc *p = td->td_proc;
struct kse *ke = td->td_kse;
struct ksegrp *kg = td->td_ksegrp;
trap.c: Vastly improved trap.c from me. This rewritten version has a variety of features, amoung them: higher performance and much higher code quality. support.s, cpufunc.h: No longer use gs override to enforce range limits - compare directly against VM_MAXUSER_ADDRESS instead. The old way caused problems in preserving the gs selector...and this method is just as fast or faster.
1994-06-06 14:54:41 +00:00
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
#ifdef INVARIANTS
/* Check that we called signotify() enough. */
Grab Giant around postsig() since sendsig() can call into the vm to grow the stack and we already needed Giant for KTRACE.
2001-07-03 05:27:53 +00:00
mtx_lock(&Giant);
- Grab the proc lock around CURSIG and postsig(). Don't release the proc lock until after grabbing the sched_lock to avoid CURSIG racing with psignal. - Don't grab Giant for addupc_task() as it isn't needed. Reported by: tegge (signal race), bde (addupc_task a while back)
2001-06-22 23:05:11 +00:00
PROC_LOCK(p);
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
mtx_lock_spin(&sched_lock);
if (SIGPENDING(p) && ((p->p_sflag & PS_NEEDSIGCHK) == 0 ||
(p->p_kse.ke_flags & KEF_ASTPENDING) == 0))
printf("failed to set signal flags proprly for ast()\n");
mtx_unlock_spin(&sched_lock);
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
PROC_UNLOCK(p);
Remove the MPSAFE keyword from the parser for syscalls.master. Instead introduce the [M] prefix to existing keywords. e.g. MSTD is the MP SAFE version of STD. This is prepatory for a massive Giant lock pushdown. The old MPSAFE keyword made syscalls.master too messy. Begin comments MP-Safe procedures with the comment: /* * MPSAFE */ This comments means that the procedure may be called without Giant held (The procedure itself may still need to obtain Giant temporarily to do its thing). sv_prepsyscall() is now MP SAFE and assumed to be MP SAFE sv_transtrap() is now MP SAFE and assumed to be MP SAFE ktrsyscall() and ktrsysret() are now MP SAFE (Giant Pushdown) trapsignal() is now MP SAFE (Giant Pushdown) Places which used to do the if (mtx_owned(&Giant)) mtx_unlock(&Giant) test in syscall[2]() in */*/trap.c now do not. Instead they explicitly unlock Giant if they previously obtained it, and then assert that it is no longer held to catch broken system calls. Rebuild syscall tables.
2001-08-30 18:50:57 +00:00
mtx_unlock(&Giant);
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
#endif
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
/*
* XXX we cheat slightly on the locking here to avoid locking in
* the usual case. Setting td_priority here is essentially an
* incomplete workaround for not setting it properly elsewhere.
* Now that some interrupt handlers are threads, not setting it
* properly elsewhere can clobber it in the window between setting
* it here and returning to user mode, so don't waste time setting
* it perfectly here.
*/
if (td->td_priority != kg->kg_user_pri) {
- Change all callers of addupc_task() to check PS_PROFIL explicitly and remove the check from addupc_task(). It would need sched_lock while testing the flag anyways. - Always read sticks while holding sched_lock using a temporary variable where needed. - Always init prticks to 0 in ast() to quiet a warning.
2001-12-18 09:06:10 +00:00
mtx_lock_spin(&sched_lock);
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
td->td_priority = kg->kg_user_pri;
mtx_unlock_spin(&sched_lock);
- Change all callers of addupc_task() to check PS_PROFIL explicitly and remove the check from addupc_task(). It would need sched_lock while testing the flag anyways. - Always read sticks while holding sched_lock using a temporary variable where needed. - Always init prticks to 0 in ast() to quiet a warning.
2001-12-18 09:06:10 +00:00
}
- Kill the have_giant parameter to userret() along with all instances of that name as a variable. Use mtx_owned(&Giant) where appropriate instead. - Proc locking. - P_FOO -> PS_FOO. - Update comments about enable interrupts during trap and why this may be bad if we trap while holding a spin mutex. - Don't bother resetting p to curproc in syscall() in case we are the child returning from fork. The child hasn't returned from fork through syscall in a while. - Remove fork_return() as it has been superseded by the MI version.
2001-01-24 09:53:49 +00:00
Check P_PROFIL flag for profiling rather than pr_scale as it makes more sense.
1995-02-10 06:25:14 +00:00
/*
* Charge system time if profiling.
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
*
* XXX should move PS_PROFIL to a place that can obviously be
* accessed safely without sched_lock.
Check P_PROFIL flag for profiling rather than pr_scale as it makes more sense.
1995-02-10 06:25:14 +00:00
*/
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
if (p->p_sflag & PS_PROFIL) {
- Change all callers of addupc_task() to check PS_PROFIL explicitly and remove the check from addupc_task(). It would need sched_lock while testing the flag anyways. - Always read sticks while holding sched_lock using a temporary variable where needed. - Always init prticks to 0 in ast() to quiet a warning.
2001-12-18 09:06:10 +00:00
quad_t ticks;
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
mtx_lock_spin(&sched_lock);
- Change all callers of addupc_task() to check PS_PROFIL explicitly and remove the check from addupc_task(). It would need sched_lock while testing the flag anyways. - Always read sticks while holding sched_lock using a temporary variable where needed. - Always init prticks to 0 in ast() to quiet a warning.
2001-12-18 09:06:10 +00:00
ticks = ke->ke_sticks - oticks;
mtx_unlock_spin(&sched_lock);
addupc_task(ke, TRAPF_PC(frame), (u_int)ticks * psratio);
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
}
trap.c: Vastly improved trap.c from me. This rewritten version has a variety of features, amoung them: higher performance and much higher code quality. support.s, cpufunc.h: No longer use gs override to enforce range limits - compare directly against VM_MAXUSER_ADDRESS instead. The old way caused problems in preserving the gs selector...and this method is just as fast or faster.
1994-06-06 14:54:41 +00:00
}
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
/*
Move ast() and userret() to sys/kern/subr_trap.c now that they are MI.
2001-06-29 19:51:37 +00:00
* Process an asynchronous software trap.
* This is relatively easy.
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
* This function will return with preemption disabled.
Initial import, 0.1 + pk 0.2.4-B1
1993-06-12 14:58:17 +00:00
*/
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
void
- Change ast() to take a pointer to a trapframe like other architectures. - Don't use an atomic operation to update cnt.v_soft in ast(). This is the only place the variable is written to, and sched_lock is always held when it is written, so it is already protected and the mutex release of sched_lock asserts a memory barrier that ensures the value will be updated in a timely fashion.
2001-02-22 18:05:15 +00:00
ast(framep)
struct trapframe *framep;
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
{
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
struct thread *td = curthread;
struct proc *p = td->td_proc;
struct kse *ke = td->td_kse;
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
struct ksegrp *kg = td->td_ksegrp;
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
u_int prticks, sticks;
int sflag;
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
int flags;
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
int sig;
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
#if defined(DEV_NPX) && !defined(SMP)
int ucode;
#endif
Major update to the way synchronization is done in the kernel. Highlights include: * Mutual exclusion is used instead of spl*(). See mutex(9). (Note: The alpha port is still in transition and currently uses both.) * Per-CPU idle processes. * Interrupts are run in their own separate kernel threads and can be preempted (i386 only). Partially contributed by: BSDi (BSD/OS) Submissions by (at least): cp, dfr, dillon, grog, jake, jhb, sheldonh
2000-09-07 01:33:02 +00:00
- Change ast() to take a pointer to a trapframe like other architectures. - Don't use an atomic operation to update cnt.v_soft in ast(). This is the only place the variable is written to, and sched_lock is always held when it is written, so it is already protected and the mutex release of sched_lock asserts a memory barrier that ensures the value will be updated in a timely fashion.
2001-02-22 18:05:15 +00:00
KASSERT(TRAPF_USERMODE(framep), ("ast in kernel mode"));
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
#ifdef WITNESS
KSE Milestone 2 Note ALL MODULES MUST BE RECOMPILED make the kernel aware that there are smaller units of scheduling than the process. (but only allow one thread per process at this time). This is functionally equivalent to teh previousl -current except that there is a thread associated with each process. Sorry john! (your next MFC will be a doosie!) Reviewed by: peter@freebsd.org, dillon@freebsd.org X-MFC after: ha ha ha ha
2001-09-12 08:38:13 +00:00
if (witness_list(td))
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
panic("Returning to user mode with mutex(s) held");
#endif
mtx_assert(&Giant, MA_NOTOWNED);
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
mtx_assert(&sched_lock, MA_NOTOWNED);
- Change all callers of addupc_task() to check PS_PROFIL explicitly and remove the check from addupc_task(). It would need sched_lock while testing the flag anyways. - Always read sticks while holding sched_lock using a temporary variable where needed. - Always init prticks to 0 in ast() to quiet a warning.
2001-12-18 09:06:10 +00:00
prticks = 0; /* XXX: Quiet warning. */
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
td->td_frame = framep;
/*
* This updates the p_sflag's for the checks below in one
* "atomic" operation with turning off the astpending flag.
* If another AST is triggered while we are handling the
* AST's saved in sflag, the astpending flag will be set and
* ast() will be called again.
*/
mtx_lock_spin(&sched_lock);
sticks = ke->ke_sticks;
sflag = p->p_sflag;
flags = ke->ke_flags;
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
p->p_sflag &= ~(PS_ALRMPEND | PS_NEEDSIGCHK | PS_PROFPEND);
ke->ke_flags &= ~(KEF_ASTPENDING | KEF_NEEDRESCHED | KEF_OWEUPC);
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
cnt.v_soft++;
if (flags & KEF_OWEUPC && sflag & PS_PROFIL) {
prticks = p->p_stats->p_prof.pr_ticks;
p->p_stats->p_prof.pr_ticks = 0;
}
mtx_unlock_spin(&sched_lock);
Add some DIAGNOSTIC code. While in userland, keep the thread's ucred reference in a shadow field so that the usual place to store it is NULL. If DIAGNOSTIC is not set, the thread ucred is kept valid until the next kernel entry, at which time it is checked against the process cred and possibly corrected. Produces a BIG speedup in kernels with INVARIANTS set. (A previous commit corrected it for the non INVARIANTS case already) Reviewed by: dillon@freebsd.org
2002-02-22 23:58:22 +00:00
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
if (td->td_ucred != p->p_ucred)
cred_update_thread(td);
if (flags & KEF_OWEUPC && sflag & PS_PROFIL)
addupc_task(ke, p->p_stats->p_prof.pr_addr, prticks);
if (sflag & PS_ALRMPEND) {
PROC_LOCK(p);
psignal(p, SIGVTALRM);
PROC_UNLOCK(p);
}
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
#if defined(DEV_NPX) && !defined(SMP)
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
if (PCPU_GET(curpcb)->pcb_flags & PCB_NPXTRAP) {
atomic_clear_int(&PCPU_GET(curpcb)->pcb_flags,
PCB_NPXTRAP);
ucode = npxtrap();
if (ucode != -1) {
trapsignal(p, SIGFPE, ucode);
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
}
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
}
Convert npx interrupts into traps instead of vice versa. This is much simpler for npx exceptions that start as traps (no assembly required...) and works better for npx exceptions that start as interrupts (there is no longer a problem for nested interrupts). Submitted by: original (pre-SMPng) version by luoqi
2001-05-22 21:20:49 +00:00
#endif
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
if (sflag & PS_PROFPEND) {
PROC_LOCK(p);
psignal(p, SIGPROF);
PROC_UNLOCK(p);
}
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
if (flags & KEF_NEEDRESCHED) {
mtx_lock_spin(&sched_lock);
td->td_priority = kg->kg_user_pri;
setrunqueue(td);
p->p_stats->p_ru.ru_nivcsw++;
mi_switch();
mtx_unlock_spin(&sched_lock);
}
if (sflag & PS_NEEDSIGCHK) {
PROC_LOCK(p);
CURSIG() is not a macro so rename it cursig(). Obtained from: KSE tree
2002-05-29 23:44:32 +00:00
while ((sig = cursig(p)) != 0)
Moved signal handling and rescheduling from userret() to ast() so that they aren't in the usual path of execution for syscalls and traps. The main complication for this is that we have to set flags to control ast() everywhere that changes the signal mask. Avoid locking in userret() in most of the remaining cases. Submitted by: luoqi (first part only, long ago, reorganized by me) Reminded by: dillon
2002-04-04 17:49:48 +00:00
postsig(sig);
PROC_UNLOCK(p);
}
The biggie: Get rid of the UPAGES from the top of the per-process address space. (!) Have each process use the kernel stack and pcb in the kvm space. Since the stacks are at a different address, we cannot copy the stack at fork() and allow the child to return up through the function call tree to return to user mode - create a new execution context and have the new process begin executing from cpu_switch() and go to user mode directly. In theory this should speed up fork a bit. Context switch the tss_esp0 pointer in the common tss. This is a lot simpler since than swithching the gdt[GPROC0_SEL].sd.sd_base pointer to each process's tss since the esp0 pointer is a 32 bit pointer, and the sd_base setting is split into three different bit sections at non-aligned boundaries and requires a lot of twiddling to reset. The 8K of memory at the top of the process space is now empty, and unmapped (and unmappable, it's higher than VM_MAXUSER_ADDRESS). Simplity the pmap code to manage process contexts, we no longer have to double map the UPAGES, this simplifies and should measuably speed up fork(). The following parts came from John Dyson: Set PG_G on the UPAGES that are now in kernel context, and invalidate them when swapping them out. Move the upages object (upobj) from the vmspace to the proc structure. Now that the UPAGES (pcb and kernel stack) are out of user space, make rfork(..RFMEM..) do what was intended by sharing the vmspace entirely via reference counting rather than simply inheriting the mappings.
1997-04-07 07:16:06 +00:00
Style fixes purposefully left out of last commit. I checked the kse tree and didn't see any changes that this conflicts with.
2002-03-29 16:45:03 +00:00
userret(td, framep, sticks);
- Close races with signals and other AST's being triggered while we are in the process of exiting the kernel. The ast() function now loops as long as the PS_ASTPENDING or PS_NEEDRESCHED flags are set. It returns with preemption disabled so that any further AST's that arrive via an interrupt will be delayed until the low-level MD code returns to user mode. - Use u_int's to store the tick counts for profiling purposes so that we do not need sched_lock just to read p_sticks. This also closes a problem where the call to addupc_task() could screw up the arithmetic due to non-atomic reads of p_sticks. - Axe need_proftick(), aston(), astoff(), astpending(), need_resched(), clear_resched(), and resched_wanted() in favor of direct bit operations on p_sflag. - Fix up locking with sched_lock some. In addupc_intr(), use sched_lock to ensure pr_addr and pr_ticks are updated atomically with setting PS_OWEUPC. In ast() we clear pr_ticks atomically with clearing PS_OWEUPC. We also do not grab the lock just to test a flag. - Simplify the handling of Giant in ast() slightly. Reviewed by: bde (mostly)
2001-08-10 22:53:32 +00:00
mtx_assert(&Giant, MA_NOTOWNED);
The biggie: Get rid of the UPAGES from the top of the per-process address space. (!) Have each process use the kernel stack and pcb in the kvm space. Since the stacks are at a different address, we cannot copy the stack at fork() and allow the child to return up through the function call tree to return to user mode - create a new execution context and have the new process begin executing from cpu_switch() and go to user mode directly. In theory this should speed up fork a bit. Context switch the tss_esp0 pointer in the common tss. This is a lot simpler since than swithching the gdt[GPROC0_SEL].sd.sd_base pointer to each process's tss since the esp0 pointer is a 32 bit pointer, and the sd_base setting is split into three different bit sections at non-aligned boundaries and requires a lot of twiddling to reset. The 8K of memory at the top of the process space is now empty, and unmapped (and unmappable, it's higher than VM_MAXUSER_ADDRESS). Simplity the pmap code to manage process contexts, we no longer have to double map the UPAGES, this simplifies and should measuably speed up fork(). The following parts came from John Dyson: Set PG_G on the UPAGES that are now in kernel context, and invalidate them when swapping them out. Move the upages object (upobj) from the vmspace to the proc structure. Now that the UPAGES (pcb and kernel stack) are out of user space, make rfork(..RFMEM..) do what was intended by sharing the vmspace entirely via reference counting rather than simply inheriting the mappings.
1997-04-07 07:16:06 +00:00
}
Reference in New Issue Copy Permalink