d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
62ae6c89ad
freebsd-dev/sys/i386/include/lock.h

186 lines
4.8 KiB
C
Raw Normal View History

Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
/*
* Copyright (c) 1997, by Steve Passe
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. The name of the developer may NOT be used to endorse or promote products
* derived from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
$Id$ -> $FreeBSD$
1999-08-28 01:08:13 +00:00
* $FreeBSD$
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
*/
#ifndef _MACHINE_LOCK_H_
#define _MACHINE_LOCK_H_
#ifdef LOCORE
#ifdef SMP
#define MPLOCKED lock ;
/*
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
* Some handy macros to allow logical organization.
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
*/
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
#define MP_LOCK call _get_mplock
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
#define MP_TRYLOCK \
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
pushl $_mp_lock ; /* GIANT_LOCK */ \
call _MPtrylock ; /* try to get lock */ \
add $4, %esp
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
#define MP_RELLOCK \
Optimize two cases in the MP locking code. First, it is not necessary to use a locked cmpexg when unlocking a lock that we already hold, since nobody else can touch the lock while we hold it. Second, it is not necessary to use a locked cmpexg when locking a lock that we already hold, for the same reason. These changes will allow MP locks to be used recursively without impacting performance. Modify two procedures that are called only by assembly and are already NOPROF entries to pass a critical argument in %edx instead of on the stack, removing a significant amount of code from the critical path as a consequence. Reviewed by: Alfred Perlstein <bright@wintelcom.net>, Peter Wemm <peter@netplex.com.au>
1999-11-19 22:47:19 +00:00
movl $_mp_lock,%edx ; /* GIANT_LOCK */ \
call _MPrellock_edx
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
/*
* Protects the IO APIC and apic_imen as a critical region.
*/
#define IMASK_LOCK \
pushl $_imen_lock ; /* address of lock */ \
call _s_lock ; /* MP-safe */ \
addl $4, %esp
#define IMASK_UNLOCK \
Modify the macros IMASK_UNLOCK, CPL_UNLOCK, and REL_FAST_INTR_LOCK to perform the s_unlock inline.
1999-08-23 19:14:18 +00:00
movl $0, _imen_lock
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#else /* SMP */
#define MPLOCKED /* NOP */
Commit major SMP cleanups and move the BGL (big giant lock) in the syscall path inward. A system call may select whether it needs the MP lock or not (the default being that it does need it). A great deal of conditional SMP code for various deadended experiments has been removed. 'cil' and 'cml' have been removed entirely, and the locking around the cpl has been removed. The conditional separately-locked fast-interrupt code has been removed, meaning that interrupts must hold the CPL now (but they pretty much had to anyway). Another reason for doing this is that the original separate-lock for interrupts just doesn't apply to the interrupt thread mechanism being contemplated. Modifications to the cpl may now ONLY occur while holding the MP lock. For example, if an otherwise MP safe syscall needs to mess with the cpl, it must hold the MP lock for the duration and must (as usual) save/restore the cpl in a nested fashion. This is precursor work for the real meat coming later: avoiding having to hold the MP lock for common syscalls and I/O's and interrupt threads. It is expected that the spl mechanisms and new interrupt threading mechanisms will be able to run in tandem, allowing a slow piecemeal transition to occur. This patch should result in a moderate performance improvement due to the considerable amount of code that has been removed from the critical path, especially the simplification of the spl*() calls. The real performance gains will come later. Approved by: jkh Reviewed by: current, bde (exception.s) Some work taken from: luoqi's patch
2000-03-28 07:16:37 +00:00
#define MP_LOCK /* NOP */
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#endif /* SMP */
#else /* LOCORE */
#ifdef SMP
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#include <machine/smptests.h> /** xxx_LOCK */
/*
* Locks regions protected in UP kernel via cli/sti.
*/
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#ifdef USE_MPINTRLOCK
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#define MPINTR_LOCK() s_lock(&mpintr_lock)
#define MPINTR_UNLOCK() s_unlock(&mpintr_lock)
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
#else
#define MPINTR_LOCK()
#define MPINTR_UNLOCK()
#endif /* USE_MPINTRLOCK */
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/*
* sio/cy lock.
* XXX should rc (RISCom/8) use this?
*/
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#ifdef USE_COMLOCK
#define COM_LOCK() s_lock(&com_lock)
#define COM_UNLOCK() s_unlock(&com_lock)
#define COM_DISABLE_INTR() \
{ __asm __volatile("cli" : : : "memory"); COM_LOCK(); }
#define COM_ENABLE_INTR() \
{ COM_UNLOCK(); __asm __volatile("sti"); }
#else
#define COM_LOCK()
#define COM_UNLOCK()
#define COM_DISABLE_INTR() disable_intr()
#define COM_ENABLE_INTR() enable_intr()
#endif /* USE_COMLOCK */
General cleanup of the lock pushdown code. They are grouped and enabled from machine/smptests.h: #define PUSHDOWN_LEVEL_1 #define PUSHDOWN_LEVEL_2 #define PUSHDOWN_LEVEL_3 #define PUSHDOWN_LEVEL_4_NOT
1997-09-07 22:04:09 +00:00
/*
* Clock hardware/struct lock.
* XXX pcaudio and friends still need this lock installed.
*/
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#ifdef USE_CLOCKLOCK
#define CLOCK_LOCK() s_lock(&clock_lock)
#define CLOCK_UNLOCK() s_unlock(&clock_lock)
#define CLOCK_DISABLE_INTR() \
{ __asm __volatile("cli" : : : "memory"); CLOCK_LOCK(); }
#define CLOCK_ENABLE_INTR() \
{ CLOCK_UNLOCK(); __asm __volatile("sti"); }
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#else
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#define CLOCK_LOCK()
#define CLOCK_UNLOCK()
#define CLOCK_DISABLE_INTR() disable_intr()
#define CLOCK_ENABLE_INTR() enable_intr()
#endif /* USE_CLOCKLOCK */
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#else /* SMP */
General cleanup of the sub-system locking macros. Eliminated the RECURSIVE_MPINTRLOCK. clock.c and microtime use clock_lock. sio.c and cy.c use com_lock. Suggestions by: Bruce Evans <bde@zeta.org.au>
1997-09-01 07:45:37 +00:00
#define MPINTR_LOCK()
#define MPINTR_UNLOCK()
#define COM_LOCK()
#define COM_UNLOCK()
#define CLOCK_LOCK()
#define CLOCK_UNLOCK()
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#endif /* SMP */
/*
* Simple spin lock.
* It is an error to hold one of these locks while a process is sleeping.
*/
struct simplelock {
volatile int lock_data;
};
/* functions in simplelock.s */
void s_lock_init __P((struct simplelock *));
void s_lock __P((struct simplelock *));
int s_lock_try __P((struct simplelock *));
void ss_lock __P((struct simplelock *));
void ss_unlock __P((struct simplelock *));
Add support for low resolution SMP kernel profiling. - A nonprofiling version of s_lock (called s_lock_np) is used by mcount. - When profiling is active, more registers are clobbered in seemingly simple assembly routines. This means that some callers needed to save/restore extra registers. - The stack pointer must have space for a 'fake' return address in idle, to avoid stack underflow.
1997-12-15 02:18:35 +00:00
void s_lock_np __P((struct simplelock *));
void s_unlock_np __P((struct simplelock *));
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
Make "s_unlock" an inline function. (Inlining this function takes less space than calling it. A callable version still exists for use by some assembly code.)
1999-08-22 05:37:18 +00:00
/* inline simplelock functions */
static __inline void
s_unlock(struct simplelock *lkp)
{
lkp->lock_data = 0;
}
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
/* global data in mp_machdep.c */
extern struct simplelock imen_lock;
extern struct simplelock cpl_lock;
extern struct simplelock fast_intr_lock;
extern struct simplelock intr_lock;
Debug version of simple_lock. This will store the CPU id of the holding CPU along with the lock. When a CPU fails to get the lock it compares its own id to the holder id. If they are the same it panic()s, as simple locks are binary, and this would cause a deadlock. Controlled by smptests.h: SL_DEBUG, ON by default. Some minor cleanup.
1997-08-31 03:17:48 +00:00
extern struct simplelock clock_lock;
extern struct simplelock com_lock;
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
extern struct simplelock mpintr_lock;
Add support for low resolution SMP kernel profiling. - A nonprofiling version of s_lock (called s_lock_np) is used by mcount. - When profiling is active, more registers are clobbered in seemingly simple assembly routines. This means that some callers needed to save/restore extra registers. - The stack pointer must have space for a 'fake' return address in idle, to avoid stack underflow.
1997-12-15 02:18:35 +00:00
extern struct simplelock mcount_lock;
Another round of lock pushdown. Add a simplelock to deal with disable_intr()/enable_intr() as used in UP kernel. UP kernel expects that this is enough to guarantee exclusive access to regions of code bracketed by these 2 functions. Add a simplelock to bracket clock accesses in clock.c: clock_lock. Help from: Bruce Evans <bde@zeta.org.au>
1997-08-30 08:08:10 +00:00
#if !defined(SIMPLELOCK_DEBUG) && NCPUS > 1
/*
* This set of defines turns on the real functions in i386/isa/apic_ipl.s.
*/
#define simple_lock_init(alp) s_lock_init(alp)
#define simple_lock(alp) s_lock(alp)
#define simple_lock_try(alp) s_lock_try(alp)
#define simple_unlock(alp) s_unlock(alp)
#endif /* !SIMPLELOCK_DEBUG && NCPUS > 1 */
#endif /* LOCORE */
#endif /* !_MACHINE_LOCK_H_ */
Reference in New Issue Copy Permalink