function to return the total number of CPUs and not the highest
CPU id.
o Define mp_maxid based on the minimum of the actual number of
CPUs in the system and MAXCPU.
o In cpu_mp_add, when the CPU id of the CPU we're trying to add
is larger than mp_maxid, don't add the CPU. Formerly this was
based on MAXCPU. Don't count CPUs when we add them. We already
know how many CPUs exist.
o Replace MAXCPU with mp_maxid when used in loops that iterate
over the id space. This avoids a couple of useless iterations.
o In cpu_mp_unleash, use the number of CPUs to determine if we
need to launch the CPUs.
o Remove mp_hardware as it's not used anymore.
o Move the IPI vector array from mp_machdep.c to sal.c. We use
the array as a centralized place to collect vector assignments.
Note that we still assign vectors to SMP specific IPIs in
non-SMP configurations. Rename the array from mp_ipi_vector to
ipi_vector.
o Add IPI_MCA_RENDEZ and IPI_MCA_CMCV. These are used by MCA.
Note that IPI_MCA_CMCV is not SMP specific.
o Initialize the ipi_vector array so that we place the IPIs in
sensible priority classes. The classes are relative to where
the AP wake-up vector is located to guarantee that it's the
highest priority (external) interrupt. Class assignment is
as follows:
class IPI notes
x AP wake-up (normally x=15)
x-1 MCA rendezvous
x-2 AST, Rendezvous, stop
x-3 CMCV, test
o Create pcb_save as the backend for savectx and cpu_switch.
o While here, use explicit bundling for pcb_save and optimize
for compactness (~87% density).
o Not part of the commit is a backend pcb_restore. restorectx()
still jumps halfway into cpu_switch().
pmap_ensure_rid(). This can happen because the function is
called for both user and kernel addresses, while the rid array
only has room for user addresses. This bug got exposed by rev
1.58 of ia64/ia64/pmap.c and rev 1.8 of ia64/include/pmap.h.
only for exceptions.
While adding this to exception_save and exception_restore, it was hard
to find a good place to put the instructions. The code sequence was
sufficiently arbitrarily ordered that the density was low (roughly 67%).
No explicit bundling was used.
Thus, I rewrote the functions to optimize for density (close to 80% now),
and added explicit bundles and nop instructions. The immediate operand
on the nop instruction has been incremented with each instance, to make
debugging a bit easier when looking at recurring patterns. Redundant
stops have been removed as much as possible. Future optimizations can
focus more on performance. A well-placed lfetch can make all the
difference here!
Also, the FRAME_Fxx defines in frame.h were mostly bogus. FRAME_F10 to
FRAME_F15 were copied from FRAME_F9 and still had the same index. We
don't use them yet, so nothing was broken.
i386/ia64/alpha - catch up to sparc64/ppc:
- replace pmap_kernel() with refs to kernel_pmap
- change kernel_pmap pointer to (&kernel_pmap_store)
(this is a speedup since ld can set these at compile/link time)
all platforms (as suggested by jake):
- gc unused pmap_reference
- gc unused pmap_destroy
- gc unused struct pmap.pm_count
(we never used pm_count - we track address space sharing at the vmspace)
collected at boot and made available through sysctl(8). At the
moment, the following MIB names are created:
hw.mca.count - The number of error records collected.
hw.mca.first - The lowest sequence number present.
hw.mca.last - The highest sequence number present.
hw.mca.<X> - The error record with sequence number <X>.
Using sysctl(8) allows us to easily detect and analyze the records,
which is very helpful during development of MCA but can also be used
in production as a way to collect machine health statistics.
the symbol index defined by the relocation. The elf_lookup() support
function is to be used by elf_reloc() when symbol lookups need to be
done. The elf_lookup() function operates on the symbol index and
will do a symbol name based lookup when such is required, otherwise
it uses the symbol index directly. This solves the problem seen on
ia64 where the symbol hash table does not contain local symbols and
a symbol name based lookup would fail for those symbols.
Don't pass the symbol name to elf_reloc(), as it isn't used any more.
check handling. In its current form, it only determines the largest
amount of state information it can get from SAL and allocates a region
7 memory block for it.
The next steps involve:
o get and log any unconsumed (NVM stored) error records across
reboots,
o register an OS_MCA handler and enable machine checks.
the SMP case. While on the subject, remove unnecessary stops. I don't
know if this resolves the memory corruption I'm seeing, but it does
have the potential. We'll see...
both Elf_Rel and Elf_Rela types of relocation, so handle them both
even though we only have Rel_Rela ATM. We don't handle 32-bit and
big-endian variants yet. Support for that is not trivial enough to
implement it without any evidence that we ever need it in the near
future.
For the FPTR relocations, we currently use the fptr_storage used by
_reloc() is locore.s. This is in no way a real solution, but for now
provides the service we need to get the basics going.
A static recursive function lookup_fdesc() is used to find the address
of a function in a way that keeps track of the load module so that
we can get the correct GP value if we need to construct an OPD (ie
there's no OPD yet for the function.
For simplicity, we create an OPD for the IPLT relocations as well and
simply fill the user provided function descriptor from the OPD. Since
the the official descriptors are unique, this has no bad side effects.
Note that we ignore the addend for FPTR relocations, but use the
addend for IPLT relocations as an offset to the function address.
This commit allows us to load and relocate modules and modules appear
to work correctly, although we probably need to make sure that we set
GP correctly in all cases when we have inter-module calls. This
especially applies to assembly coded functions that have cross module
calls.
here mostly mirror the changes made in
boot/efi/libefi/arch/ia64/start.S rev 1.5
Significant difference: We don't handle the IPLT relocation here.
For barebones KLD support, we make the fptr_storage global.
environment needed at boot time to a dynamic subsystem when VM is
up. The dynamic kernel environment is protected by an sx lock.
This adds some new functions to manipulate the kernel environment :
freeenv(), setenv(), unsetenv() and testenv(). freeenv() has to be
called after every getenv() when you have finished using the string.
testenv() only tests if an environment variable is present, and
doesn't require a freeenv() call. setenv() and unsetenv() are self
explanatory.
The kenv(2) syscall exports these new functionalities to userland,
mainly for kenv(1).
Reviewed by: peter
and pmap_copy_page(). This gets rid of a couple more physical addresses
in upper layers, with the eventual aim of supporting PAE and dealing with
the physical addressing mostly within pmap. (We will need either 64 bit
physical addresses or page indexes, possibly both depending on the
circumstances. Leaving this to pmap itself gives more flexibilitly.)
Reviewed by: jake
Tested on: i386, ia64 and (I believe) sparc64. (my alpha was hosed)
o Use chunk instead of region when we talk about a memory range.
Region can be confused with region register and we already
call it chunk in machdep.c
o Update the twiddle every 16MB
without a few patches for the rest of the kernel to allow the image
activator to override exec_copyout_strings and setregs.
None of the syscall argument translation has been done. Possibly, this
translation layer can be shared with any platform that wants to support
running ILP32 binaries on an LP64 host (e.g. sparc32 binaries?)
to be sure that it is always correct and this was not true for the first
call to cpu_switch. When thread0 resumed later, it ended up calling
pmap_install with a null pmap, which is bad.
_BYTE_ORDER. These are far more useful than their non-underscored
equivalents as these can be used in restricted namespace environments.
Mark the non-underscored variants as deprecated.
the program headers. As a result of this, dumplo was advanced too
much causing the end of the dump and most notably the trailing
dump header to be written beyond the end of the the dump medium.
most cases NULL is passed, but in some cases such as network driver locks
(which use the MTX_NETWORK_LOCK macro) and UMA zone locks, a name is used.
Tested on: i386, alpha, sparc64
In the i386 case, options BOOTP requires options NFS_ROOT as well as
options NFSCLIENT. With *both* the NFS options, a bootpc_init()
prototype is brought in by nfsclient/nfsdiskless.h.
In the ia64 case, it just doesn't work and my change just pushes it
further away from working.
Suggested to be wrong by: bde
they aren't in the usual path of execution for syscalls and traps.
The main complication for this is that we have to set flags to control
ast() everywhere that changes the signal mask.
Avoid locking in userret() in most of the remaining cases.
Submitted by: luoqi (first part only, long ago, reorganized by me)
Reminded by: dillon
in dump byte order (=network byte order). Swap blocksize and dumptime
to avoid extraneous padding on 64-bit architectures. Use CTASSERT
instead of runtime checks to make sure the header is 512 bytes large.
Various style(9) fixes.
Reviewed by: phk, bde, mike
emitted the total number of pages it still had to dump prior to
dumping a block of up to 16 pages. For a 128MB region this would
result in 8M number of printf()s. Barf!
The problem in general is that memory typically has one really
big region and a number of "scattered" smaller regions. Some may
even be just a few pages. The twiddle works best for now, but
it doesn't really give a good progress indication for the large
regions. Those are the cases where you definitely want good PI
to avoid having the user turn into a twiddle :-)
various machdep.c's to being declared in kern_mutex.c.
- Add a new function mutex_init() used to perform early initialization
needed for mutexes such as setting up thread0's contested lock list
and initializing MI mutexes. Change the various MD startup routines
to call this function instead of duplicating all the code themselves.
Tested on: alpha, i386
constructs an ELF image, consisting of the ELF header, for
each memory region a program header, followed by the memory
contents for each region. It does blocked I/O for the headers
as they are typically smaller than DEV_BSIZE.
and cpu_critical_exit() and moves associated critical prototypes into their
own header file, <arch>/<arch>/critical.h, which is only included by the
three MI source files that need it.
Backout and re-apply improperly comitted syntactical cleanups made to files
that were still under active development. Backout improperly comitted program
structure changes that moved localized declarations to the top of two
procedures. Partially re-apply one of the program structure changes to
move 'mask' into an intermediate block rather then in three separate
sub-blocks to make the code more readable. Re-integrate bug fixes that Jake
made to the sparc64 code.
Note: In general, developers should not gratuitously move declarations out
of sub-blocks. They are where they are for reasons of structure, grouping,
readability, compiler-localizability, and to avoid developer-introduced bugs
similar to several found in recent years in the VFS and VM code.
Reviewed by: jake
general cleanup of the API. The entire API now consists of two functions
similar to the pre-KSE API. The suser() function takes a thread pointer
as its only argument. The td_ucred member of this thread must be valid
so the only valid thread pointers are curthread and a few kernel threads
such as thread0. The suser_cred() function takes a pointer to a struct
ucred as its first argument and an integer flag as its second argument.
The flag is currently only used for the PRISON_ROOT flag.
Discussed on: smp@
bootinfo block in register r8. In locore.s we save the address
in the global variable 'pa_bootinfo'. In machdep.c we compare
this value against the hardwired address, but don't depend on its
validity yet (ie: we still expect the bootinfo block to be at the
hardwired address). After a small amount of time, we'll flip the
switch and depend on the loader to pass us the address. From that
moment on the loader is free to put it anywhere it likes, provided
the machine itself likes it as well.
Add some verbosity to aid in the transition. We emit a message if
the loader didn't pass the address and we also emit a message if
there's no bootinfo block at the hardwired address.
While in locore.s, reduce the number of redundant serialization
instructions. A srlz.i is a proper superset of a srlz.d and thus
is a valid replacement. Also slightly reorder the movl instructions
to improve bundle density.
back into the calling MD code. The MD code must ensure no races between
checking the astpening flag and returning to usermode.
Submitted by: peter (ia64 bits)
Tested on: alpha (peter, jeff), i386, ia64 (peter), sparc64
with this flag. Remove the dup_list and dup_ok code from subr_witness. Now
we just check for the flag instead of doing string compares.
Also, switch the process lock, process group lock, and uma per cpu locks over
to this interface. The original mechanism did not work well for uma because
per cpu lock names are unique to each zone.
Approved by: jhb
disablement assumptions in kern_fork.c by adding another API call,
cpu_critical_fork_exit(). Cleanup the td_savecrit field by moving it
from MI to MD. Temporarily move cpu_critical*() from <arch>/include/cpufunc.h
to <arch>/<arch>/critical.c (stage-2 will clean this up).
Implement interrupt deferral for i386 that allows interrupts to remain
enabled inside critical sections. This also fixes an IPI interlock bug,
and requires uses of icu_lock to be enclosed in a true interrupt disablement.
This is the stage-1 commit. Stage-2 will occur after stage-1 has stabilized,
and will move cpu_critical*() into its own header file(s) + other things.
This commit may break non-i386 architectures in trivial ways. This should
be temporary.
Reviewed by: core
Approved by: core
Instead of caching the ucred reference, just go ahead and eat the
decerement and increment of the refcount. Now that Giant is pushed down
into crfree(), we no longer have to get Giant in the common case. In the
case when we are actually free'ing the ucred, we would normally free it on
the next kernel entry, so the cost there is not new, just in a different
place. This also removse td_cache_ucred from struct thread. This is
still only done #ifdef DIAGNOSTIC.
Tested on: i386, alpha
a bit before handing it over to the OS. I occasionally have 11
segments with several 8K or so fragments depending on nvram settings and
what I have done under loader(8) before booting. This needs to be
revisited.
a language feature that I do not know about. gcc is complaining about
a left shift >= sizeof type, even when shifting a (cast) 64 bit type left
by 43 bits.
lose if a process is preempted while pmap is temporarily switched to
another pmap.
* For SMP, drop the high-fp state when a thread is switched away from
so that if another cpu resumes that thread, it doesn't have to play
games with IPI to get ahold of the correct register values.
* Don't call ast() from interrupt() - if we switch, then we will miss
writing cr.eoi which will prevent the current cpu from receiving
interrupts until the current thread is resumed. The call to ast()
happens magically in exception_restore where it is safe.
* Add DDB 'show irq' command to examine interrupt hardware state.
* Use ptc.g instead of ptc.l so that TLB shootdowns are broadcast to the
coherence domain.
* Use smp_rendezvous for pmap_invalidate_all to ensure it happens on all
cpus.
* Dike out a DIAGNOSTIC printf which didn't compile.
* Protect the internals of pmap_install with cpu_critical_enter/exit.
o In i386's <machine/endian.h>, macros have some advantages over
inlines, so change some inlines to macros.
o In i386's <machine/endian.h>, ungarbage collect word_swap_int()
(previously __uint16_swap_uint32), it has some uses on i386's with
PDP endianness.
Submitted by: bde
o Move a comment up in <machine/endian.h> that was accidentially moved
down a few revisions ago.
o Reenable userland's use of optimized inline-asm versions of
byteorder(3) functions.
o Fix ordering of prototypes vs. redefinition of byteorder(3)
functions, so that the non-GCC (libc asm) case has proper
prototypes.
o Add proper prototypes for byteorder(3) functions in <sys/param.h>.
o Prevent redundant duplicate prototypes by making use of the
_BYTEORDER_PROTOTYPED define.
o Move the bswap16(), bswap32(), bswap64() C functions into MD space
for platforms in which asm versions don't exist. This significantly
reduces the complexity of some things at the cost of duplicate code.
Reviewed by: bde
* Move the section which manipulates ia64_pal_base to after cninit() so
that we don't risk printing anything before we have a console.
* Don't call ia64_probe_sapics() for a SKI build. This should really
be dependant on ACPICA being present or something.
In order to determine what to page out, the vm_daemon checks
reference bits on all pages belonging to all processes. Unfortunately,
the algorithm used reacted badly with shared pages; each shared page
would be checked once per process sharing it; this caused an O(N^2)
growth of tlb invalidations. The algorithm has been changed so that
each page will be checked only 16 times.
Prior to this change, a fork/sleepbomb of 1300 processes could cause
the vm_daemon to take over 60 seconds to complete, effectively
freezing the system for that time period. With this change
in place, the vm_daemon completes in less than a second. Any system
with hundreds of processes sharing pages should benefit from this change.
Note that the vm_daemon is only run when the system is under extreme
memory pressure. It is likely that many people with loaded systems saw
no symptoms of this problem until they reached the point where swapping
began.
Special thanks go to dillon, peter, and Chuck Cranor, who helped me
get up to speed with vm internals.
PR: 33542, 20393
Reviewed by: dillon
MFC after: 1 week
