- style(9) fixes to mips/rmi platform files
- update pic.h to add pic_setup_intr() and use pic_setup_intr() for setting
up interrupts which are routed thru PIC.
- remove rmi_spin_mutex_safe and haslock, and make sure that the functions
are called only after mutexes are available.
- move PIC code to xlr_machdep.c
- move fast message ring code completely to on_chip.c
- move memory initialization to a new function xlr_mem_init()
- style fixes
* Add a function to write to the relevant PLL register
* Break out the PLL configuration for the AR71XX into the CPU ops,
lifted from if_arge.c.
* Add the AR91XX PLL configuration ops, using the AR91XX register
definitions.
This is untested but should at least allow an AR724X to boot.
The current code is lacking the detail needed to expose the PCIe bus.
It is also lacking any NIC, PLL or flush/WB code.
This works well enough to bring a system up to single-user mode
using an MDROOT.
Known Issues:
* The EHCI USB doesn't currently work and will panic the kernel during
attach.
* The onboard ethernet won't work until the PLL routines have been
fleshed out and shoe-horned into if_arge.
* The WMAC device glue (and quite likely the if_ath support)
hasn't yet been implemented.
* Implement a SoC probe function, from Linux, which determines the
SoC family, type and revision. This only probes the AR71xx series
SoC and (currently) panics on others.
* Migrate some of the AR71XX specific hardware init (USB device, determining
system frequencies) into using the cpuops introduced in an earlier commit.
Other SoC specific hardware stuff (per-device flush/WB, GPIO pin wiring,
Ethernet PLL setup, other things I've likely missed) will be introduced in
subsequent commits.
Reviewed by: imp@
Obtained from: (partially) Linux
1. On n64, use XKPHYS to map page table pages instead of KSEG0. Maintain
just one freepages list on n64.
The changes are mainly to introduce MIPS_PHYS_TO_DIRECT(pa),
MIPS_DIRECT_TO_PHYS(), which will use KSEG0 in 32 bit compilation
and XKPHYS in 64 bit compilation.
2. Change macro based PMAP_LMEM_MAP1(), PMAP_LMEM_MAP2(), PMAP_LMEM_UNMAP()
to inline functions.
3. Introduce MIPS_DIRECT_MAPPABLE(pa), which will further reduce the cases
in which we will need to have a special case for 64 bit compilation.
4. Update CP0 hazard definitions for CPU_RMI - the cpu does not need any
nops
Reviewed by: neel
Each of these SoCs have different devices, different hardware initialisation
methods and, quite likely, different quirks. These functions will abstract
out the SoC differences and keep these differences out of the drivers (eg
USB init, if_arge, etc.)
In particular, provide pagesize and pagesizes array, the canary value
for SSP use, number of host CPUs and osreldate.
Tested by: marius (sparc64)
MFC after: 1 month
- Enable KX and UX bits on CPU startup for non-boot CPUs
- Keep the KX bit when in userspace - XTLB handler needs it to access
PCPU data
- revert r210638 partly - we don't need to enable KX on kernel entry
now
Reviewed by: jmallett, imp
1. Move dirty bit emulation code that is duplicted for kernel and user
in trap.c to a function pmap_emulate_modified() in pmap.c.
2. While doing dirty bit emulation, it is not necessary to update the
TLB entry on all CPUs using smp_rendezvous(), we can just update the
TLB entry on the current CPU, and let the other CPUs update their TLB
entry lazily if they get an exception.
Reviewed by: alc, neel
per-cpu variants are also available to be called. The per-cpu variants
are needed for some later optimizations.
Also remove unnecessary casts, do some style fixes.
Reviewed by: alc, neel
r211130 in favor of this more general fix.
This fixes a compilation error for mips 64-bit little endian build.
libexec/rtld-elf/mips/reloc.c:196: warning: right shift count >= width of type
Suggested by: stefanf, jchandra, bde
IPI to a specific CPU by its cpuid. Replace calls to ipi_selected() that
constructed a mask for a single CPU with calls to ipi_cpu() instead. This
will matter more in the future when we transition from cpumask_t to
cpuset_t for CPU masks in which case building a CPU mask is more expensive.
Submitted by: peter, sbruno
Reviewed by: rookie
Obtained from: Yahoo! (x86)
MFC after: 1 month
MIPS doesn't really need to use atomic_cmpset_int() in situations like
this because the software dirty bit emulation in trap.c acquires
the pmap lock. Atomics like this appear to be a carryover from i386
where the hardware-managed TLB might concurrently set the modified bit.
Reviewed by: alc
pmap_page_wired_mappings() counts the number of pv entries for the
specified page that have the pv entry wired flag set to TRUE.
pmap_enter() correctly initializes this flag. However,
pmap_change_wiring() doesn't update the corresponding pv entry flag,
only the PTE. So, the count returned by pmap_page_wired_mappings()
will sometimes be wrong.
In the short term, the best fix would be to eliminate the pv entry
flag and use only the PTE. That flag is wasting non-trivial memory.
Remove pv_wired flag, and use PTE flag to count the wired mappings.
Reviewed by: alc
'counter_upper' and 'counter_lower_last'. The race exists because
interrupts are enabled even though tick_ticker() executes in a
critical section.
Fix a bug in clock_intr() in how it updates the cached values of
'counter_upper' and 'counter_lower_last'. They are updated only
when the COUNT register rolls over. More interestingly it will *never*
update the cached values if 'counter_lower_last' happens to be zero.
Get rid of superfluous critical section in clock_intr(). There is no
reason to do this because clock_intr() executes in hard interrupt
context.
Switch back to using 'tick_ticker()' as the cpu ticker for Sibyte.
Reviewed by: jmallett, mav
- 32 bit compilation will still use old 2 level page tables
- re-arrange pmap code so that adding another level is easier
- pmap code for 3 level page tables for n64
- update TLB handler to traverse 3 levels in n64
Reviewed by: jmallett
The emulation of 'ld' and 'sd' instructions only works for ABIs that support
64-bit registers and the instructions 'ldl' and 'ldr' that operate on those
registers.
Reviewed by: jmallett
that with a 32-bit ABI on a system with 64-bit registers can attempt to
access an invalid (well, kernel) memory address rather than the intended
user address for stack-relative loads and stores. Lowering the stack
pointer works around this. [1]
o) Make TRAP_DEBUG code conditional on the trap_debug variable. Make
trap_debug default to 0 instead of 1 now but make it possible to change it
at runtime using sysctl.
o) Kill programs that attempt an unaligned access of a kernel address. Note
that with some ABIs, calling useracc() is not sufficient since the register
may be 64-bit but vm_offset_t is 32-bit so a kernel address could be
truncated to what looks like a valid user address, allowing the user to
crash the kernel.
o) Clean up unaligned access emulation to support unaligned 16-bit and 64-bit
accesses. (For 16-bit accesses it was checking for user access to too much
memory (4 bytes) and there was no 64-bit support.) This still lacks support
for unaligned load-linked and store-conditional.
Reviewed by: [1] gonzo
now it uses a very dumb first-touch allocation policy. This will change in
the future.
- Each architecture indicates the maximum number of supported memory domains
via a new VM_NDOMAIN parameter in <machine/vmparam.h>.
- Each cpu now has a PCPU_GET(domain) member to indicate the memory domain
a CPU belongs to. Domain values are dense and numbered from 0.
- When a platform supports multiple domains, the default freelist
(VM_FREELIST_DEFAULT) is split up into N freelists, one for each domain.
The MD code is required to populate an array of mem_affinity structures.
Each entry in the array defines a range of memory (start and end) and a
domain for the range. Multiple entries may be present for a single
domain. The list is terminated by an entry where all fields are zero.
This array of structures is used to split up phys_avail[] regions that
fall in VM_FREELIST_DEFAULT into per-domain freelists.
- Each memory domain has a separate lookup-array of freelists that is
used when fulfulling a physical memory allocation. Right now the
per-domain freelists are listed in a round-robin order for each domain.
In the future a table such as the ACPI SLIT table may be used to order
the per-domain lookup lists based on the penalty for each memory domain
relative to a specific domain. The lookup lists may be examined via a
new vm.phys.lookup_lists sysctl.
- The first-touch policy is implemented by using PCPU_GET(domain) to
pick a lookup list when allocating memory.
Reviewed by: alc
booting again.
The code is a copy of the mips/mips/tick.c with minor modifications for
XLR interrupt handling. Disable mips/rmi/clock.c for now, the PIC based
timer code will be added later.
alc@.
The UMA zone based allocation is replaced by a scheme that creates
a new free page list for the KSEG0 region, and a new function
in sys/vm that allocates pages from a specific free page list.
This also fixes a race condition introduced by the UMA based page table
page allocation code. Dropping the page queue and pmap locks before
the call to uma_zfree, and re-acquiring them afterwards will introduce
a race condtion(noted by alc@).
The changes are :
- Revert the earlier changes in MIPS pmap.c that added UMA zone for
page table pages.
- Add a new freelist VM_FREELIST_HIGHMEM to MIPS vmparam.h for memory that
is not directly mapped (in 32bit kernel). Normal page allocations will first
try the HIGHMEM freelist and then the default(direct mapped) freelist.
- Add a new function 'vm_page_t vm_page_alloc_freelist(int flind, int
order, int req)' to vm/vm_page.c to allocate a page from a specified
freelist. The MIPS page table pages will be allocated using this function
from the freelist containing direct mapped pages.
- Move the page initialization code from vm_phys_alloc_contig() to a
new function vm_page_alloc_init(), and use this function to initialize
pages in vm_page_alloc_freelist() too.
- Split the function vm_phys_alloc_pages(int pool, int order) to create
vm_phys_alloc_freelist_pages(int flind, int pool, int order), and use
this function from both vm_page_alloc_freelist() and vm_phys_alloc_pages().
Reviewed by: alc
on-board USB controller. It is not currently enabled because there are
known problems with device communication and until those are fixed I am not
certain that it won't destabilize the system. [1]
o) Add the "cryptocteon" opencrypto device based on the OCF device written by
David McCullough. It is not currently enabled because until support for
saving/restoring coprocessor 2 state on context switch is available, it runs
with interrupts disabled, which tends to pessimize performance over using a
software crypto facility. Tests using this driver which are not negatively
affected by it running with interrupts disabled show it to be substantially
faster than software for large blocks.
Submitted by: hps [1]
library:
o) Increase inline unit / large function growth limits for MIPS to accommodate
the needs of the Simple Executive, which uses a shocking amount of inlining.
o) Remove TARGET_OCTEON and use CPU_CNMIPS to do things required by cnMIPS and
the Octeon SoC.
o) Add OCTEON_VENDOR_LANNER to use Lanner's allocation of vendor-specific
board numbers, specifically to support the MR320.
o) Add OCTEON_BOARD_CAPK_0100ND to hard-wire configuration for the CAPK-0100nd,
which improperly uses an evaluation board's board number and breaks board
detection at runtime. This board is sold by Portwell as the CAM-0100.
o) Add support for the RTC available on some Octeon boards.
o) Add support for the Octeon PCI bus. Note that rman_[sg]et_virtual for IO
ports can not work unless building for n64.
o) Clean up the CompactFlash driver to use Simple Executive macros and
structures where possible (it would be advisable to use the Simple Executive
API to set the PIO mode, too, but that is not done presently.) Also use
structures from FreeBSD's ATA layer rather than structures copied from
Linux.
o) Print available Octeon SoC features on boot.
o) Add support for the Octeon timecounter.
o) Use the Simple Executive's routines rather than local copies for doing reads
and writes to 64-bit addresses and use its macros for various device
addresses rather than using local copies.
o) Rename octeon_board_real to octeon_is_simulation to reduce differences with
Cavium-provided code originally written for Linux. Also make it use the
same simplified test that the Simple Executive and Linux both use rather
than our complex one.
o) Add support for the Octeon CIU, which is the main interrupt unit, as a bus
to use normal interrupt allocation and setup routines.
o) Use the Simple Executive's bootmem facility to allocate physical memory for
the kernel, rather than assuming we know which addresses we can steal.
NB: This may reduce the amount of RAM the kernel reports you as having if
you are leaving large temporary allocations made by U-Boot allocated
when starting FreeBSD.
o) Add a port of the Cavium-provided Ethernet driver for Linux. This changes
Ethernet interface naming from rgmxN to octeN. The new driver has vast
improvements over the old one, both in performance and functionality, but
does still have some features which have not been ported entirely and there
may be unimplemented code that can be hit in everyday use. I will make
every effort to correct those as they are reported.
o) Support loading the kernel on non-contiguous cores.
o) Add very conservative support for harvesting randomness from the Octeon
random number device.
o) Turn SMP on by default.
o) Clean up the style of the Octeon kernel configurations a little and make
them compile with -march=octeon.
o) Add support for the Lanner MR320 and the CAPK-0100nd to the Simple
Executive.
o) Modify the Simple Executive to build on FreeBSD and to build without
executive-config.h or cvmx-config.h. In the future we may want to
revert part of these changes and supply executive-config.h and
cvmx-config.h and access to the options contained in those files via
kernel configuration files.
o) Modify the Simple Executive USB routines to support getting and setting
of the USB PID.
Move inappropriate stuff in cpu.h elsewhere:
{s,g}et_intr_mask -> md_var.h
num_tlbentries -> tlb.h
Remove #define clockframe trapframe and fix clock, which was the only place
this was used.
All the rest of this stuff was unused.
# we're not quite minimal yet, since we duplicate a few status register things
# here...
Inspired by: bde@
calls mips_cpu_call via an obfuscated assembler call. Instead, delete
the current cpu_throw, and rename mips_cpu_throw to cpu_throw. This
is nicer to the cache on each context switch (since fixed jumps can be
prefected, while jumps through a register can't). Incidentally, it
also saves about 5 or 6 instructions.
Reviewed by: jmallet@
Use int32/intptr casts for exception vector names.
Define MIPS_SR_INT_MASK again
Change MIPS_XKPHYS_CCA_* to MIPS_CCA_* since we can use them in many contexts
Minor gratuitous whitespace churn
The problem with setting it there is that the last CPU to come up
wins, it seems. This also removes one more ifdef in locore.S, a noble
goal too. Since they are unused, and pollute cpu.h, remove them.
Submitted by: bde.h (cpu.h pollution)
Approved in theory by: jmallet@
Merge changes for initial n64 support in pmap.c. Use direct mapped (XKPHYS)
access for a lot of operations that earlier needed temporary mapping. Add
support for using XKSEG for kernel mappings.
Reviewed by: imp
Obtained from: jmallett (http://svn.freebsd.org/base/user/jmallett/octeon)
The existing code only checked the alignment of the first mbuf and
didn't enforce the size constraints.
This commit introduces a simple function to check the alignment and
size of all mbufs in the list. This fixes the initial issue in the
PR.
PR: kern/148307
Reviewed by: gonzo@
Updated PTE/PDE macros from http://svn.freebsd.org/base/user/jmallett/octeon
Introduce pmap_segshift() macro, use pmap_segmap() in place of pmap_pde, and
remove pmap_pde().
Approved by: rrs (mentor)
Obtained from: jmallett@
If we save/restore the PageMask, the value set by the bootloader will
persist, and will cause problems later in TLB exception handler.
This caused a crash in AR71xx boards.
Also fixes the EntryHi mask in pte.h
Reported by: Luiz Otavio O Souza <lists.br@gmail.com>
Tested by: Luiz Otavio O Souza <lists.br@gmail.com>
Approved by: rrs (mentor)
Initial support for n32 and n64 ABIs from
http://svn.freebsd.org/base/user/jmallett/octeon
Changes are:
- syscall, exception and trap support for n32/n64 ABIs
- 64-bit address space defines
- _jmp_buf for n32/n64
- casts between registers and ptr/int updated to work on n32/n64
Approved by: rrs(mentor), jmallett
PTE flag cleanup from http://svn.freebsd.org/base/user/jmallett/octeon
- Rename PTE_xx flags to match their MIPS names
- Use the new pte_set/test/clear macros uniformly, instead of a mixture
of mips_pg_xxx(), pmap_pte_x() macros and direct access.
- Remove unused macros and defines from pte.h and pmap.c
Discussed on freebsd-mips@
Approved by: rrs(mentor), jmallett
* Add some per-device sysctl entries which record the watchdog state -
whether it is armed; whether the last reboot was due to the watchdog.
* Add a per-device sysctl debug flag to enable logging watchdog arming/
disarming.
Reviewed by: gonzo@
allow pmap_enter() to be performed on an unmanaged page that doesn't have
VPO_BUSY set. Having VPO_BUSY set really only matters for managed pages.
(See, for example, pmap_remove_write().)
PG_REFERENCED changes in vm_pageout_object_deactivate_pages().
Simplify this function's inner loop using TAILQ_FOREACH(), and shorten
some of its overly long lines. Update a stale comment.
Assert that PG_REFERENCED may be cleared only if the object containing
the page is locked. Add a comment documenting this.
Assert that a caller to vm_page_requeue() holds the page queues lock,
and assert that the page is on a page queue.
Push down the page queues lock into pmap_ts_referenced() and
pmap_page_exists_quick(). (As of now, there are no longer any pmap
functions that expect to be called with the page queues lock held.)
Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever
be passed an unmanaged page. Assert this rather than returning "0"
and "FALSE" respectively.
ARM:
Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH().
Push down the page queues lock inside of pmap_clearbit(), simplifying
pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write().
Additionally, this allows for avoiding the acquisition of the page
queues lock in some cases.
PowerPC/AIM:
moea*_page_exits_quick() and moea*_page_wired_mappings() will never be
called before pmap initialization is complete. Therefore, the check
for moea_initialized can be eliminated.
Push down the page queues lock inside of moea*_clear_bit(),
simplifying moea*_clear_modify() and moea*_clear_reference().
The last parameter to moea*_clear_bit() is never used. Eliminate it.
PowerPC/BookE:
Simplify mmu_booke_page_exists_quick()'s control flow.
Reviewed by: kib@
the page is managed.
Don't set the machine-independent layer's dirty field for the page being
mapped in init_pte_prot(). (The dirty field is only supposed to set when
a mapping is removed or write-protected and the page was managed and
modified.)
Determine whether or not to perform dirty bit emulation based on whether
or not the page is managed, i.e., pageable, not based on whether the page
is being mapped into the kernel address space. Nearly all of the kernel
address space consists of unmanaged pages, so this has neglible impact on
the overhead of dirty bit emulation for the kernel address space. However,
there can also exist unmanaged pages in the user address space. Previously,
dirty bit emulation was unnecessarily performed on these pages.
Tested by: jchandra@
fails to allocate MIPS page table pages. The current usage of VM_WAIT in
case of vm_phys_alloc_contig() failure is not correct, because:
"There is no guarantee that any of the available free (or cached) pages
after the VM_WAIT will fall within the range of suitable physical
addresses. Every time this function sleeps and a single page is freed
(or cached) by someone else, this function will be reawakened. With
a little bad luck, you could spin indefinitely."
We also add low and high parameters to vm_contig_grow_cache() and
vm_contig_launder() so that we restrict vm_contig_launder() to the range
of pages we are interested in.
Reported by: alc
Reviewed by: alc
Approved by: rrs (mentor)
When I pushed down the page queues lock into pmap_is_modified(), I created
an ordering dependence: A pmap operation that clears PG_WRITEABLE and calls
vm_page_dirty() must perform the call first. Otherwise, pmap_is_modified()
could return FALSE without acquiring the page queues lock because the page
is not (currently) writeable, and the caller to pmap_is_modified() might
believe that the page's dirty field is clear because it has not seen the
effect of the vm_page_dirty() call.
When I pushed down the page queues lock into pmap_is_modified(), I
overlooked one place where this ordering dependence is violated:
pmap_enter(). In a rare situation pmap_enter() can be called to replace a
dirty mapping to one page with a mapping to another page. (I say rare
because replacements generally occur as a result of a copy-on-write fault,
and so the old page is not dirty.) This change delays clearing PG_WRITEABLE
until after vm_page_dirty() has been called.
Fixing the ordering dependency also makes it easy to introduce a small
optimization: When pmap_enter() used to replace a mapping to one page with a
mapping to another page, it freed the pv entry for the first mapping and
later called the pv entry allocator for the new mapping. Now, pmap_enter()
attempts to recycle the old pv entry, saving two calls to the pv entry
allocator.
will be called automatically by 'timer1clock()'.
Do profiling as often as possible by running it as the same frequency as
'timer1hz'. The statistics clock is run as close to 128Hz as possible.
Pointed out by: mav@
pmap_is_referenced(). Eliminate the corresponding page queues lock
acquisitions from vm_map_pmap_enter() and mincore(), respectively. In
mincore(), this allows some additional cases to complete without ever
acquiring the page queues lock.
Assert that the page is managed in pmap_is_referenced().
On powerpc/aim, push down the page queues lock acquisition from
moea*_is_modified() and moea*_is_referenced() into moea*_query_bit().
Again, this will allow some additional cases to complete without ever
acquiring the page queues lock.
Reorder a few statements in vm_page_dontneed() so that a race can't lead
to an old reference persisting. This scenario is described in detail by a
comment.
Correct a spelling error in vm_page_dontneed().
Assert that the object is locked in vm_page_clear_dirty(), and restrict the
page queues lock assertion to just those cases in which the page is
currently writeable.
Add object locking to vnode_pager_generic_putpages(). This was the one
and only place where vm_page_clear_dirty() was being called without the
object being locked.
Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call
to vm_page_clear_dirty().
Change vnode_pager_generic_putpages() to the modern-style of function
definition. Also, change the name of one of the parameters to follow
virtual memory system naming conventions.
Reviewed by: kib
independent code. Move this code into mincore(), and eliminate the
page queues lock from pmap_mincore().
Push down the page queues lock into pmap_clear_modify(),
pmap_clear_reference(), and pmap_is_modified(). Assert that these
functions are never passed an unmanaged page.
Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m:
Contrary to what the comment says, pmap_mincore() is not simply an
optimization. Without a complete pmap_mincore() implementation,
mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED
because only the pmap can provide this information.
Eliminate the page queues lock from vfs_setdirty_locked_object(),
vm_pageout_clean(), vm_object_page_collect_flush(), and
vm_object_page_clean(). Generally speaking, these are all accesses
to the page's dirty field, which are synchronized by the containing
vm object's lock.
Reduce the scope of the page queues lock in vm_object_madvise() and
vm_page_dontneed().
Reviewed by: kib (an earlier version)
Extend struct sysvec with three new elements:
sv_fetch_syscall_args - the method to fetch syscall arguments from
usermode into struct syscall_args. The structure is machine-depended
(this might be reconsidered after all architectures are converted).
sv_set_syscall_retval - the method to set a return value for usermode
from the syscall. It is a generalization of
cpu_set_syscall_retval(9) to allow ABIs to override the way to set a
return value.
sv_syscallnames - the table of syscall names.
Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding
the call to cpu_set_syscall_retval().
The new functions syscallenter(9) and syscallret(9) are provided that
use sv_*syscall* pointers and contain the common repeated code from
the syscall() implementations for the architecture-specific syscall
trap handlers.
Syscallenter() fetches arguments, calls syscall implementation from
ABI sysent table, and set up return frame. The end of syscall
bookkeeping is done by syscallret().
Take advantage of single place for MI syscall handling code and
implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and
PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the
thread is stopped at syscall entry or return point respectively. The
EXEC flag augments SCX and notifies debugger that the process address
space was changed by one of exec(2)-family syscalls.
The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are
changed to use syscallenter()/syscallret(). MIPS and arm are not
converted and use the mostly unchanged syscall() implementation.
Reviewed by: jhb, marcel, marius, nwhitehorn, stas
Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc),
stas (mips)
MFC after: 1 month
memory on a platform. Tested on the Sibyte with 256MB and 1GB memory
configurations.
- Replace vtophys() with MIPS_KSEG0_TO_PHYS() to convert a page table
page's virtual address to physical. We can safely do this because
page table pages are allocated out of KSEG0.
- Add an assertion to verify that when a page table page is freed it
contains all zeroes. We can now use it after allocation without
zeroing it.
DDB so that all the fields line up.
- Print out the tid of the per-CPU idlethread instead of the pid since
the idle process is now shared across all idle threads.
MFC after: 1 month
- Adds re-partitioning TLB per core for enabled threads.
- Adds hardware thread id to cpuid mapping
- updates rge driver packet distribution and message ring handling
threads to be started based on hardware thread id.
- remove unused early debugging code to set control registers.
- coding style fixes
Approved by: rrs (mentor)
here, make the style of assertion used by pmap_enter() consistent
across all architectures.
On entry to pmap_remove_write(), assert that the page is neither
unmanaged nor fictitious, since we cannot remove write access to
either kind of page.
With the push down of the page queues lock, pmap_remove_write() cannot
condition its behavior on the state of the PG_WRITEABLE flag if the
page is busy. Assert that the object containing the page is locked.
This allows us to know that the page will neither become busy nor will
PG_WRITEABLE be set on it while pmap_remove_write() is running.
Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly
do copy-on-write-based zero-copy transmit on unmanaged or fictitious
pages, so don't even try. Previously, the call to pmap_remove_write()
would have failed silently.
am now able to run 32 cores ok.. but I still will hang
on buildworld with a NFS problem. I suspect I am missing
a patch for the netlogic rge driver.
JC check and see if I am missing anything except your
core-mask changes
Obtained from: JC
vm_page_try_to_free(). Consequently, push down the page queues lock into
pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and
pmap_remove_write().
Push down the page queues lock into Xen's pmap_page_is_mapped(). (I
overlooked the Xen pmap in r207702.)
Switch to a per-processor counter for the total number of pages cached.
queue length. The default value for this parameter is 50, which is
quite low for many of today's uses and the only way to modify this
parameter right now is to edit if_var.h file. Also add read-only
sysctl with the same name, so that it's possible to retrieve the
current value.
MFC after: 1 month
architecture from page queue lock to a hashed array of page locks
(based on a patch by Jeff Roberson), I've implemented page lock
support in the MI code and have only moved vm_page's hold_count
out from under page queue mutex to page lock. This changes
pmap_extract_and_hold on all pmaps.
Supported by: Bitgravity Inc.
Discussed with: alc, jeffr, and kib
pmap_ts_referenced() is not always appropriate for checking whether or
not pages have been referenced because it clears any reference bits
that it encounters. For example, in mincore(), clearing the reference
bits has two negative consequences. First, it throws off the activity
count calculations performed by the page daemon. Specifically, a page
on which mincore() has called pmap_ts_referenced() looks less active
to the page daemon than it should. Consequently, the page could be
deactivated prematurely by the page daemon. Arguably, this problem
could be fixed by having mincore() duplicate the activity count
calculation on the page. However, there is a second problem for which
that is not a solution. In order to clear a reference on a 4KB page,
it may be necessary to demote a 2/4MB page mapping. Thus, a mincore()
by one process can have the side effect of demoting a superpage
mapping within another process!
physical addresses.
o) Set a local maxmem in sb_machdep.c to avoid trying to use pages over 2^64
under 32-bit ABIs. Our pmap needs corrected to use vm_paddr_t consistently,
then we can make vm_paddr_t 64-bit under 32-bit ABIs and add code in pmap
to limit phys_avail by the maximum PFN that a 32-bit PTE can hold.
o) Use <machine/asm.h> macros for register-width, etc., rather than doing it
by hand in a few more assembly files.
o) Reduce diffs between various bits of TLB refill code in exception.S and
between interrupt processing code.
o) Use PTR_* to operate on registers that are pointers (e.g. sp).
o) Add and use a macro, CLEAR_PTE_SWBITS rather than using the
mysteriously-named WIRED_SHIFT to select bits to truncate when loading PTEs.
o) Don't doubly disable interrupts by moving zero to the status register,
especially since that has the nasty side-effect of taking us out of 64-bit
mode.
o) Use CLEAR_STATUS to disable interrupts the first time.
o) Keep SR_PX set as well as SR_[KSU]X when doing exception processing. This
is the bit that determines whether 64-bit operations are allowed.
o) Don't enable interrupts until configure_final(), like most other ports.
attributes for XKPHYS.
o) Make coprocessor 0 accessor function macros for register+selector registers
take the full name so that e.g. (as done in this commit), prid selector 1
can be written through mips_wr_ebase() rather than mips_wr_prid1().
o) Allow for sign extension of 32-bit segment addresses.
o) Remove an unused MIPS-I register number.
address space for an address as aligned by the new pmap_align_tlb()
function, which is for constraints imposed by the TLB. [1]
o) Add a kmem_alloc_nofault_space() function, which acts like
kmem_alloc_nofault() but allows the caller to specify which find-space
option to use. [1]
o) Use kmem_alloc_nofault_space() with VMFS_TLB_ALIGNED_SPACE to allocate the
kernel stack address on MIPS. [1]
o) Make pmap_align_tlb() on MIPS align addresses so that they do not start on
an odd boundary within the TLB, so that they are suitable for insertion as
wired entries and do not have to share a TLB entry with another mapping,
assuming they are appropriately-sized.
o) Eliminate md_realstack now that the kstack will be appropriately-aligned on
MIPS.
o) Increase the number of guard pages to 2 so that we retain the proper
alignment of the kstack address.
Reviewed by: [1] alc
X-MFC-after: Making sure alc has not come up with a better interface.
o) Mask off PAGE_MASK bits in pmap_update_page, etc., rather than modifying the
badvaddr in trapframe. Some nearby interfaces already did this.
o) Make PTEs "unsigned int" for now, not "unsigned long" -- we are only ready
for them to be 32-bit on 64-bit platforms.
o) Rather than using pmap_segmap and calculating the offset into the page table
by hand in trap.c, use pmap_pte().
o) Remove unused quad_syscall variable in trap.c.
o) Log things for illegal instructions like we do for bad page faults.
o) Various cast cleanups related to how to print registers.
o) When logging page faults, show the page table information not just for the
program counter, but for the fault address.
o) Modify support.S to use ABI-neutral macros for operating on pointers.
o) Consistently use CALLFRAME_SIZ rather than STAND_FRAME_SIZE, etc.
o) Remove unused insque/remque functions.
o) Remove some coprocessor 0 accessor functions implemented in assembly that
are unused and have inline assembly counterparts.
o) Remove NBPG, PGOFSET and PGSHIFT. Use the standard names.
o) Remove some unused macros and move things from param.h to vmparam.h that
belong in the latter. (Actually, all of the kernel segment values, virtual
addresses, etc., belong in one place, but this is a step in the right
direction.)
same time.
o) Remove some unused trivial uart functions from octeon_machdep now that the
uart part is fully working and they are unused.
o) Use __func__ instead of __FUNCTION__.
o) Use intr_*() instead of other routines that do the same thing.
o) Remove some duplicate printfs from the Octeon port, as well as duplicate
setting of Maxmem.
o) Use the right frequency divider on Octeon.
o) Use PCPU_GET(cpuid) consistently to get the cpuid of the running core.
o) Remove some unused macros in the Octeon port.
o) Use mips_sync() around use of the global dpcpu, whose value may not be
visible to APs at first.
o) When loading the first thread's stack, use macros to make the code correct
for n64 as well.
o) Remove stub, do-nothing FAU init/enable/disable functions from the RGMX
driver.
that turned out to be unrelated, and the rest was, as pointed out by Neel,
just wrong-headed.
o) Tweak mem.c to fix use of /dev/kmem for direct-mapped addresses.
ones implemented using assembly.
o) Use TRAPF_USERMODE() consistently rather than USERMODE(). Eliminate
<machine/psl.h> as a result.
o) Use intr_*() rather than *intr(), consistently.
o) Use register_t instead of u_int in some trap code.
o) Merge some more endian-related macros to machine/asm.h from NetBSD.
o) Add PTR_LI macro, which loads an address with the correct sign-extension for
a pointer.
o) Restore interrupts when bailing out due to an excessive IRQ in
nexus_setup_intr().
o) Remove unused functions from psraccess.S.
o) Enter temporary virtual entries for large memory access into the page tables
rather than simply hoping they stay resident in the TLB and we don't need to
do a refill for them.
o) Abstract out large memory mapping setup/teardown using some macros.
o) Do mips_dcache_wbinv_range() when using temporary virtual addresses just
like we do when we can use the direct map.
sparc64.
o) Use uiomove_fromphys rather than the broken fpage mechanism for /dev/mem.
o) Update sf_buf allocator to not share buffers and to do a pmap_qremove when
done with an sf_buf so as to better track valid mappings.
bit.
o) Remove some unused inlines.
o) Generate CP0 access functions for 64-bit TLB registers when building for
n64.
o) Add an inline function version of the COP0_SYNC macro.
subsequently in pmap_pinit() with the following signature:
panic: lock "pmap" 0xc7878bc8 already initialized
This bug was uncovered by the changes made to vm_map.c in r206140.
This causes a panic in vm_thread_dispose() when it tries to add this kstack
to the kstack cache. This happens only when 'td_kstack' is not (PAGE_SIZE * 2)
bytes aligned and we have unmapped the page at that address in cpu_thread_alloc.
Pointed out by: nwhitehorn@
due to rounding the buffer's physical address to the beginning of its
page. This fixes a panic in arge(4) when using PPPoE.
Reported by: Jakob van Santen <vansanten at wisc dot edu>
Reviewed by: gonzo
Obtained from: amd64
to the image_params struct instead of several members of that struct
individually. This makes it easier to expand its arguments in the future
without touching all platforms.
Reviewed by: jhb
frequency. This counter can be accessed coherently from both cores.
Use this as the preferred timecounter for the SWARM kernels.
The CP0 COUNT register is unusable as the timecounter on SMP platforms because
the COUNT registers on different CPUs are not guaranteed to be in sync.
when sb_load64() returns.
Some 32-bit arithmetic operations (e.g. subu) have unpredicatable results
when operating on 64-bit registers that are not properly sign-extended.
mapped kseg0 region.
The basic idea is to use KVA from the kseg2 region for mapping page
table pages that lie beyond the direct mapped region.
The TLB miss handler can now recursively fault into the TLB invalid
handler if it dereferences a kseg2 page table page address that is not
in the TLB.
Tested by: JC (c.jayachandran@gmail.com)
pointer, rather than octeon_fpa_alloc.
o) Report half duplex status properly.
o) Do not unconditionally update the last known link status in the softc. If
report_link isn't set, when octeon_rgmx_config_speed is called the first
time it will tell the driver (essentially) that we have already marked the
interface up. Likewise, don't change media speed and duplex if only the
link status is at issue. [1]
o) Remove manual changing of link state and let octeon_rgmx_config_speed do the
heavy lifting. [1]
Reviewed by: [1] imp
Sponsored by: Packet Forensics
o) Properly configure the CAM to handle IFF_PROMISC and note where IFF_ALLMULTI
handling would go if we didn't already force the NIC to receive all
multicast traffic.
Reviewed by: imp
Sponsored by: Packet Forensics
o) Inline octeon_rgmx_mark_ready into octeon_rgmx_init.
o) Add a media status handler that reports link and media status.
o) Set link state when if_init is called.
o) Remove some printfs related to driver state changes.
o) Remove some gratuitous comments.
Reviewed by: imp
Sponsored by: Packet Forensics
than spinning forever. This fixes booting with CF ejected.
NB: I've made the driver pretty chatty about errors in case there's hardware
that operates differently to mine, so we can easily track down any issues.
Reviewed by: imp
Sponsored by: Packet Forensics
redundant implementations.
o) Use ABI, not ISA, to determine address length.
o) Disable and restore interrupts around any operation that uses all 64 bits of
a register. In kernels using the O32 ABI, the upper 32 bits of those
registers is likely to be corrupted by an interrupt.
Sponsored by: Packet Forensics
- We don't need to fall back to uncacheable memory to satisfy BUS_DMA_COHERENT
requests on these CPUs.
- The bus_dmamap_sync() is a no-op for these CPUs.
A side-effect of this change is rename DMAMAP_COHERENT flag to
DMAMAP_UNCACHEABLE. This conveys the purpose of the flag more accurately.
Reviewed by: gonzo, imp
- remove unused and commented code (MIPS_BUS_SPACE_PCI, pic_usb_ack)
- use rmi_pci_bus_space for USB too (needs byteswap)
- uncomment xls_ehci.c in files.xlr
- changes to xls_ehci.c - updated with dev/usb/controller/ehci_*.c as
Obtained from: JC - c.jayachandran@gmail.com
- add bus_space_rmi_pci.c for PCI bus space
- files.xlr update for changes in files
- pcibus.c merged into xlr_pci.c (they were small files with inter-dependencies)
- xlr_pci.c - lot of changes here with few fixes, formatting cleanup
Obtained from: C. Jayachandran (JC) - c.jayachandran@gmail.com
- remove pci related code from bus_space_rmi.c, we will have another
file for PCI bus space functions which will do byte-swapping.
- remove local SWAP implementation
- added TODO stub for unimplemented functions
Obtained from: C. Jayachandran - c.jayachandran@gmail.com
- (cleanup) remove rmi specific 'struct mips_intrhand' - this is no
longer needed since 'struct intr_event' have all the required hooks
- add xlr_cpu_establish_hardintr, which has args for pre/post ithread
and filter hooks, so that the PCI code can add the PCI controller
interrupt ack code here
- make 'cpu_establish_hardintr' use the above function.
- (fix) change type of eirr/eimr from register_t to uint64_t. These
have to be 64bit otherwise we cannot handle interrupts from 32.
- (fix) use eimr to mask eirr before checking interrupts, so that we
will not handle masked interrupts.
Obtained from: C. Jayachandran - c.jayachandran@gmail.com
The backtrace code tries to look for an instruction of the form "sw ra, x(sp)"
to figure out the program counter of the calling function. When we generate
the kernel exception frame we store the 'ra' at the time of the exception
using an instruction of the same form. The problem is that the 'ra' at the
time of the exception is not the same as the 'program counter' at the time
of the exception.
The fix is to save the 'exception program counter' register by staging
it through the 'ra' register.
The RX overflow is reported in bit 2 on real hardware and Linux driver
for the same device already has this defined correctly.
This fixes frequent interrupt storms seen on RouterStation Pro boards.
Discussed with: gonzo
Getting the little-endian PCI bus working on the big-endian CPU proved to be
quite challenging. We let the PCI devices be mapped in the "match byte lanes"
address window. This is where they are mapped by the CFE and DMA transfers
generated to or from addresses within this window are not subject to automatic
byte-swapping.
However any access by the driver to memory-mapped pci space is redirected
via the "match bit lanes" address window. We get the benefit of automatic
byte swapping through this address window and drivers don't need to change
to deal with CPU big-endianness.
this in the Sibyte PCI hostbridge driver instead.
The nexus driver sees resource allocation requests for memory and irq
resources only. These are legitimate resources on all MIPS platforms.
Suggested by: imp
the 'debugging' section of any HEAD kernel and enable for the mainstream
ones, excluding the embedded architectures.
It may, of course, enabled on a case-by-case basis.
Sponsored by: Sandvine Incorporated
Requested by: emaste
Discussed with: kib
have code that detects this and makes two
transmit descriptors. However its possible
that the algorithm detects when the second
page is not used (when the data aligns perfectly
to the bottom of the page). This caused a 0
len descriptor to be added which locks up the
rge device. Skip such things with a continue.
JC provided this patch... Thanks JC :-)
Obtained from: JC (c.jayachandran@gmail.com)
Specifically on an SMP kernel it was observed that if both the
processors are doing an exit1() via ast()->postsig()->sigexit()
then we will deadlock.
This happens because exit1() calls vmspace_exit() that in turn
calls pmap_invalidate_all(). This function tries to do a
smp_rendezvous() which blocks because the other processor is not
responding to IPIs - because it too is doing AST processing with
interrupts disabled.
The platform that supports SMP currently is a SWARM with a dual-core Sibyte
processor. The kernel config file to use is SWARM_SMP.
Reviewed by: imp, rrs
disabling interrupts.
Simplify register usage - we can directly load 'curpcb' into 'k1' after
interrupts are disabled. There is no need to do so indirectly through 'a1'.
