independent code. Move this code into mincore(), and eliminate the
page queues lock from pmap_mincore().
Push down the page queues lock into pmap_clear_modify(),
pmap_clear_reference(), and pmap_is_modified(). Assert that these
functions are never passed an unmanaged page.
Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m:
Contrary to what the comment says, pmap_mincore() is not simply an
optimization. Without a complete pmap_mincore() implementation,
mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED
because only the pmap can provide this information.
Eliminate the page queues lock from vfs_setdirty_locked_object(),
vm_pageout_clean(), vm_object_page_collect_flush(), and
vm_object_page_clean(). Generally speaking, these are all accesses
to the page's dirty field, which are synchronized by the containing
vm object's lock.
Reduce the scope of the page queues lock in vm_object_madvise() and
vm_page_dontneed().
Reviewed by: kib (an earlier version)
Extend struct sysvec with three new elements:
sv_fetch_syscall_args - the method to fetch syscall arguments from
usermode into struct syscall_args. The structure is machine-depended
(this might be reconsidered after all architectures are converted).
sv_set_syscall_retval - the method to set a return value for usermode
from the syscall. It is a generalization of
cpu_set_syscall_retval(9) to allow ABIs to override the way to set a
return value.
sv_syscallnames - the table of syscall names.
Use sv_set_syscall_retval in kern_sigsuspend() instead of hardcoding
the call to cpu_set_syscall_retval().
The new functions syscallenter(9) and syscallret(9) are provided that
use sv_*syscall* pointers and contain the common repeated code from
the syscall() implementations for the architecture-specific syscall
trap handlers.
Syscallenter() fetches arguments, calls syscall implementation from
ABI sysent table, and set up return frame. The end of syscall
bookkeeping is done by syscallret().
Take advantage of single place for MI syscall handling code and
implement ptrace_lwpinfo pl_flags PL_FLAG_SCE, PL_FLAG_SCX and
PL_FLAG_EXEC. The SCE and SCX flags notify the debugger that the
thread is stopped at syscall entry or return point respectively. The
EXEC flag augments SCX and notifies debugger that the process address
space was changed by one of exec(2)-family syscalls.
The i386, amd64, sparc64, sun4v, powerpc and ia64 syscall()s are
changed to use syscallenter()/syscallret(). MIPS and arm are not
converted and use the mostly unchanged syscall() implementation.
Reviewed by: jhb, marcel, marius, nwhitehorn, stas
Tested by: marcel (ia64), marius (sparc64), nwhitehorn (powerpc),
stas (mips)
MFC after: 1 month
memory on a platform. Tested on the Sibyte with 256MB and 1GB memory
configurations.
- Replace vtophys() with MIPS_KSEG0_TO_PHYS() to convert a page table
page's virtual address to physical. We can safely do this because
page table pages are allocated out of KSEG0.
- Add an assertion to verify that when a page table page is freed it
contains all zeroes. We can now use it after allocation without
zeroing it.
DDB so that all the fields line up.
- Print out the tid of the per-CPU idlethread instead of the pid since
the idle process is now shared across all idle threads.
MFC after: 1 month
- Adds re-partitioning TLB per core for enabled threads.
- Adds hardware thread id to cpuid mapping
- updates rge driver packet distribution and message ring handling
threads to be started based on hardware thread id.
- remove unused early debugging code to set control registers.
- coding style fixes
Approved by: rrs (mentor)
here, make the style of assertion used by pmap_enter() consistent
across all architectures.
On entry to pmap_remove_write(), assert that the page is neither
unmanaged nor fictitious, since we cannot remove write access to
either kind of page.
With the push down of the page queues lock, pmap_remove_write() cannot
condition its behavior on the state of the PG_WRITEABLE flag if the
page is busy. Assert that the object containing the page is locked.
This allows us to know that the page will neither become busy nor will
PG_WRITEABLE be set on it while pmap_remove_write() is running.
Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly
do copy-on-write-based zero-copy transmit on unmanaged or fictitious
pages, so don't even try. Previously, the call to pmap_remove_write()
would have failed silently.
am now able to run 32 cores ok.. but I still will hang
on buildworld with a NFS problem. I suspect I am missing
a patch for the netlogic rge driver.
JC check and see if I am missing anything except your
core-mask changes
Obtained from: JC
vm_page_try_to_free(). Consequently, push down the page queues lock into
pmap_enter_quick(), pmap_page_wired_mapped(), pmap_remove_all(), and
pmap_remove_write().
Push down the page queues lock into Xen's pmap_page_is_mapped(). (I
overlooked the Xen pmap in r207702.)
Switch to a per-processor counter for the total number of pages cached.
queue length. The default value for this parameter is 50, which is
quite low for many of today's uses and the only way to modify this
parameter right now is to edit if_var.h file. Also add read-only
sysctl with the same name, so that it's possible to retrieve the
current value.
MFC after: 1 month
architecture from page queue lock to a hashed array of page locks
(based on a patch by Jeff Roberson), I've implemented page lock
support in the MI code and have only moved vm_page's hold_count
out from under page queue mutex to page lock. This changes
pmap_extract_and_hold on all pmaps.
Supported by: Bitgravity Inc.
Discussed with: alc, jeffr, and kib
pmap_ts_referenced() is not always appropriate for checking whether or
not pages have been referenced because it clears any reference bits
that it encounters. For example, in mincore(), clearing the reference
bits has two negative consequences. First, it throws off the activity
count calculations performed by the page daemon. Specifically, a page
on which mincore() has called pmap_ts_referenced() looks less active
to the page daemon than it should. Consequently, the page could be
deactivated prematurely by the page daemon. Arguably, this problem
could be fixed by having mincore() duplicate the activity count
calculation on the page. However, there is a second problem for which
that is not a solution. In order to clear a reference on a 4KB page,
it may be necessary to demote a 2/4MB page mapping. Thus, a mincore()
by one process can have the side effect of demoting a superpage
mapping within another process!
physical addresses.
o) Set a local maxmem in sb_machdep.c to avoid trying to use pages over 2^64
under 32-bit ABIs. Our pmap needs corrected to use vm_paddr_t consistently,
then we can make vm_paddr_t 64-bit under 32-bit ABIs and add code in pmap
to limit phys_avail by the maximum PFN that a 32-bit PTE can hold.
o) Use <machine/asm.h> macros for register-width, etc., rather than doing it
by hand in a few more assembly files.
o) Reduce diffs between various bits of TLB refill code in exception.S and
between interrupt processing code.
o) Use PTR_* to operate on registers that are pointers (e.g. sp).
o) Add and use a macro, CLEAR_PTE_SWBITS rather than using the
mysteriously-named WIRED_SHIFT to select bits to truncate when loading PTEs.
o) Don't doubly disable interrupts by moving zero to the status register,
especially since that has the nasty side-effect of taking us out of 64-bit
mode.
o) Use CLEAR_STATUS to disable interrupts the first time.
o) Keep SR_PX set as well as SR_[KSU]X when doing exception processing. This
is the bit that determines whether 64-bit operations are allowed.
o) Don't enable interrupts until configure_final(), like most other ports.
attributes for XKPHYS.
o) Make coprocessor 0 accessor function macros for register+selector registers
take the full name so that e.g. (as done in this commit), prid selector 1
can be written through mips_wr_ebase() rather than mips_wr_prid1().
o) Allow for sign extension of 32-bit segment addresses.
o) Remove an unused MIPS-I register number.
address space for an address as aligned by the new pmap_align_tlb()
function, which is for constraints imposed by the TLB. [1]
o) Add a kmem_alloc_nofault_space() function, which acts like
kmem_alloc_nofault() but allows the caller to specify which find-space
option to use. [1]
o) Use kmem_alloc_nofault_space() with VMFS_TLB_ALIGNED_SPACE to allocate the
kernel stack address on MIPS. [1]
o) Make pmap_align_tlb() on MIPS align addresses so that they do not start on
an odd boundary within the TLB, so that they are suitable for insertion as
wired entries and do not have to share a TLB entry with another mapping,
assuming they are appropriately-sized.
o) Eliminate md_realstack now that the kstack will be appropriately-aligned on
MIPS.
o) Increase the number of guard pages to 2 so that we retain the proper
alignment of the kstack address.
Reviewed by: [1] alc
X-MFC-after: Making sure alc has not come up with a better interface.
o) Mask off PAGE_MASK bits in pmap_update_page, etc., rather than modifying the
badvaddr in trapframe. Some nearby interfaces already did this.
o) Make PTEs "unsigned int" for now, not "unsigned long" -- we are only ready
for them to be 32-bit on 64-bit platforms.
o) Rather than using pmap_segmap and calculating the offset into the page table
by hand in trap.c, use pmap_pte().
o) Remove unused quad_syscall variable in trap.c.
o) Log things for illegal instructions like we do for bad page faults.
o) Various cast cleanups related to how to print registers.
o) When logging page faults, show the page table information not just for the
program counter, but for the fault address.
o) Modify support.S to use ABI-neutral macros for operating on pointers.
o) Consistently use CALLFRAME_SIZ rather than STAND_FRAME_SIZE, etc.
o) Remove unused insque/remque functions.
o) Remove some coprocessor 0 accessor functions implemented in assembly that
are unused and have inline assembly counterparts.
o) Remove NBPG, PGOFSET and PGSHIFT. Use the standard names.
o) Remove some unused macros and move things from param.h to vmparam.h that
belong in the latter. (Actually, all of the kernel segment values, virtual
addresses, etc., belong in one place, but this is a step in the right
direction.)
same time.
o) Remove some unused trivial uart functions from octeon_machdep now that the
uart part is fully working and they are unused.
o) Use __func__ instead of __FUNCTION__.
o) Use intr_*() instead of other routines that do the same thing.
o) Remove some duplicate printfs from the Octeon port, as well as duplicate
setting of Maxmem.
o) Use the right frequency divider on Octeon.
o) Use PCPU_GET(cpuid) consistently to get the cpuid of the running core.
o) Remove some unused macros in the Octeon port.
o) Use mips_sync() around use of the global dpcpu, whose value may not be
visible to APs at first.
o) When loading the first thread's stack, use macros to make the code correct
for n64 as well.
o) Remove stub, do-nothing FAU init/enable/disable functions from the RGMX
driver.
that turned out to be unrelated, and the rest was, as pointed out by Neel,
just wrong-headed.
o) Tweak mem.c to fix use of /dev/kmem for direct-mapped addresses.
ones implemented using assembly.
o) Use TRAPF_USERMODE() consistently rather than USERMODE(). Eliminate
<machine/psl.h> as a result.
o) Use intr_*() rather than *intr(), consistently.
o) Use register_t instead of u_int in some trap code.
o) Merge some more endian-related macros to machine/asm.h from NetBSD.
o) Add PTR_LI macro, which loads an address with the correct sign-extension for
a pointer.
o) Restore interrupts when bailing out due to an excessive IRQ in
nexus_setup_intr().
o) Remove unused functions from psraccess.S.
o) Enter temporary virtual entries for large memory access into the page tables
rather than simply hoping they stay resident in the TLB and we don't need to
do a refill for them.
o) Abstract out large memory mapping setup/teardown using some macros.
o) Do mips_dcache_wbinv_range() when using temporary virtual addresses just
like we do when we can use the direct map.
sparc64.
o) Use uiomove_fromphys rather than the broken fpage mechanism for /dev/mem.
o) Update sf_buf allocator to not share buffers and to do a pmap_qremove when
done with an sf_buf so as to better track valid mappings.
bit.
o) Remove some unused inlines.
o) Generate CP0 access functions for 64-bit TLB registers when building for
n64.
o) Add an inline function version of the COP0_SYNC macro.
subsequently in pmap_pinit() with the following signature:
panic: lock "pmap" 0xc7878bc8 already initialized
This bug was uncovered by the changes made to vm_map.c in r206140.
This causes a panic in vm_thread_dispose() when it tries to add this kstack
to the kstack cache. This happens only when 'td_kstack' is not (PAGE_SIZE * 2)
bytes aligned and we have unmapped the page at that address in cpu_thread_alloc.
Pointed out by: nwhitehorn@
due to rounding the buffer's physical address to the beginning of its
page. This fixes a panic in arge(4) when using PPPoE.
Reported by: Jakob van Santen <vansanten at wisc dot edu>
Reviewed by: gonzo
Obtained from: amd64
to the image_params struct instead of several members of that struct
individually. This makes it easier to expand its arguments in the future
without touching all platforms.
Reviewed by: jhb
frequency. This counter can be accessed coherently from both cores.
Use this as the preferred timecounter for the SWARM kernels.
The CP0 COUNT register is unusable as the timecounter on SMP platforms because
the COUNT registers on different CPUs are not guaranteed to be in sync.
when sb_load64() returns.
Some 32-bit arithmetic operations (e.g. subu) have unpredicatable results
when operating on 64-bit registers that are not properly sign-extended.
mapped kseg0 region.
The basic idea is to use KVA from the kseg2 region for mapping page
table pages that lie beyond the direct mapped region.
The TLB miss handler can now recursively fault into the TLB invalid
handler if it dereferences a kseg2 page table page address that is not
in the TLB.
Tested by: JC (c.jayachandran@gmail.com)
pointer, rather than octeon_fpa_alloc.
o) Report half duplex status properly.
o) Do not unconditionally update the last known link status in the softc. If
report_link isn't set, when octeon_rgmx_config_speed is called the first
time it will tell the driver (essentially) that we have already marked the
interface up. Likewise, don't change media speed and duplex if only the
link status is at issue. [1]
o) Remove manual changing of link state and let octeon_rgmx_config_speed do the
heavy lifting. [1]
Reviewed by: [1] imp
Sponsored by: Packet Forensics
o) Properly configure the CAM to handle IFF_PROMISC and note where IFF_ALLMULTI
handling would go if we didn't already force the NIC to receive all
multicast traffic.
Reviewed by: imp
Sponsored by: Packet Forensics
o) Inline octeon_rgmx_mark_ready into octeon_rgmx_init.
o) Add a media status handler that reports link and media status.
o) Set link state when if_init is called.
o) Remove some printfs related to driver state changes.
o) Remove some gratuitous comments.
Reviewed by: imp
Sponsored by: Packet Forensics
than spinning forever. This fixes booting with CF ejected.
NB: I've made the driver pretty chatty about errors in case there's hardware
that operates differently to mine, so we can easily track down any issues.
Reviewed by: imp
Sponsored by: Packet Forensics
redundant implementations.
o) Use ABI, not ISA, to determine address length.
o) Disable and restore interrupts around any operation that uses all 64 bits of
a register. In kernels using the O32 ABI, the upper 32 bits of those
registers is likely to be corrupted by an interrupt.
Sponsored by: Packet Forensics
- We don't need to fall back to uncacheable memory to satisfy BUS_DMA_COHERENT
requests on these CPUs.
- The bus_dmamap_sync() is a no-op for these CPUs.
A side-effect of this change is rename DMAMAP_COHERENT flag to
DMAMAP_UNCACHEABLE. This conveys the purpose of the flag more accurately.
Reviewed by: gonzo, imp
- remove unused and commented code (MIPS_BUS_SPACE_PCI, pic_usb_ack)
- use rmi_pci_bus_space for USB too (needs byteswap)
- uncomment xls_ehci.c in files.xlr
- changes to xls_ehci.c - updated with dev/usb/controller/ehci_*.c as
Obtained from: JC - c.jayachandran@gmail.com
- add bus_space_rmi_pci.c for PCI bus space
- files.xlr update for changes in files
- pcibus.c merged into xlr_pci.c (they were small files with inter-dependencies)
- xlr_pci.c - lot of changes here with few fixes, formatting cleanup
Obtained from: C. Jayachandran (JC) - c.jayachandran@gmail.com
- remove pci related code from bus_space_rmi.c, we will have another
file for PCI bus space functions which will do byte-swapping.
- remove local SWAP implementation
- added TODO stub for unimplemented functions
Obtained from: C. Jayachandran - c.jayachandran@gmail.com
- (cleanup) remove rmi specific 'struct mips_intrhand' - this is no
longer needed since 'struct intr_event' have all the required hooks
- add xlr_cpu_establish_hardintr, which has args for pre/post ithread
and filter hooks, so that the PCI code can add the PCI controller
interrupt ack code here
- make 'cpu_establish_hardintr' use the above function.
- (fix) change type of eirr/eimr from register_t to uint64_t. These
have to be 64bit otherwise we cannot handle interrupts from 32.
- (fix) use eimr to mask eirr before checking interrupts, so that we
will not handle masked interrupts.
Obtained from: C. Jayachandran - c.jayachandran@gmail.com
The backtrace code tries to look for an instruction of the form "sw ra, x(sp)"
to figure out the program counter of the calling function. When we generate
the kernel exception frame we store the 'ra' at the time of the exception
using an instruction of the same form. The problem is that the 'ra' at the
time of the exception is not the same as the 'program counter' at the time
of the exception.
The fix is to save the 'exception program counter' register by staging
it through the 'ra' register.
The RX overflow is reported in bit 2 on real hardware and Linux driver
for the same device already has this defined correctly.
This fixes frequent interrupt storms seen on RouterStation Pro boards.
Discussed with: gonzo
Getting the little-endian PCI bus working on the big-endian CPU proved to be
quite challenging. We let the PCI devices be mapped in the "match byte lanes"
address window. This is where they are mapped by the CFE and DMA transfers
generated to or from addresses within this window are not subject to automatic
byte-swapping.
However any access by the driver to memory-mapped pci space is redirected
via the "match bit lanes" address window. We get the benefit of automatic
byte swapping through this address window and drivers don't need to change
to deal with CPU big-endianness.
this in the Sibyte PCI hostbridge driver instead.
The nexus driver sees resource allocation requests for memory and irq
resources only. These are legitimate resources on all MIPS platforms.
Suggested by: imp
the 'debugging' section of any HEAD kernel and enable for the mainstream
ones, excluding the embedded architectures.
It may, of course, enabled on a case-by-case basis.
Sponsored by: Sandvine Incorporated
Requested by: emaste
Discussed with: kib
have code that detects this and makes two
transmit descriptors. However its possible
that the algorithm detects when the second
page is not used (when the data aligns perfectly
to the bottom of the page). This caused a 0
len descriptor to be added which locks up the
rge device. Skip such things with a continue.
JC provided this patch... Thanks JC :-)
Obtained from: JC (c.jayachandran@gmail.com)
Specifically on an SMP kernel it was observed that if both the
processors are doing an exit1() via ast()->postsig()->sigexit()
then we will deadlock.
This happens because exit1() calls vmspace_exit() that in turn
calls pmap_invalidate_all(). This function tries to do a
smp_rendezvous() which blocks because the other processor is not
responding to IPIs - because it too is doing AST processing with
interrupts disabled.
The platform that supports SMP currently is a SWARM with a dual-core Sibyte
processor. The kernel config file to use is SWARM_SMP.
Reviewed by: imp, rrs
disabling interrupts.
Simplify register usage - we can directly load 'curpcb' into 'k1' after
interrupts are disabled. There is no need to do so indirectly through 'a1'.
The only reason we need to have the sb_load64() and sb_store64()
functions in assembly is to cheat the compiler and generate the
'ld' and 'sd' instructions which it otherwise will not do when
compiling for a 32-bit architecture. There are some 64-bit
registers in the SCD unit that must be accessed using 64-bit
load and store instructions.
The basic idea is to use a the same virtual address as a window onto
distinct physical memory locations - one per processor. The physical
address that you access through this mapping depends on which cpu you
are currently executing on. We can now use the same virtual address
on any processor to access its per-cpu area.
The details are:
- The virtual address for 'struct pcpu *pcpup' is obtained by
stealing 2 pages worth of KVA in pmap_bootstrap().
- The mapping from the constant virtual address to a distinct
physical page is done in cpu_pcpu_init() through a wired TLB entry.
- A side-effect of this is that we reserve 2 pages worth of memory
for the pcpu but in reality it needs much less than that. The unused
memory is now used as the boot stack for the BSP and APs.
Remove SMP-specific bits from locore.S. The plan is to use a separate
mpboot.S for AP bootstrap.
Discussed on: freebsd-mips
Approved by: imp (mentor)
interrupts must be disabled through the
page_zero's or copys etc. Note that the
temporary mapping used by panic's may
cause us pain since int's may not be disabled.
When we get dumps working we may have to revist
this. Note that with this fix the build got
much much further.. until it hung on disk IO (I
would imagine thats the rge/msgring driver acting
up).
above 4Gig. If so when we add the base address with
the size we will wrap. So for now we just ignore
such memory and only use what we can. When we
get 64 bit working then we will be much better ;->
lacking a copyright/license statement. All these files were in the
Cavium FreeBSD source drop and appear to be written by Cavium (some
are nearly verbatim copies of files from the cnusers' 1.9.0 SDK, which
also uses this copyright).
routine in one place was setting the valid2 bit to
2 not 1. This meant the PTE was NOT valid and so
you would crash.
In Zero Page there was a incorrect setting of
the valid bit AFTER the actual zero (opps)..
Hopefully this will fix the 0xc0000000 crashes
that I have been seeing (unless of course there are
other problems with these old hacks of mine to get
to memory above 512Meg)
the RMI/XLR has the COP0 and COP2 bits enabled
Plus it needs SX too. Thanks again for JC in
catching this ;-)
Submitted by: JC (jayachandranc@netlogicmicro.com
to invalidate memory chunk that starts or ends in the middle of
cache line.
This was responsible for one half of the problem preventing umass
to work reliably on some MIPS32 platforms. USBng needs to stop
sharing cache lines between DMA-able memory and other structures
to cure the other half.
Discussed with: imp, gonzo
locks up in make buildworld.
You need to follow the mips wiki for building
the nfs partition and setup things to mount there
(in the conf and in your bootp setup).
This is a workaround for the fact that the CFE is compiled as a 64-bit
application and therefore sets the SR_KX bit every time we call into
it (for e.g. console).
A TLB miss for any address above 0xc0000000 with the SR_KX bit set will
end up at the XTLB exception vector. We workaround this by copying the
standard TLB handler at the XTLB exception vector.
Approved by: imp (mentor)
through fork_trampoline.
This was caused because we were clearing the SR_INT_IE and setting
SR_EXL bits of the status register at the same time. This meant
that if an interrupt happened while this MTC0 was making its way
through the pipeline the exception processing would see the
status register with SR_EXL bit set. This in turn would mean that
the COP_0_EXC_PC would not be updated so the return from exception
would be to an incorrect address.
It is easy to verify this fix by a program that forks in a loop
and the child just exits:
while (1) {
pid_t pid = vfork();
if (pid == 0)
_exit(0);
if (pid != -1)
waitpid(pid, NULL, 0);
}
Also remove two instances where we set SR_EXL bit gratuitously in exception.S.
Approved by: imp (mentor)
obio. Take advantage of the fact that obio only really supports uart
at the moment to use the uart bus tag always for IOPORT allocations.
# this needs to be redone to conform to FreeBSD standards and allow for
# additional drivers for SoC hardware to attach
in the pcb at the td_lock offset, rather than in the struct thread at
the td_lock offset. And we were storing a pointer to the old thread
rather than to the mutex. Why this didn't always kill us, I'll never
know.
Fix an obsolete comment and update the prototype in the comments.
Also note what variables are in what registers since this function is
a little complex...
neel@ found this problem and proposed this fix. This cures a number
of different problem reports out there, and gets us booting octeon to
the login prompt...
Submitted by: neel@
Reviewed by: rrs@, gonzo@
