completely full, we'd not complete any of the mbufs due to the fence
post error (this creates a large leak). When this is fixed, we still
leak, but at a much smaller rate due to a race between ateintr and
atestart_locked as well as an asymmetry where atestart_locked is
called from elsewhere. Ensure that we free in-flight packets that
have completed there as well. Also remove needless check for NULL on
mb, checked earlier in the loop and simplify a redundant if.
MFC after: 3 days
this change we may end up with a panic (Fatal kernel mode data abort:
'External Non-Linefetch Abort (S)') as described in
http://e2e.ti.com/support/arm/sitara_arm/f/791/t/276862.aspx.
It is now possible to bring up I2C1 and I2C2 on BBB.
Approved by: adrian (mentor)
vm_max_virtual_address to be KERNVIRTADDR + 256MB. This allows some
future shock protection since the KVA requirements have gone up since
the unmapped changes have gone in, as well as preventing us from
overlapping with the hardware devices, which we map at 0xd0000000,
which we'd hit with anything more than 85MB...
MFC after: 3 days
always returning '0' for all the reads, even for the outputs. It is now
known to work with gpioiic(4) and gpioled(4).
Approved by: adrian (mentor)
Tested on: BBB
shifts into the sign bit. Instead use (1U << 31) which gets the
expected result.
This fix is not ideal as it assumes a 32 bit int, but does fix the issue
for most cases.
A similar change was made in OpenBSD.
Discussed with: -arch, rdivacky
Reviewed by: cperciva
enables data cache and other chip-specific features. It was previously
done via an early SYSINIT, but it was being done after pmap and vm setup,
and those setups need to use mutexes. On some modern ARM platforms,
the ldrex/strex instructions that implement mutexes require the data cache
to be enabled.
A nice side effect of enabling caching earlier is that it eliminates the
multi-second pause that used to happen early in boot while physical memory
and pmap and vm were being set up. On boards with 1 GB or more of ram
this pause was very noticible, sometimes 5-6 seconds.
PR: arm/183740
When entering a mapping via pmap_enter() unmanaged pages ought to be
naturally excluded from the "modified" and "referenced" emulation.
RW permission should be granted implicitly when requested,
otherwise unmanaged page will not recover from the permission fault
since there will be no PV entry to indicate that the page can be written.
In addition, only managed pages that participate in "modified"
emulation need to be marked as "dirty" and "writeable" when entered
with RW permissions. Likewise with "referenced" flag for managed pages.
Unmanaged ones however should not be marked as such.
Reviewed by: cognet, gber
When emulating modified bit the executable attribute was cleared by
mistake when calling pmap_set_prot(). This was not a problem before
changes to ref/mod emulation since all the pages were created RW basing
on the "prot" argument in pmap_enter(). Now however not all pages are RW
and the RW permission can be cleared in the process.
Added proper KTRs accordingly.
Spotted by: cognet
Reviewed by: gber
host.host_ocr, examine the correct field when setting up the hardware. Also,
the offset for the capabilties register should be 0x140, not 0x240.
Submitted by: Ilya Bakulin <ilya@bakulin.de>
Pointy hat to: me
rely only on checking the device unit to indentify the BSC unit we are
attaching to. Make use of the device base address to identify our BSC unit.
Approved by: adrian (mentor)
words, every architecture is now auto-sizing the kmem arena. This revision
changes kmeminit() so that the definition of VM_KMEM_SIZE_SCALE becomes
mandatory and the definition of VM_KMEM_SIZE becomes optional.
Replace or eliminate all existing definitions of VM_KMEM_SIZE. With
auto-sizing enabled, VM_KMEM_SIZE effectively became an alternate spelling
for VM_KMEM_SIZE_MIN on most architectures. Use VM_KMEM_SIZE_MIN for
clarity.
Change kmeminit() so that the effect of defining VM_KMEM_SIZE is similar to
that of setting the tunable vm.kmem_size. Whereas the macros
VM_KMEM_SIZE_{MAX,MIN,SCALE} have had the same effect as the tunables
vm.kmem_size_{max,min,scale}, the effects of VM_KMEM_SIZE and vm.kmem_size
have been distinct. In particular, whereas VM_KMEM_SIZE was overridden by
VM_KMEM_SIZE_{MAX,MIN,SCALE} and vm.kmem_size_{max,min,scale}, vm.kmem_size
was not. Remedy this inconsistency. Now, VM_KMEM_SIZE can be used to set
the size of the kmem arena at compile-time without that value being
overridden by auto-sizing.
Update the nearby comments to reflect the kmem submap being replaced by the
kmem arena. Stop duplicating the auto-sizing formula in every machine-
dependent vmparam.h and place it in kmeminit() where auto-sizing takes
place.
Reviewed by: kib (an earlier version)
Sponsored by: EMC / Isilon Storage Division
fdtbus in most cases. This brings ARM and MIPS more in line with existing
Open Firmware platforms like sparc64 and powerpc, as well as preventing
double-enumeration of the OF tree on embedded PowerPC (first through nexus,
then through fdtbus).
This change is also designed to simplify resource management on FDT platforms
by letting there exist a platform-defined root bus resource_activate() call
instead of replying on fdtbus to do the right thing through fdt_bs_tag.
The OFW_BUS_MAP_INTR() and OFW_BUS_CONFIG_INTR() kobj methods are also
available to implement for similar purposes.
Discussed on: -arm, -mips
Tested by: zbb, brooks, imp, and others
MFC after: 6 weeks
allocates kva space from the top down for the device mappings and builds
entries in an internal table which is automatically used later by
arm_devmap_bootstrap(). The platform code just calls the new
arm_devmap_add_entry() function as many times as it needs to (up to 32
entries allowed; most platforms use 2 or 3 at most).
There is also a new arm_devmap_lastaddr() function that returns the lowest
kva address allocated; this can be used to implement initarm_lastaddr()
which is used to initialize vm_max_kernel_address.
The new code is based on a similar concept developed for the imx family
SoCs recently. They will soon be converted to use this new common code.
static device mappings, rather than as the first of the initializations
that a platform can hook into. This allows a platform to allocate KVA
from the top of the address space downwards for things like static device
mapping, and return the final "last usable address" result after that and
other early init work is done.
Because some platforms were doing work in initarm_lastaddr() that needs to
be done early, add a new initarm_early_init() routine and move the early
init code to that routine on those platforms.
Rename platform_devmap_init() to initarm_devmap_init() to match all the
other init routines called from initarm() that are designed to be
implemented by platform code.
Add a comment block that explains when these routines are called and the
type of work expected to be done in each of them.
new devmap.[ch] files. Emphasize the MD nature of these things by using
the prefix arm_devmap_ on the function and type names (already a few of
these things found their way into MI code, hopefully it will be harder to
do by accident in the future).
out common code related to mapping device memory into a new devmap.c file.
Remove the growing duplication of code that used pmap_devmap_find_pa() and
then did some math with the returned results to generate a virtual address,
and likewise in reverse to get a physical address. Now there are a pair
of functions, arm_devmap_vtop() and arm_devmap_ptov(), to do that. The
bus_space_map() implementations are rewritten in terms of these.
the essentially identical generic implementations instead. The generic
implementations differ only in the spelling of a couple variable names
and some formatting differences.
include it accordingly. The build machinery for universe and tinderbox
tries to build every kernel config whose name begins and ends with [A-Z0-9]
and the common include file that has most of the options isn't buildable
by itself, so the new lowercase .common will avoid building it.
accessed through the direct map unless the kernel configuration actually
includes a direct map. Only a few configurations do, and for the rest the
unnecessary free page pool is a small pessimization.
Tested by: zbb
MFC after: 6 weeks
Use the published compatible strings (our own invention, "ti,mmchs" is
still accepted as well, for now).
Don't blindly turn on 8-bit bus mode, because even though the conroller
supports it, the board has to be wired appropriately as well. Use the
published property (bus-width=<n>) and honor all the valid values (1,4,8).
The eMMC device on a Beaglebone Black is wired for 8-bit, update the dts.
The mmchs controller can inherently do both 1.8v and 3.0v on the first
device and 1.8v only on other devices, unless an external transceiver is
used. Set the voltage automatically for the first device and honor
the published fdt property (ti,dualvolt) for other devices.
Thanks go to Ilya Bakulin for figuring out the voltage compatibility stuff.
threaded handlers.
It's not easy to see from the diffs of this change exactly how it
accomplishes the above. The arm_mask_irq() and arm_unmask_irq() functions
are, respectively, the pre_thread and post_thread hooks. Not seen in
these diffs, the arm_post_filter() routine also EOIs. The post_filter
routine runs after filter handlers if there will be no threaded handlers,
so it just EOIs. The pre_thread routine masks the interrupt (at the
controller, not the source) and EOIs. So one way or another, the EOI
happens at the point where filter handlers are done.
the interrupt handler. If the event callback starts a new short timeout,
the timer can fire before returning from the event callback, and clearing
the interrupt status after that loses the interrupt and hangs until the
counter wraps. Fixing all of this removes the need for the do-nothing
loop at the top of the handler which really just waited for the counter to
roll over and reach the one-shot count again.
Also add a missing return(0) in the periodic timer start case.
Use values of the correct defines to determine statement's result.
ARM_ARCH_ symbols are always defined, hence only values are relevant.
Reviewed by: cognet
Sheeva PJ4Bv6 - based chips were only prototypes for V7 class Armada
SoC family. Current in-tree support for PJ4Bv6 will not work and also
there should be no platforms in active use that would incorporate that
CPU revision.
Loading kernel to 0xf00000 has no practical reason.
Starting it from the u-boot's highest possible end address
(2MB counting from 0x0) makes more sense.
Tested by: kevlo
Depending on u-boot's flavor some boards have their SoC registers
base address configured to 0xD0000000 and other to 0xF1000000.
U-boot is passing currently set value via CP15 register.
In order to create proper mapping for SoC registers and allow further
successful initialization it is necessary to replace fdt_immr_pa with
the real value and eventually fix-up device tree blob.
Tested by: kevlo
Armada XP initialization flow requires SoC registers to be
mapped very early in order to configure Snoop Filter for SMP.
Additional mapping in locore.S is redundant as proper mapping is
made in pmap_devmap_bootstrap() prior to calling cpu_setup() which
configures the Snoop Filter.
For secondaru CPUs it is better to pass VA of the SoC
registers defined in MV_BASE and PA consistent with the value
in the Device Tree.
Tested by: kevlo
vm_pages. Provide trivial implementation which forwards the load to
_bus_dmamap_load_phys() page by page. Right now all architectures use
bus_dmamap_load_ma_triv().
Tested by: pho (as part of the functional patch)
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
The only remaining user was the code that allocates bounce pages for armv4
busdma. It's not clear why bounce pages would need uncached memory, but
if that ever changes, kmem_alloc_attr() would be the way to get it.
really need it. That would be almost everywhere it was included. Add
it in a couple files that really do need it and were previously getting
it by accident via another header.
included by vm/pmap.h, which is a prerequisite for arm/machine/pmap.h
so there's no reason to ever include it directly.
Thanks to alc@ for pointing this out.
of the address space downwards, and then returning the lowest mapped
device address from initarm_lastaddr(). This adds over 500MB of kva
space compared to the old way of hardcoding the end address as 0xE0000000.
Also, pre-map most of the SoC's common memory-mapped devices using 1MB
section mappings so that all device access uses just a few TLB entries.
Graphics devices aren't mapped this way yet, but probably should be.
To provide this new functionality without pasting identical code into
multiple imxNN_machdep.c files, rework the imx machdep code so that
things common to the whole family of SoCs are in a new imx_machdep.c file.
The rewritten imxNN_machdep.c files contain just things specific to an
individual SoC.
- The new allocator won't return coherent memory for any size > PAGE_SIZE,
so don't assume we have coherent memory, and explicitely use
bus_dmamap_sync().
Freescale SoCs including the i.MX series. This also works for the newer
SoCs with the ENET gigabit controller, but doesn't use any of the new
hardware features other than enabling gigabit speed.
the right register for power managment. It
was incorrectly using the clock register
which also caused the status to be the
opposite of what it is supposed to be.
1 - its disabled
0 - its enabled
Per kirkwood spec FSS_88F6180_9x_6281_OpenSource.pdf
correctly by doing nothing, then add a panic for the default case, because
that implies that some driver asked for a sync (probably incorrectly) and
nothing was done.
there as "kern.ipc.sendfile.readahead".
- Push all nsfbuf related tunables into MD code. Don't move them
to new namespace in favor of POLA.
Reviewed by: scottl
Approved by: re (gjb)
Previous change applied in r255613 fixed build for ARMv6 but
broke it for previous architecture revisions. This commit
eventually fixes GCC build for all ARM revisions.
Approved by: cognet (mentor)
Approved by: re (kib)
pmap_clear_reference() has had exactly one caller in the kernel for
several years, more precisely, since FreeBSD 8. Now, that call no
longer exists.
Approved by: re (kib)
Sponsored by: EMC / Isilon Storage Division
While here, correct all consumers to pass NULL instead of 0 as we pass
capability rights as pointers now, not uint64_t.
Reported by: Daniel Peyrolon
Tested by: Daniel Peyrolon
Approved by: re (marius)
Apply the given advice to the specified range of addresses within the
given pmap. Depending on the advice, clear the referenced and/or
modified flags in each mapping. Superpage within the given range will
be demoted or destroyed.
Reviewed by: alc
Approved by: cognet (mentor)
Approved by: re
When clearing the modification status of the superpage, one of the
base pages produced during demotion should be marked as write disabled.
The intention is that subsequent write access may repromote.
In the current implementation this was done wrong as write permission was
granted instead of forbidden.
Approved by: cognet (mentor)
Approved by: re
for ARM.
This is quite ugly, because it has to work around a clang bug that does not
allow built-in functions to be defined, even when they're ones that are
expected to be built as part of a library.
Reviewed by: ed
MADV_DONTNEED) and madvise(..., MADV_FREE). Specifically, introduce a new
pmap function, pmap_advise(), that operates on a range of virtual addresses
within the specified pmap, allowing for a more efficient implementation of
MADV_DONTNEED and MADV_FREE. Previously, the implementation of
MADV_DONTNEED and MADV_FREE relied on per-page pmap operations, such as
pmap_clear_reference(). Intuitively, the problem with this implementation
is that the pmap-level locks are acquired and released and the page table
traversed repeatedly, once for each resident page in the range
that was specified to madvise(2). A more subtle flaw with the previous
implementation is that pmap_clear_reference() would clear the reference bit
on all mappings to the specified page, not just the mapping in the range
specified to madvise(2).
Since our malloc(3) makes heavy use of madvise(2), this change can have a
measureable impact. For example, the system time for completing a parallel
"buildworld" on a 6-core amd64 machine was reduced by about 1.5% to 2.0%.
Note: This change only contains pmap_advise() implementations for a subset
of our supported architectures. I will commit implementations for the
remaining architectures after further testing. For now, a stub function is
sufficient because of the advisory nature of pmap_advise().
Discussed with: jeff, jhb, kib
Tested by: pho (i386), marcel (ia64)
Sponsored by: EMC / Isilon Storage Division
Promoting base pages to superpages can increase TLB coverage and allow for
efficient use of page table entries. This development provides FreeBSD/ARM
with superpages management mechanism roughly equivalent to what we have for
i386 and amd64 architectures.
1. Add mechanism for automatic promotion of 4KB page mappings to 1MB section
mappings (and demotion when not needed, respectively).
2. Managed and non-kernel mappings are now superpages-aware.
3. The functionality can be enabled by setting "vm.pmap.sp_enabled" tunable to
a non-zero value (either in loader.conf or by modifying "sp_enabled"
variable in pmap-v6.c file). By default, automatic promotion is currently
disabled.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: alc
Sponsored by: The FreeBSD Foundation, Semihalf
This allows for enabling and configuring superpages reservation mechanism in
order to allocate and populate 256 4KB base pages (for the purpose of
promotion to a 1MB superpage).
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: alc
Sponsored by: The FreeBSD Foundation, Semihalf
which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE
zone. Initialize the pmap lock in the vmspace zone init function, and
remove pmap lock initialization and destruction from pmap_pinit() and
pmap_release().
Suggested and reviewed by: alc (previous version)
Tested by: pho
Sponsored by: The FreeBSD Foundation
The TI uart hardware is ns16550-compatible, except that before it can
be used the clocks and power have to be enabled and a non-standard
mode control register has to be set to put the device in uart mode
(as opposed to irDa or other serial protocols). This adds the extra
code in an extension to the standard ns8250 probe routine, and the
rest of the driver is just the standard ns8250 code.
The MMCHS hardware is pretty much a standard SDHCI v2.0 controller with a
couple quirks, which are now supported by sdhci(4) as of r254507.
This should work for all TI SoCs that use the MMCHS hardware, but it has
only been tested on AM335x right now, so this enables it on those platforms
but leaves the existing ti_mmchs driver in place for other OMAP variants
until they can be tested.
This initial incarnation lacks DMA support (coming soon). Even without it
this improves performance pretty noticibly over the ti_mmchs driver,
primarily because it now does multiblock IO.
There is no need for calling vm_page_dirty() when clearing "modified" flag as
it is already set for that page in pmap_fault_fixup() or pmap_enter() thanks
to "modified" bit emulation.
Also, there is no need for checking PTE "referenced" or "writeable" flags. If
there is a request to clear a particular flag we should just do it.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: gber
Sponsored by: The FreeBSD Foundation, Semihalf
Last input argument in pmap_modify_pv() should be a mask of flags to be set.
In pmap_change_wiring() however, the straight wired status was used, which
does not represent valid flags (and is of type boolean).
This commit fixes the issue so that wired flag is passed to pmap_modify_pv()
properly.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: gber
Sponsored by: The FreeBSD Foundation, Semihalf
Revise L2_S_PROT_MASK to include all of the protection bits. Notice that
clearing these bits does not always take away the corresponding permissions
(for example, permission is granted when the bit is cleared). The bits are
cleared but are to be set or left cleared accordingly in pmap_set_prot(),
pmap_enter_locked(), etc.
Clear L2_XN along with L2_S_PROT_MASK in pmap_set_prot() so that all
permissions related bits are cleared before actual configuration.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: gber
Sponsored by: The FreeBSD Foundation, Semihalf
- PGA_WRITEABLE indicates that there *might be* a writable mapping for the
particular page, so to avoid frequent sweeping of the pv_entries whenever
pmap_nuke_pv(), pmap_modify_pv(), etc. is called, it is sufficient to
clear that flag if there are no managed mappings for that page anymore
(notice that only pmap_enter is authorized to set this flag).
- Avoid redundant checking for PVF_WIRED flag when this flag cannot be set
anyway.
- Clear PGA_WRITEABLE only once for each vm_page instead of multiple,
redundant clearing it in loop when there are no writeable mappings
to that page anymore.
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Reviewed by: gber
Sponsored by: The FreeBSD Foundation, Semihalf
- Use the right address when calling kva_free()
(Is there any reason why the s3c2xx0 comes with its own version of bs_map/
bs_unmap ? It seems to be just the same as in bus_space_generic.c)
Unify the 2 concept into a real, minimal, sxlock where the shared
acquisition represent the soft busy and the exclusive acquisition
represent the hard busy.
The old VPO_WANTED mechanism becames the hard-path for this new lock
and it becomes per-page rather than per-object.
The vm_object lock becames an interlock for this functionality:
it can be held in both read or write mode.
However, if the vm_object lock is held in read mode while acquiring
or releasing the busy state, the thread owner cannot make any
assumption on the busy state unless it is also busying it.
Also:
- Add a new flag to directly shared busy pages while vm_page_alloc
and vm_page_grab are being executed. This will be very helpful
once these functions happen under a read object lock.
- Move the swapping sleep into its own per-object flag
The KPI is heavilly changed this is why the version is bumped.
It is very likely that some VM ports users will need to change
their own code.
Sponsored by: EMC / Isilon storage division
Discussed with: alc
Reviewed by: jeff, kib
Tested by: gavin, bapt (older version)
Tested by: pho, scottl
line boundary. It has never been 100% correct, and it can't work on SMP,
because nothing prevents another core from accessing data from an unrelated
buffer in the same cache line while we invalidated it. Just use bounce pages
instead.
Reviewed by: ian
Approved by: mux (mentor) (implicit)
Add support for A20 timer.
Correct interrupt offset depending from chip.
Add basic code for CPU configuration module.
For now, add kernel config and dts file
(only FDT blob related problem needs to be solved later in
order to have one kernel for both cubieboard1 and 2).
Approved by: ray@
transparent layering and better fragmentation.
- Normalize functions that allocate memory to use kmem_*
- Those that allocate address space are named kva_*
- Those that operate on maps are named kmap_*
- Implement recursive allocation handling for kmem_arena in vmem.
Reviewed by: alc
Tested by: pho
Sponsored by: EMC / Isilon Storage Division
