Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.
Submitted by: Jan Schaumann <jschauma@stevens.edu>
Pull Request: https://github.com/freebsd/freebsd/pull/96
On arm64 use atomics. Then, both arm and arm64 do not need a critical
section around update. Replace all cpus loop by CPU_FOREACH().
This brings arm and arm64 counter(9) implementation closer to current
amd64, but being more RISC-y, arm* version cannot avoid atomics.
Reported by: Alexandre Martins <alexandre.martins@stormshield.eu>
Reviewed by: andrew
Tested by: Alexandre Martins, andrew
Sponsored by: The FreeBSD Foundation
MFC after: 2 weeks
The types are for the byte offset and page index in vm object. They
are similar to off_t, which is defined as 64bit MI integer. Using MI
definitions will allow to provide consistent MD values of vm
object-related maximum sizes.
Reviewed by: alc
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
- Use new option SMP_ON_UP instead of (mis)using specific CPU type.
By this, any SMP kernel can be compiled with SMP_ON_UP support.
- Enable runtime detection of CPU multiprocessor extensions only
if SMP_ON_UP option is used. In other cases (pure SMP or UP),
statically compile only required variant.
- Don't leak multiprocessor instructions to UP kernel.
- Correctly handle data cache write back to point of unification.
DCCMVAU is supported on all armv7 cpus.
- For SMP_ON_UP kernels, detect proper TTB flags on runtime.
Differential Revision: https://reviews.freebsd.org/D9133
- Replace pcpu_find(curcpu) with get_pcpu(), which is much
more direct.
- Remove armv4 pcpu fields which I added in r286296 but never
needed to use.
- armv6 pc_qmap_addr was leftover from the old armv6 pmap
implementation. Rename it and put it to use in the new one.
Noted by: skra
Reviewed by: skra
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D9312
mappings for armv6 pmap zero and copy operations to the MD PCPU region.
Change sysmap initialization to only allocate KVA pages for CPUs that
are actually present.
While here, collapse CMAP3 into CMAP2 (their use was mutually exclusive
anyway) and "recover" some space in PCPU padding that has always been
available due to 64-byte cacheline padding.
Reviewed by: skra
MFC after: 1 week
Differential Revision: https://reviews.freebsd.org/D9172
it into pmap-v4.h where they are used. Other than those few lines of
support for different MMU types, nothing in cpuconf.h has been used in our
code for quite a while.
The file existed to set up a variety of symbols to describe the
architecture. Over the past few years we have converted all of our source
to use the new architecture symbols standardized by ARM Inc, and predefined
by both clang and gcc.
PR: 216104
This function is referenced, but never called from DRM2 code. Also,
real behavior of pmap_mapdev_attr() in ARM world is unclear as we don't
have any additional attribute for a device memory type.
MFC after: 2 weeks
machine/armreg.h requires access to the __ARM_ARCH macro, which is not
always properly defined (especially by gcc 4.2.1). We should include
sys/cdefs.h in order to get the definitions in machine/acle-compat.h,
which would properly define the __ARM_ARCH macro in these cases.
So, in cases where machine/armreg.h is included without _SYS_CDEFS_H_
being defined - generate an #error.
Reviewed by: andrew
Sponsored by: Smartcom - Bulgaria AD
Differential Revision: https://reviews.freebsd.org/D8460
* Rename ARM_HAVE_MP_EXTENSIONS to ARM_USE_MP_EXTENSIONS and extend it to
handle more cases, including when SMP is not enabled.
* Check ARM_USE_MP_EXTENSIONS when building for ARMv7+, even if no SMP.
* Use ARM_USE_MP_EXTENSIONS in pmap-v6.c to detect when to set PRRR_NS1.
With this we should be able to boot on all ARMv7+ Cortex-A cores with
32-bit support.
Reviewed by: mmel, imp (earlier version)
Relnotes: yes
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D8335
hardware that supports the mp extensions. If so it should use the broadcast
tlb invalidate instructions as other CPUs or devices may need to know about
the invalidation.
To simplify the code have the compiler optimise out the else case when not
builing for Cortex-A8.
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D8092
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.
Submitted by: kib@, dim@, zbb@, emaste@
for later Cortex-A CPUs that support the Multiprocessor Extensions. This
will be needed to support both in a single GENERIC kernel while still
being able to only build for a single SoC.
Reviewed by: mmel
Relnotes: yes
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D8138
On amd64, arm64 and i386, we have the possibility to switch between TLS
areas in userspace. The nice thing about this is that it makes it easier
to do light-weight threading, if we ever feel like doing that. On armv6,
let's go into the same direction by making it possible to safely use the
TPIDRURW register, which is intended for this purpose.
Clean up the ARMv6 code to remove md_tp entirely. Simply add a dedicated
field to the PCB to hold the value of TPIDRURW across context switches,
like we do for any other register. As userspace currently uses the
read-only TPIDRURO register, simply ensure that we keep both values in
sync where possible. The system calls for modifying the read-only
register will simply write the intended value into both registers, so
that it lazily ends up in the PCB during the next context switch.
Reviewed by: https://reviews.freebsd.org/D7951
Approved by: andrew
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D7951
Right now, userspace (fast) gettimeofday(2) on x86 only works for
RDTSC. For older machines, like Core2, where RDTSC is not C2/C3
invariant, and which fall to HPET hardware, this means that the call
has both the penalty of the syscall and of the uncached hw behind the
QPI or PCIe connection to the sought bridge. Nothing can me done
against the access latency, but the syscall overhead can be removed.
System already provides mappable /dev/hpetX devices, which gives
straight access to the HPET registers page.
Add yet another algorithm to the x86 'vdso' timehands. Libc is updated
to handle both RDTSC and HPET. For HPET, the index of the hpet device
to mmap is passed from kernel to userspace, index might be changed and
libc invalidates its mapping as needed.
Remove cpu_fill_vdso_timehands() KPI, instead require that
timecounters which can be used from userspace, to provide
tc_fill_vdso_timehands{,32}() methods. Merge i386 and amd64
libc/<arch>/sys/__vdso_gettc.c into one source file in the new
libc/x86/sys location. __vdso_gettc() internal interface is changed
to move timecounter algorithm detection into the MD code.
Measurements show that RDTSC even with the syscall overhead is faster
than userspace HPET access. But still, userspace HPET is three-four
times faster than syscall HPET on several Core2 and SandyBridge
machines.
Tested by: Howard Su <howard0su@gmail.com>
Sponsored by: The FreeBSD Foundation
MFC after: 1 month
Differential revision: https://reviews.freebsd.org/D7473
fully-pessimized implementation that requires a type to be aligned to
its natural size.
On armv6+ the compiler might generate load-/store-multiple instructions
which require 4-byte alignment even though the source code is only
accessing individual uint32_t values in a way that doesn't require any
particular alignment at all. The compiler apparently feels free to
combine multiple accesses into a single instruction that requires a
more-strict alignment, and no set of compiler flags seems to disable
this behavior (at least in clang 3.8).
This fixes alignment faults on arm systems using wifi adapters. The
wifi code uses ALIGNED_POINTER(p, uint32_t) to decide whether it needs
to copy-align tcp headers. Because clang is combining several uint32_t
accesses into a single ldm instruction, we need to say that accessing a
uint32_t requires 4-byte alignment.
Approved by: re(gjb)
are no longer natural-alignment strict, there are still some restrictions.
FreeBSD network code assumes data is naturally-aligned or is running
on a platform with no restrictions; pointers are not annotated to
indicate the data pointed to may be packed or unaligned. The clang
optimizer can sometimes combine the load or store of a pair of adjacent
32-bit values into a single doubleword load/store, and that operation
requires at least 4-byte alignment. __NO_STRICT_ALIGNMENT can lead
to tcp headers being only 2-byte aligned.
Note that alignment faults remain disabled on armv6, this change reverts
only the defining of the symbol which leads to some overly-agressive code
shortcuts when building common/shared drivers and network code for arm.
Approved by: re(kib)
the exact CPU we are running on to set the cpu functions. Relax the check
to ignore the CPU revision. Even so this may still be too specific.
Reviewed by: mmel
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D6504
- Reset debug architecture and enable monitor for secondary
CPUs in init_secondary() rather than when configuring watchpoint, etc.
- Disable HW debugging capabilities when one of the CPU cores fails
to set up.
- Use dbg_capable() in a more atomic manner to avoid any mismatch
between CPUs.
Differential Revision: https://reviews.freebsd.org/D6009
to match the new state of affairs. The hardware we support has always been
able to do unaligned accesses, we've just never enabled it until now.
This brings FreeBSD into line with all the other major OSes, and should help
with the growing volume of 3rd-party software that assumes unaligned access
will just work on armv6 and armv7.
have ACLE support built in. The ACLE (ARM C Language Extensions) defines
a set of standardized symbols which indicate the architecture version and
features available. ACLE support is built in to modern compilers (both
clang and gcc), but absent from gcc prior to 4.4.
ARM (the company) provides the acle-compat.h header file to define the
right symbols for older versions of gcc. Basically, acle-compat.h does
for arm about the same thing cdefs.h does for freebsd: defines
standardized macros that work no matter which compiler you use. If ARM
hadn't provided this file we would have ended up with a big #ifdef __arm__
section in cdefs.h with our own compatibility shims.
Remove #include <machine/acle-compat.h> from the zillion other places (an
ever-growing list) that it appears. Since style(9) requires sys/types.h
or sys/param.h early in the include list, and both of those lead to
including cdefs.h, only a couple special cases still need to include
acle-compat.h directly.
Loves it: imp
where possible. In the places that doesn't work (multi-line inline asm,
and places where the old armv4 cpufuncs mechanism is used), annotate the
accesses with a comment that includes SCTLR. Now a grep -i sctlr can find
all the system control register manipulations.
No functional changes.
compilers can emit arm instructions that require 8-byte alignment. The
alignment-sensitive instructions were added in armv5, which has to be
supported by our combined v4/v5 kernels, so the value is set uncoditionally
for all arm architecture versions.
Also adjust the comment to explain in more detail why the macros have the
form and values they do.
Per advice from bde@, maintain the unsignedness of the value of _ALIGNBYTES
(but do so using his second choice of allowing sizeof() to supply the
unsignedness, rather than just hardcoding '8U', which in my mind would
require an even more verbose comment to explain why it's right). Also
explain in the comment that the resulting type of _ALIGN() is equivelent
to uinptr_t on arm (32-bit unsigned int), but it's purposely spelled as
"unsigned" to avoid problems with including other header files. Even
including machine/_types.h to allow use of __uintptr_t causes compilation
failures because of this header being included (indirectly) in asm code.
The discussion that led to this change (albeit at a glacial pace) is at
https://lists.freebsd.org/pipermail/svn-src-head/2014-November/064593.html
implementations. Early in the boot the kernel will use an approximate,
however after the timer has been probed it will switch to a more accurate
implementation.
Reviewed by: manu
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5762
and R/W emulation aborts under pmap lock.
There were two reasons for using of atomic operations:
(1) the pmap code is based on i386 one where they are used,
(2) there was an idea that access and R/W emulation aborts should be
handled as quick as possible, without pmap locking.
However, the atomic operations in i386 pmap code are used only because
page table entries may be modified by hardware. At the beginning, we
were not sure that it's the only reason. So even if arm hardware does
not modify them, we did not risk to not use them at that time. Further,
it turns out after some testing that using of pmap lock for access and
R/W emulation aborts does not bring any extra cost and there was no
measurable difference. Thus, we have decided finally to use pmap lock
for all operations on page table entries and so, there is no reason for
atomic operations on them. This makes the code cleaner and safer.
This decision introduce a question if it's safe to use pmap lock for
access and R/W emulation aborts. Anyhow, there may happen two cases in
general:
(A) Aborts while the pmap lock is locked already - this should not
happen as pmap lock is not recursive. However, under pmap lock only
internal kernel data should be accessed and such data should be mapped
with A bit set and NM bit cleared. If double abort happens, then
a mapping of data which has caused it must be fixed.
(B) Aborts while another lock(s) is/are locked - this already can
happen. There is no difference here if it's either access or R/W
emulation abort, or if it's some other abort.
Reviewed by: kib
(PL1) and unprivileged (PL0) read/write access. As cp15 virtual to
physical address translation operations are used, interrupts must be
disabled to get consistent result when they are called.
These functions should be used only in very specific occasions like
during abort handling or kernel debugging. One of them is going to be
used in pmap_fault(). However, complete function set is added. It cost
nothing, as they are inlined.
While here, fix comment of #endif.
Reviewed by: kib
universal.
(1) New struct intr_map_data is defined as a container for arbitrary
description of an interrupt used by a device. Typically, an interrupt
number and configuration relevant to an interrupt controller is encoded
in such description. However, any additional information may be encoded
too like a set of cpus on which an interrupt should be enabled or vendor
specific data needed for setup of an interrupt in controller. The struct
intr_map_data itself is meant to be opaque for INTRNG.
(2) An intr_map_irq() function is created which takes an interrupt
controller identification and struct intr_map_data as arguments and
returns global interrupt number which identifies an interrupt.
(3) A set of functions to be used by bus drivers is created as well as
a corresponding set of methods for interrupt controller drivers. These
sets take both struct resource and struct intr_map_data as one of the
arguments. There is a goal to keep struct intr_map_data in struct
resource, however, this way a final solution is not limited to that.
(4) Other small changes are done to reflect new situation.
This is only first step aiming to create stable interface for interrupt
controller drivers. Thus, some temporary solution is taken. Interrupt
descriptions for devices are stored in INTRNG and two specific mapping
function are created to be temporary used by bus drivers. That's why
the struct intr_map_data is not opaque for INTRNG now. This temporary
solution will be replaced by final one in next step.
Differential Revision: https://reviews.freebsd.org/D5730
- Don't convert atags address passed from U-Boot. It's real physical
address (and we have 1:1 mapping).
- Size of tags is encoded in words, not in bytes
This allow us to boot FreeBSD kernel (using uImage encapsulation) directly
from U-boot using 'bootm' command or by Android fastboot loader.
For now, kernel uImage must be marked as Linux, but we can add support for
FreeBSD into U-Boot later.
controller IPI provider.
New struct intr_ipi is defined which keeps all info about an IPI:
its name, counter, send and dispatch methods. Generic intr_ipi_setup(),
intr_ipi_send() and intr_ipi_dispatch() functions are implemented.
An IPI provider must implement two functions:
(1) an intr_ipi_send_t function which is able to send an IPI,
(2) a setup function which initializes itself for an IPI and
calls intr_ipi_setup() with appropriate arguments.
Differential Revision: https://reviews.freebsd.org/D5700
the interrupt framework is also going to be used by another (MIPS)
architecture. IPI implementations may vary much across different
architectures.
An IPI implementation should still define INTR_IPI_COUNT and use
intr_ipi_setup_counters() to setup IPI counters which are inside of
intrcnt[] and intrnames[] arrays. Those are used for sysctl and ddb.
Then, intr_ipi_increment_count() should be used to increment obtained
counter.
Reviewed by: imp
Differential Revision: https://reviews.freebsd.org/D5459
intr_pic_init_secondary. Replace them with a direct call. On BCM2836
and ARMADA XP we need to add this function, but it can be empty.
Reviewed by: ian, imp
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5460
slightly wrong on the others. We should just check if mp_ncpus is set to
more than one CPU as we may wish to run on a single core even when SMP is
available.
Reviewed by: ian
Sponsored by: ABT Systems Ltd
Differential Revision: https://reviews.freebsd.org/D5458
or pte-v6.h in files which needs it.
There are quite internal definitions in pte-v4.h and pte-v6.h headers
specific for corresponding pmap implementation. These headers should be
included only in very few files and an intention is to not hide for
which implementation such files are.
Further, sys/arm/arm/elf_trampoline.c is an example of file which
uses armv4 like pmap implementation for both armv4 and armv6 platforms.
This is another reason why pte.h which includes specific header
according to __ARM_ARCH is not created.
Create new pmap.h which includes specific header according to
__ARM_ARCH.
Note that <machine/pmap.h> is included from <vm/pmap.h> so one common
<machine/pmap.h> must exist.
are not utilized there. Only domain #0 is used and there is no reference
to it in the whole pmap-v6.c. Thus initialize domain access register in
locore-v6.c without reference too.
It turned out that devmap.c is not only file in which PTE_DEVICE
is used and simultaneously, built for both armv4 and armv6 platforms.
When I tried to build all arm kernels before r295168 commit, it was
hid by some other local changes in my tree. I hope that this is just
temporary workaround before VM_MEMATTR_DEVICE could be used instead of
PTE_DEVICE outside of pmap code for __ARM_ARCH < 6.
instead of hiding behind pmap_map_chunk(). It's not longer needed
after old pmap-v6 code was removed.
For compatibility with __ARM_ARCH < 6, define PTE_DEVICE in devmap.c
file. Certainly, it would be nice if VM_MEMATTR_DEVICE could be used
even for __ARM_ARCH < 6.
do not depend on pmap internals. This is a preparation for hiding
internal pmap definitions as much as possible from the rest of system.
Simultaneously, the protection argument evaluation is fixed. Happily,
it did not effect the mappings. And it's the reason why it was not fixed
earlier.
Use per-CPU structure to store HW watchpoints registers state
for each CPU present in the system. Those registers will be restored
upon wake up from the STOP state if requested by the debug_monitor
code. The method is similar to the one introduced to AMD64.
We store all possible 16 registers for HW watchpoints
(maximum allowed by the architecture).
HW breakpoints are not maintained since they are used for single
stepping only.
Pointed out by: kib
Reviewed by: wma
No strong objections from: kib
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by: Juniper Networks Inc.
Differential Revision: https://reviews.freebsd.org/D4338
This allows, for example, UEFI pass a memory map with some ram in this
region, but for us to ignore it. This is the case when running under the
qemu virt machine type.
Sponsored by: ABT Systems Ltd
Allows for using hardware watchpoints for 1, 2, 4, 8 byte long addresses.
The default configuration of watchpoint is RW but code allows to select
RO or WO and X.
Since debugging registers are per-CPU (CP14) the watchpoint is set on
the CPU that was lucky (or not) to enter DDB.
HW breakpoints are used to perform single step in KDB.
When HW breakpoint is enabled all watchpoints are temporary disabled
to avoid recursive abort on both watchpoint and breakpoint.
In case of branch, the breakpoint is set to both - next instruction
and possible branch address. This requires at least 2 breakpoints
supported in the CPU however this is a must for ARMv6/v7 CPUs.
Reviewed by: imp
Submitted by: Zbigniew Bodek <zbb@semihalf.com>
Obtained from: Semihalf
Sponsored by: Juniper Networks Inc.
Differential Revision: https://reviews.freebsd.org/D4037
pmap implementations on ARM. This way minidump code can be used without
any platform specific modification.
Also, this is the last piece missing for ARM_NEW_PMAP.
Differential Revision: https://reviews.freebsd.org/D5023
into a new function that other platforms can share.
This creates a new ofw_reg_to_paddr() function (in a new ofw_subr.c file)
that contains most of the existing ppc implementation, mostly unchanged.
The ppc code now calls the new MI code from the MD code, then creates a
ppc-specific bus_space mapping from the results. The new arm implementation
does the same in an arm-specific way.
This also moves the declaration of OF_decode_addr() from ofw_machdep.h to
openfirm.h, except on sparc64 which uses a different function signature.
This will help all FDT platforms to set up early console access using
OF_decode_addr().
The ci20 port (by kan@) is going to reuse almost all of the intrng code
since the SoC in question looks suspiciously like someone took an ARM
SoC design and replaced the ARM core with a MIPS core.
* migrate out the code;
* rename ARM_ -> INTR_;
* rename arm_ -> intr_;
* move the interrupt flush routine from intr.c / intrng.c into
arm/machdep_intr.c - removing the code duplication and removing
the ARM specific bits from here.
Thanks to the Star Wars: The Force Awakens premiere line for allowing
me a couple hours of quiet time to finish the universe builds.
Tested:
* make universe
TODO:
* The structure definitions in subr_intr.c still includes machine/intr.h
which requires one duplicates all of the intrng definitions in
the platform code (which kan has done, and I think we don't have to.)
Instead I should break out the generic things (function declarations,
common intr structures, etc) into a separate header.
* Kan has requested I make the PIC based IPI stuff optional.
(1) The pmap argument passed to the function must be current pmap only.
(2) The process must be single threaded as the function is called either
when a process is exiting or from exec_new_vmspace().
Remove pmap_tlb_flush_ng() which is not used anywhere now.
Approved by: kib (mentor)
clock_gettime(2) on ARMv7 and ARMv8 systems which have architectural
generic timer hardware. It is similar how the RDTSC timer is used in
userspace on x86.
Fix a permission problem where generic timer access from EL0 (or
userspace on v7) was not properly initialized on APs.
For ARMv7, mark the stack non-executable. The shared page is added for
all arms (including ARMv8 64bit), and the signal trampoline code is
moved to the page.
Reviewed by: andrew
Discussed with: emaste, mmel
Sponsored by: The FreeBSD Foundation
Differential revision: https://reviews.freebsd.org/D4209
Boundary Trace to assembly to reduce the overhead of these checks.
Submitted by: Howard Su <howard0su@gmail.com>
Relnotes: Yes
Differential Revision: https://reviews.freebsd.org/D4266
- add new TEX class for WT cacheable memory
- export new TEX class to kernel as VM_MEMATTR_WT attribute
- add new aliases VM_MEMATTR_WRITE_COMBINING and
VM_MEMATTR_WRITE_BACK, it's used in DRM code
Note:
Only Cortex A8 supports WT caching in HW. On rest of Cortex CPUs,
WT requests is treated as uncacheable.
Approved by: kib (mentor)
Replace tlb_flush_local() by tlb_flush() as even not global mappings
could be fetched to TLB(s) on other cores by speculative table walk.
From OS point of view, it was not a problem as either such mappings
were not used anymore or they were flushed from TLB(s) when reused.
However, from hardware point of view, it was a problem. Not flushed
mappings could be a target for speculative reads or prefetches (which
might be quite aggresive on ARM cores). As speculative read can fill
cacheline, it can cause a real problem, when physical page is reused,
but mapped with different memory attributes.
Anyhow, it's good to have only valid mappings in TLB(s).
Approved by: kib (mentor)
This structure must be binary compatible regardless of PMAP
version being used. Create reserved section for NEW_PMAP to
make other variables be placed exactly in the same memory
addresses. This fixes kgdb/gdb behavoiur, which uses pcb.h stuctures.
The NEW_PMAP is kernel flag, so it does not propagate to the buildworld,
what makes the tools using pcb.h unable to parse PCB data.
Reviewed by: mmel, kib
Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Juniper Networks Inc.
Differential Revision: https://reviews.freebsd.org/D4011
Allow manipulation with PSR_A bit on ARMv6+.
Remove declaration of unused functions.
This effectively enables asynchronous aborts on early bootstrap stage,
which previously was not enabled due to an error in enable_interrupts().
PR: 201434
Reported by: Gregory Soutade <soutade at gmail.com>
Approved by: kib (mentor)
function which checks an address for privileged (PL1) write access.
The function is inlined so it does not bring any cost, but makes
function set for checking privileged access complete.
Approved by: kib (mentor)
little-endian configuration for 64-bit variant is supported.
Reviewed by: mmel
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D4113
as of r288992 use it to manage the CCNT.
Use the CNNT for get_cyclecount() instead of binuptime() when device pmu
is compiled in; if it fails to attach, fall back to the former method.
Enable by default for the BeagleBoneBlack configuration.
Optained from: Cambridge/L41
Sponsored by: DARPA/AFRL
Reviewed by: andrew
Differential Revision: https://reviews.freebsd.org/D3837
In the old days, device drivers passed NULL for the parent tag when creating
a new tag, and on arm platforms that resulted in a global tag representing
overall platform constraints being substituted in the busdma code. Now all
drivers use bus_get_dma_tag() and if there is a need to represent overall
platform constraints they will be inherited from a tag supplied by nexus or
some bus driver in the hierarchy.
The only arm platforms still relying on the old global-tag scheme were some
xscale boards with special PCI-bus constraints. This change provides those
constraints through a tag supplied by the xscale PCI bus driver, and
eliminates the few remaining references to the old global var.
Reviewed by: cognet
Previous code supported only "continuous" code without any kind of
branch instructions. To change that, new function was implemented
which parses current instruction and returns an addres where
the jump might happen (alternative addr).
mdthread structure was extended to support two breakpoints
(one directly below current instruction and the second placed
at the alternative location).
One of them must trigger regardless the instruction has or has not been
executed due to condition field.
Upon cleanup, both software breakpoints are removed.
This implementation parses only the most common instructions
that are present in the code (like 99.99% of all), but there
is a chance there are some left, not covered by the parsing routine.
Parsing is done only for 32-bit instruction, no Thumb nor Thumb-2
support is provided.
Reviewed by: kib
Submitted by: Wojciech Macek <wma@semihalf.com>
Obtained from: Semihalf
Sponsored by: Juniper Networks Inc.
Differential Revision: https://reviews.freebsd.org/D4021
is a dcache invalidate to point of coherency just like dcache_inv_poc(), but
a slightly different version specific to dma operations. Elaborate the
comment about how and why it's different.
Similar to r286787 for x86, this treats userspace buffers the same as unmapped buffers and no longer borrows the UVA for sync operations.
Submitted by: Svatopluk Kraus <onwahe@gmail.com> (earlier revision)
Tested by: Svatopluk Kraus
Differential Revision: https://reviews.freebsd.org/D3869
The bits in the aux control register vary based on the processor type. In
the past we've always just set the 'smp' and "broadcast tlb/cache ops' bits,
which worked fine for the first few SoCs we supported. Now that we support
most of the cortex-a series processors, it's important to get the right bits
set based on the processor type.
Submitted by: Svatopluk Kraus <onwahe@gmail.com>
The interrupts-extended property is a list of controller-specific
interrupt tuples for more than one controller. The decode routine of
every PIC gets called in the pre-INTRNG code (nexus doesn't know which
device instance belongs to which fdt node), so the GIC code has to
check each FDT node it is asked to decode to ensure it is the owner.
Because in the pre-INTRNG world there can only be one instance of a GIC,
it's safe to cache the results of a positive lookup in a static variable
to avoid the expensive lookups on subsequent calls.
Submitted by: Svatopluk Kraus <onwahe@gmail.com>
Differential Revision: https://reviews.freebsd.org/D2345
and armv6 architecures. The primary enhancement over the old design is
support for hierarchical interrupt controllers (such as a gpio driver
which can receive interrupts from a root PIC and act as a PIC itself for
clients interested in handling a change of gpio pin state as an
interrupt). The new code also provides an infrastructure for mapping
interrupts described in metadata in the form of a "controller reference
plus interrupt number" tuple into the simple "0-n" flat numeric space
understood by rman and the bus resource mechanisms.
Use of the new code is enabled by setting the ARM_INTRNG option, and by
making a few simple changes to the platform's support code. In addition
each existing PIC driver needs changes to be ready for INTRNG; this commit
contains the changes for the arm/gic driver, which most armv6 SoCs use, but
it does not enable the new code yet on any platform.
This project has been many years in the making, starting as a GSoC project
by Jakub Klama (jceel@) in 2012. That didn't get committed right away and
the source base evolved out from under it to some degree. In 2014 I rebased
the diffs to then -current and did some enhancements in the area of mapping
interrupt numbers and storing associated fdt data, then the project went
cold again for a while. Eventually Svata Kraus took that work in progress
and did another big round of work on it, removing most of the remaining
rough edges. Finally I took that and made one more pass through it, mostly
disabling the "INTR_SOLO" feature for now, pending further design
discussions on how to most efficiently dispatch a pending interrupt through
more than one layer of PIC. The current code with the INTR_SOLO feature
disabled uses approximate 100 extra cpu cycles for each cascaded PIC the
interrupt has to be passed to, so what's left to do is about efficiency, not
correct operation.
Differential Revision: https://reviews.freebsd.org/D2047
the name the function will have when the new ARM_INTRNG code is integrated,
and doing this rename first will make it easier to toggle the new interrupt
handling code on/off with a config option for debugging.
casuword(9) and others, use LDRT and STRT instructions to access
memory with the privileges of userspace. If the *RT instruction
faults on the kernel address, then additional checks must be done to
not confuse the VM system with invalid kernel-mode faults.
Put ARM on line with other FreeBSD architectures and disallow usermode
buffers which intersect with the kernel address space in advance,
before any accesses are performed. In other words, vm_fault(9) is no
longer called when e.g. suword(9) stores to invalid (i.e. not
userspace) address.
Also, switch ARM to use fueword(9) and casueword(9).
Note: there is a pending patch in D3617, which adds the special
processing for faults from LDRT and STRT. The addition of the
processing is useful for potential other uses of the instructions and
for completeness, but standard userspace accessors are better served
by not allowing such faults beforehand.
Reviewed by: andrew
Sponsored by: The FreeBSD Foundation
Differential Revision: https://reviews.freebsd.org/D3816
MFC after: 2 weeks
pre-VFPv3 processors, since they do require software support code to
handle denormals. For VFPv3 and later, enable flush-to-zero if
hardware does not claim full denormals arithmetic support by VMVFR1_FZ
field in mvfr1 register.
The end result is that we do use correct fpu environment on Cortexes
with VFPv3, while ARM11 (e.g. rpi) is in non-compliant flush-to-zero
mode. At least CPUs without complete hardware implementation of
IEEE 754 do not cause unhandled floating point exception on underflow,
as it was before r288492.
Noted by: ian
Tested by: gjb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
ARMv6/7:
- Define _SAVE() macro to allow unwind data to be conditionally defined for
ARM assembly code in the kernel.
- Use _SAVE() to provide unwind information for bcopy_page(), and two (of
many) instances of copyin() and copyout().
Reviewed by: andrew, imp
MFC after: 3 days
Sponsored by: University of Cambridge
self-consistent, there is no need in anything but compiler barrier in
the implementation of atomic_thread_fence_*() on ARMv5. Split
implementation of fences for ARMv4/5 and ARMv6; the former use
compiler barriers, the later also perform hardware barriers.
An issue which is fixed by the change is the faults from the CP15
coprocessor accesses in the user mode. This was uncovered by the
pthread_once() changes in r287556.
Reported by: Mattia Rossi <mattia.rossi.mailinglists@gmail.com>
Discussed with: alc, cognet, jhb
Sponsored by: The FreeBSD Foundation
MFC after: 1 week
initial thread stack is not adjusted by the tunable, the stack is
allocated too early to get access to the kernel environment. See
TD0_KSTACK_PAGES for the thread0 stack sizing on i386.
The tunable was tested on x86 only. From the visual inspection, it
seems that it might work on arm and powerpc. The arm
USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already
incorrect for the threads with non-default kstack size. I only
changed the macros to use variable instead of constant, since I cannot
test.
On arm64, mips and sparc64, some static data structures are sized by
KSTACK_PAGES, so the tunable is disabled.
Sponsored by: The FreeBSD Foundation
MFC after: 2 week
vm_offset_t pmap_quick_enter_page(vm_page_t m)
void pmap_quick_remove_page(vm_offset_t kva)
These will create and destroy a temporary, CPU-local KVA mapping of a specified page.
Guarantees:
--Will not sleep and will not fail.
--Safe to call under a non-sleepable lock or from an ithread
Restrictions:
--Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms
--Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page.
--MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page
The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm.
NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing.
Reviewed by: kib
Approved by: kib (mentor)
Differential Revision: http://reviews.freebsd.org/D3013
provide a semantic defined by the C11 fences with corresponding
memory_order.
atomic_thread_fence_acq() gives r | r, w, where r and w are read and
write accesses, and | denotes the fence itself.
atomic_thread_fence_rel() is r, w | w.
atomic_thread_fence_acq_rel() is the combination of the acquire and
release in single operation. Note that reads after the acq+rel fence
could be made visible before writes preceeding the fence.
atomic_thread_fence_seq_cst() orders all accesses before/after the
fence, and the fence itself is globally ordered against other
sequentially consistent atomic operations.
Reviewed by: alc
Discussed with: bde
Sponsored by: The FreeBSD Foundation
MFC after: 3 weeks
discrimination between different subarch binaries, at least for mips
and arm. Arm is implemented, mips is still tbd, so not currently
exported. aarch64 does not export this because aarch64 binaries use
different tags and flags than arm.
Differential Revision: https://reviews.freebsd.org/D2611
1. Align to a 64-bit address so 64-bit data will be correctly aligned.
2. Add a comment explaining why.
3. Remove an unneeded value from the struct.
This fixes an issue where the struct may not be correctly aligned on the
stack in the syscall function. This may lead to accesing a 64-bit value
at a non 64-bit. This will raise an exception and panic the kernel.
We have been lucky where on arm and armv6 both clang and gcc correctly
align the data, even without us asking to, however, on armeb with clang to
not be the case. This tells the compiler we really do need this to be
aligned.
Reported and tested by: jmg (on armeb with clang)
MFC after: 1 Week [1, 2]
Perform cache writebacks and invalidations in the correct (inner to outer
or vice versa) order, and add comments that explain that.
Consistantly use 'va' as the variable name for virtual addresses.
Submitted by: Michal Meloun <meloun@miracle.cz>
For consistency with the naming conventions used by the other
implementations kill armv7_sleep and keep armv7_cpu_sleep.
Differential Revision: https://reviews.freebsd.org/D2537
Submitted by: John Wehle
Reviewed by: ian@, andrew@
because the i386 pmap on which the new armv6 pmap is based had it, and in
r281707 pmap_lazyfix() was removed from the i386 pmap.
Discussed with: kib
Submitted by: Michal Meloun (via Svatopluk Kraus)
Offet for the power control register was specified incorrectly (it had
the same value as the prefetch control register.) This change corrects
the offset value to 0xF80, per the ARM PL310 documentation.
Submitted by: Steve Kiernan <stevek@juniper.net>
Obtained from: Juniper Networks, Inc.
Previously we used pmap_kremove(), but with ARM_NEW_PMAP it does the remove
in a way that isn't SMP-coherent (which is appropriate in some circumstances
such as mapping/unmapping sf buffers). With matching enter/remove routines
for device mappings, each low-level implementation can do the right thing.
Reviewed by: Svatopluk Kraus <onwahe@gmail.com>
the PMC_IN_KERNEL() macro definition.
Add missing macros to extract the return address (LR) from the trapframe.
Discussed with: andrew
Obtained from: Cambridge/L41
Sponsored by: DARPA, AFRL
MFC after: 2 weeks
This is pretty much a complete rewrite based on the existing i386 code. The
patches have been circulating for a couple years and have been looked at by
plenty of people, but I'm not putting anybody on the hook as having reviewed
this in any formal sense except myself.
After this has gotten wider testing from the user community, ARM_NEW_PMAP
will become the default and various dregs of the old pmap code will be
removed.
Submitted by: Svatopluk Kraus <onwahe@gmail.com>,
Michal Meloun <meloun@miracle.cz>
Each plaform performs virtual memory split between kernel and user space
and assigns kernel certain amount of memory space. However, is is sometimes
reasonable to change the default values. Such situation may happen on
systems where the demand for kernel buffers is high, many devices occupying
memory etc. This of course comes with the cost of decreasing user space
memory range so shall be used with care. Most embedded systems will not
suffer from this limtation but rather take advantage of this potential
since default behavior is left unchanged.
Submitted by: Wojciech Macek <wma@semihalf.com>
Reviewed by: imp
Obtained from: Semihalf
