Commit Graph

869 Commits

Author SHA1 Message Date
hselasky
dd98a579d3 Implement atomic_swap_xxx() for all platforms.
Differential Revision:	https://reviews.freebsd.org/D18450
Reviewed by:		kib@
MFC after:		3 days
Sponsored by:		Mellanox Technologies
2018-12-10 13:38:13 +00:00
mmel
4b670ed804 Fix cut&paste typo in atomic_fetchadd_64().
Reported by:	Jia-Shiun Li <jiashiun@gmail.com>
MFC after:	1 week
2018-12-07 11:10:27 +00:00
jhb
81a93c8824 Add a KPI for the delay while spinning on a spin lock.
Replace a call to DELAY(1) with a new cpu_lock_delay() KPI.  Currently
cpu_lock_delay() is defined to DELAY(1) on all platforms.  However,
platforms with a DELAY() implementation that uses spin locks should
implement a custom cpu_lock_delay() doesn't use locks.

Reviewed by:	kib
MFC after:	3 days
2018-11-05 21:34:17 +00:00
kib
0c625bd638 Move the fixed base for PIE loading on arm.
Existing base causes conflicts for direct execution of ld-elf.so.1
because default linking base for non-PIE binaries is 0x10000.

Reported and tested by:	Mark Millard <marklmi26-fbsd@yahoo.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2018-11-04 19:11:32 +00:00
brooks
fcc5d25798 Consolidate identical ELF auxargs type defintions.
All platforms except powerpc use the same values and powerpc shares a
majority of them.

Go ahead and declare AT_NOTELF, AT_UID, and AT_EUID in favor of the
unused AT_DCACHEBSIZE, AT_ICACHEBSIZE, and AT_UCACHEBSIZE for powerpc.

Reviewed by:	jhb, imp
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D17397
2018-10-22 22:24:32 +00:00
andrew
28df9253f1 Remove the L1 and L2 xscale page defines and rename the generic macros to
the common name. While here move the macros to check these into pmap-v4.c
as they're only used there.

Sponsored by:	DARPA, AFRL
2018-08-16 10:00:51 +00:00
andrew
a8ffd76314 Remove pmap_kenter_section from the arm pmap. It's unused.
Sponsored by:	DARPA, AFRL
2018-08-15 14:57:34 +00:00
andrew
80017a7fd5 Remove ARM_HAVE_SUPERSECTIONS. It was only supported on some XScale CPUs.
Sponsored by:	DARPA, AFRL
2018-08-15 14:52:56 +00:00
andrew
c4122d2439 Make code and data only used within the arm pmap code as static.
Sponsored by:	DARPA, AFRL
2018-08-15 14:45:01 +00:00
andrew
5298a3d72f Remove arm pmap variables that are only ever set and never read.
Sponsored by:	DARPA, AFRL
2018-08-15 14:29:04 +00:00
andrew
506cd8abfa Remove ARM_MMU_GENERIC, it's the only ARMV4/v5 MMU we support.
Sponsored by:	DARPA, AFRL
2018-08-15 14:19:07 +00:00
andrew
abddf550b5 Remove checks for now unsupported CPU_* values in arm headers.
Sponsored by:	DARPA, AFRL
2018-08-15 13:48:59 +00:00
andrew
6a336d045d Start to remove XScale support from the ARMv4/v5 pmap. Support for XScale
has been removed from the kernel so we can remove it from here to help
simplify the code.

Sponsored by:	DARPA, AFRL
2018-08-15 13:40:16 +00:00
andrew
09c72a94ea Remove cpu_pfr from arm. It's unused. 2018-08-14 16:01:25 +00:00
alc
4cde6a5313 Add support for pmap_enter(..., psind=1) to the armv6 pmap. In other words,
add support for explicitly requesting that pmap_enter() create a 1 MB page
mapping.  (Essentially, this feature allows the machine-independent layer
to create superpage mappings preemptively, and not wait for automatic
promotion to occur.)

Export pmap_ps_enabled() to the machine-independent layer.

Add a flag to pmap_pv_insert_pte1() that specifies whether it should fail
or reclaim a PV entry when one is not available.

Refactor pmap_enter_pte1() into two functions, one by the same name, that
is a general-purpose function for creating pte1 mappings, and another,
pmap_enter_1mpage(), that is used to prefault 1 MB read- and/or execute-
only mappings for execve(2), mmap(2), and shmat(2).

In addition, as an optimization to pmap_enter(..., psind=0), eliminate the
use of pte2_is_managed() from pmap_enter().  Unlike the x86 pmap
implementations, armv6 does not have a managed bit defined within the PTE.
So, pte2_is_managed() is actually a call to PHYS_TO_VM_PAGE(), which is O(n)
in the number of vm_phys_segs[].  All but one call to PHYS_TO_VM_PAGE() in
pmap_enter() can be avoided.

Reviewed by:	kib, markj, mmel
Tested by:	mmel
MFC after:	6 weeks
Differential Revision:	https://reviews.freebsd.org/D16555
2018-08-08 16:55:01 +00:00
marius
4641f7c338 Implement atomic_swap_64(9). 2018-08-07 18:56:01 +00:00
andrew
ade951d0c6 Use the cp15 functions to read cp15 registers rather than using assembly
functions. The former are static inline functions so will compile to a
single instruction.
2018-07-28 17:21:34 +00:00
andrew
c935c49309 Make the arm cpu setup functions static. Any other place that needs these
functions will use the function pointer we create for them.
2018-07-28 12:20:42 +00:00
andrew
b411b4b9dd Remove old CPU_ values from the arm cpufunc code. These have been removed. 2018-07-28 12:00:32 +00:00
andrew
388c937191 Only support INTRNG in the SMP code on arm. We already require INTRNG on
anything that could be multicore on arm.
2018-07-28 07:54:21 +00:00
imp
de47afa9bf Revert r336773: it removed too much.
r336773 removed all things xscale. However, some things xscale are
really armv5. Revert that entirely. A more modest removal will follow.

Noticed by: andrew@
2018-07-27 21:25:01 +00:00
imp
0a4723bbc5 Remove xscale support
The OLD XSCALE stuff hasn't been useful in a while. The original
committer (cognet@) was the only one that had boards for it. He's
blessed this removal. Newer XSCALE (GUMSTIX) is for hardware that's
quite old. After discussion on arm@, it was clear there was no support
for keeping it.

Differential Review: https://reviews.freebsd.org/D16313
2018-07-27 18:33:09 +00:00
imp
a6cf0e2f77 Remove Atmel AT91RM9200 and AT91SAM9 support.
The last known robust version of this code base was FreeBSD 8.2. There
are no users of this on current, and all users of it have abandoned
this platform or are in legacy mode with a prior version of FreeBSD.

All known users on arm@ approved this removal, and there were no
objections.

Differential Revision: https://reviews.freebsd.org/D16312
2018-07-27 18:28:22 +00:00
manu
2e0566a9ef arm: Implement cpu_est_clockrate for armv[67] 2018-07-19 11:27:11 +00:00
mmel
19d66463c0 Remove the dead code from ARM cpufunc_* files.
The elf trampoline was never been supported for ARMv6 and ARMv7 and was
disconnected from kernel build many months ago.

MFC after:	2 weeks
2018-07-18 10:33:07 +00:00
andrew
f4bf04615a Also check if __ARM_ARCH_6KZ__ is defined when detecting when to use ARMv6
instructions. There is some code that still uses the _ARM_ARCH_* macros.

Sponsored by:	DARPA, AFRL
2018-07-18 09:17:37 +00:00
imp
000ad38a32 Remove kernel support for armeb
Remove all the big-endian arm architectures (ixp425 and ixp435)
support in the kernel and associated drivers.

Differential Revision:  https://reviews.freebsd.org/D16257
2018-07-17 23:23:45 +00:00
jhb
465d5a960e Export a breakpoint() function to userland for arm and arm64.
Enable ptrace() tests using breakpoint() on these architectures.

Reviewed by:	andrew
Differential Revision:	https://reviews.freebsd.org/D15191
2018-07-06 23:49:17 +00:00
mmacy
ff20311f27 Back pcpu zone with domain correct pages
- Change pcpu zone consumers to use a stride size of PAGE_SIZE.
  (defined as UMA_PCPU_ALLOC_SIZE to make future identification easier)

- Allocate page from the correct domain for a given cpu.

- Don't initialize pc_domain to non-zero value if NUMA is not defined
  There are some misconceptions surrounding this field. It is the
  _VM_ NUMA domain and should only ever correspond to valid domain
  values as understood by the VM.

The former slab size of sizeof(struct pcpu) was somewhat arbitrary.
The new value is PAGE_SIZE because that's the smallest granularity
which the VM can allocate a slab for a given domain. If you have
fewer than PAGE_SIZE/8 counters on your system there will be some
memory wasted, but this is obviously something where you want the
cache line to be coming from the correct domain.

Reviewed by: jeff
Sponsored by: Limelight Networks
Differential Revision:  https://reviews.freebsd.org/D15933
2018-07-06 02:06:03 +00:00
andrew
efd8540f2e Allow us to read the physmap data into our own array and use this to build
the DMAP region on arm64.

We already have the needed information to build these tables, we just need
to extract it. This significantly simplifies the code.

Obtained from:	ABT Systems Ltd
Sponsored by:	Turing Robotic Industries
2018-05-24 15:32:49 +00:00
imp
4a902ab86d Remove 'All Rights Reserved.' from all of my Copyrights in sys/arm and
always use 'M. Warner Losh' for consistency.

'All Rights Reserved.' was prescribed by the Buenos Aires Copyright
Convention of 1910, but has been mostly dead since the early 1990's
and completely meaningless since 2000 when Nicaragua ratified the
Berne convention.

Some files not done due to ambiguity of various types.
2018-05-01 16:21:01 +00:00
jhb
5c30edda81 Implement 32-bit atomic_fcmpset() in userland for armv4/v5.
- Add an implementation of atomic_fcmpset_32() using RAS for armv4/v5.
  This fixes recent world breakage due to use of atomic_fcmpset() in
  userland.
- While here, be more careful to not expose wrapper macros for 64-bit
  atomic_*cmpset to userland for armv4/v5 as only 32-bit cmpset is
  implemented.

This has been reviewed, but not runtime-tested, but should fix the arm.arm
and arm.armeb worlds that have been broken for a while.

Reviewed by:	imp
MFC after:	1 month
Differential Revision:	https://reviews.freebsd.org/D15147
2018-04-23 17:00:15 +00:00
jhb
07f4bcbf2f Fix some harmless type mismatches in the ARM atomic_cmpset implementations.
The return value of atomic_cmpset() and atomic_fcmpset() is an int (which
is really a bool) that has the values 0 or 1.  Some of the inlines were
using the type being operated on (e.g. uint32_t) as either the return type
of the function, or the type of a local 'ret' variable used to hold the
return value.  Fix all of these to just use plain 'int'.  Due to C promotion
rules and the fact that the value can only be 0 or 1, these should all be
harmless.

Reviewed by:	imp (only the v4 ones)
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D15147
2018-04-23 16:50:37 +00:00
imp
5cf1761272 No need to force md code to define a macro that's the same as
_BYTE_ORDER. Use that instead.
2018-04-16 13:52:23 +00:00
mw
06ba4aaddd Enable ArmadaXP using INTRNG interrupt controller
Defining INTRNG remove some necessary registers and declarations of
pic_init_secondary, pic_ipi_send, pic_ipi_read and pic_ipi_clear.
Because Marvell ArmadaXP and Armada38X always use INTRNG, include all
INTRNG code and remove code that does not use it.
Separate pic registers declarations for Armada38X are unnecessary, it
works properly with ArmadaXP config.

Submitted by: Rafal Kozik <rk@semihalf.com>
Reviewed by: andrew
Obtained from: Semihalf
Sponsored by: Stormshield
Differential Revision: https://reviews.freebsd.org/D14734
2018-04-03 21:17:19 +00:00
jtl
8e9b6569cb amd64: Protect the kernel text, data, and BSS by setting the RW/NX bits
correctly for the data contained on each memory page.

There are several components to this change:
 * Add a variable to indicate the start of the R/W portion of the
   initial memory.
 * Stop detecting NX bit support for each AP.  Instead, use the value
   from the BSP and, if supported, activate the feature on the other
   APs just before loading the correct page table.  (Functionally, we
   already assume that the BSP and all APs had the same support or
   lack of support for the NX bit.)
 * Set the RW and NX bits correctly for the kernel text, data, and
   BSS (subject to some caveats below).
 * Ensure DDB can write to memory when necessary (such as to set a
   breakpoint).
 * Ensure GDB can write to memory when necessary (such as to set a
   breakpoint).  For this purpose, add new MD functions gdb_begin_write()
   and gdb_end_write() which the GDB support code can call before and
   after writing to memory.

This change is not comprehensive:
 * It doesn't do anything to protect modules.
 * It doesn't do anything for kernel memory allocated after the kernel
   starts running.
 * In order to avoid excessive memory inefficiency, it may let multiple
   types of data share a 2M page, and assigns the most permissions
   needed for data on that page.

Reviewed by:	jhb, kib
Discussed with:	emaste
MFC after:	2 weeks
Sponsored by:	Netflix
Differential Revision:	https://reviews.freebsd.org/D14282
2018-03-06 14:28:37 +00:00
imp
f735e1eb15 Do not include float interfaces when using libsa.
We don't support float in the boot loaders, so don't include
interfaces for float or double in systems headers. In addition, take
the unusual step of spiking double and float to prevent any more
accidental seepage.
2018-02-23 04:04:25 +00:00
imp
8256f7ec08 Move __va_list and related defines to sys/sys/_types.h
__va_list and related defines are identical in all the
ARCH/include/_types.h files. Move them to sys/sys/_types.h

Sponsored by: Netflix
2018-02-12 14:48:20 +00:00
mmel
6d90ed5025 Implement mitigation for Spectre version 2 attacks on ARMv7.
Similarly as we already do for arm64, for mitigation is necessary to
flush branch predictor when we:
- do task switch
- receive prefetch abort on non-userspace address

The user can disable this mitigation by setting 'machdep.disable_bp_hardening'
sysctl variable, or it can check actual system status by reading
'machdep.spectre_v2_safe'

The situation is complicated by fact that:
- for Cortex-A8, the BPIALL instruction is effectively NOP until the IBE bit
  in ACTLR is set.
- for Cortex-A15, the BPIALL is always NOP. The branch predictor can be
  only flushed by doing ICIALLU with special bit (Enable invalidates  of BTB)
  set in ACTLR.

Since access to the ACTLR register is locked to secure monitor/firmware on
most boards, they will also need update of firmware / U-boot.
In worst case, when secure monitor is on-chip ROM (e.g. PandaBoard),
the board is unfixable.

MFC after:	2 weeks
Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D13931
2018-01-27 11:19:41 +00:00
nwhitehorn
49a1f46412 Define PHYS_TO_DMAP() and DMAP_TO_PHYS() as panics on the architectures
(i386 and arm) that never implement them. This allows the removal of
#ifdef PHYS_TO_DMAP on code otherwise protected by a runtime check on
PMAP_HAS_DMAP. It also fixes the build on ARM and i386 after I forgot an
#ifdef in r328168.

Reported by:	Milan Obuch
Pointy hat to:	me
2018-01-19 22:17:13 +00:00
nwhitehorn
e79f2b9178 Remove SFBUF_OPTIONAL_DIRECT_MAP and such hacks, replacing them across the
kernel by PHYS_TO_DMAP() as previously present on amd64, arm64, riscv, and
powerpc64. This introduces a new MI macro (PMAP_HAS_DMAP) that can be
evaluated at runtime to determine if the architecture has a direct map;
if it does not (or does) unconditionally and PMAP_HAS_DMAP is either 0 or
1, the compiler can remove the conditional logic.

As part of this, implement PHYS_TO_DMAP() on sparc64 and mips64, which had
similar things but spelled differently. 32-bit MIPS has a partial direct-map
that maps poorly to this concept and is unchanged.

Reviewed by:		kib
Suggestions from:	marius, alc, kib
Runtime tested on:	amd64, powerpc64, powerpc, mips64
2018-01-19 17:46:31 +00:00
ian
9df22297e3 Make kernel option KERNVIRTADDR optional, remove it from std.<platform>
files that can use the default value.

It used to be required that the low-order bits of KERNVIRTADDR matched
the low-order bits of the physical load address for all arm platforms.
That hasn't been a requirement for armv6 platforms since FreeBSD 10.
There is no longer any relationship between load addr and KERNVIRTADDR
except that both must be aligned to a 2 MiB boundary.

This change makes the default KERNVIRTADDR value 0xc0000000, and removes the
options from all the platforms that can use the default value.  The default
is now defined in vmparam.h, and that file is now included in a few new
places that reference KERNVIRTADDR, since it may not come in via the
forced-include of opt_global.h on the compile command line.
2017-12-30 00:20:49 +00:00
phk
1642f8ba74 Introduce an architecture-agnostic <sys/_stdarg.h> to reduce
platform divergence.

Only architectures which pass arguments in registers (mips)
and platforms which use really weird compilers (any?) would
need to augment the contents of <sys/_stdarg.h>

Convert x86, arm and arm64 architectures to use <sys/_stdarg.h>
2017-12-25 20:54:00 +00:00
ian
231505e8c9 Remove arm-specific implementations of atomic_load/store_xxx() now that
they are provided by sys/atomic_common.h.
2017-12-20 20:41:51 +00:00
kib
3d18a9d66f Add atomic_load(9) and atomic_store(9) operations.
They provide relaxed-ordered atomic access semantic.  Due to the
FreeBSD memory model, the operations are syntaxical wrappers around
the volatile accesses.  The volatile qualifier is used to ensure that
the access not optimized out and in turn depends on the volatile
semantic as implemented by supported compilers.

The motivation for adding the operation is to help people coming from
other systems or knowing the C11/C++ standards where atomics have
special type and require use of the special access operations.  It is
still the case that FreeBSD requires plain load and stores of aligned
integer types to be atomic.

Suggested by:	jhb
Reviewed by:	alc, jhb
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
Differential revision:	https://reviews.freebsd.org/D13534
2017-12-19 09:59:20 +00:00
andrew
c25c4cb4fc Add the virtualisation special register definitions.
Submitted by:	Mihai Carabas <mihai.carabas@gmail.com>
Reviewed by:	mmel
Differential Revision:	https://reviews.freebsd.org/D11846
2017-12-14 13:19:43 +00:00
pfg
1d4c34376a sys/arm: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:04:10 +00:00
pfg
4736ccfd9c sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:43:44 +00:00
pfg
9da7bdde06 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
mmel
1f066c248f All CP15 registers are bit fields or counters, don't use signed type
when accessing them.

MFC after:	3 weeks
2017-11-05 16:52:54 +00:00