Commit Graph

1563 Commits

Author SHA1 Message Date
Andrew Turner
c30e5927b6 Move the SYSINIT to allow userspace access to the ARM generic timer later
in the boot. It doesn't need to be early, so move it to the SI_ORDER_ANY
stage of SI_SUB_SMP.

Sponsored by:	DARPA, AFRL
2018-06-20 11:13:10 +00:00
Andrew Turner
8214ff0f9e Move common GIC interrupt numbers to the common header. These are the same
across the GICv2 and GICv3 drivers so we only need a single copy of them.

Sponsored by:	Turing Robotic Industries
2018-06-19 16:14:23 +00:00
Emmanuel Vadot
aab8510031 arm timer: Use the default get_cntxct by default
Reported by:	kevans
2018-06-14 17:32:23 +00:00
Emmanuel Vadot
38d3befe9c arm timer: Add workaround for Allwinner A64 timer
The timer present in allwinner A64 SoC is unstable, value can jump backward
or forward.
It was found that when bit 11 and upper roll over the low bits can sometimes
being read as all as 1 or all as 0.
Simply ignore the values for those cases.
2018-06-14 17:18:15 +00:00
Matt Macy
eb7c901995 hwpmc: simplify calling convention for hwpmc interrupt handling
pmc_process_interrupt takes 5 arguments when only 3 are needed.
cpu is always available in curcpu and inuserspace can always be
derived from the passed trapframe.

While facially a reasonable cleanup this change was motivated
by the need to workaround a compiler bug.

core2_intr(cpu, tf) ->
  pmc_process_interrupt(cpu, ring, pmc, tf, inuserspace) ->
    pmc_add_sample(cpu, ring, pm, tf, inuserspace)

In the process of optimizing the tail call the tf pointer was getting
clobbered:

(kgdb) up
    at /storage/mmacy/devel/freebsd/sys/dev/hwpmc/hwpmc_mod.c:4709
4709                                pmc_save_kernel_callchain(ps->ps_pc,
(kgdb) up
1205                    error = pmc_process_interrupt(cpu, PMC_HR, pm, tf,

resulting in a crash in pmc_save_kernel_callchain.
2018-06-08 04:58:03 +00:00
Andrew Turner
9f1a80706c Allow us to read the physmap data into our own array and use this to build
the DMAP region on arm64.

We already have the needed information to build these tables, we just need
to extract it. This significantly simplifies the code.

Obtained from:	ABT Systems Ltd
Sponsored by:	Turing Robotic Industries
2018-05-24 15:32:49 +00:00
Andrew Turner
3a327967cc Pass the array length into regions_to_avail.
On arm64 we will need to get the phys_avail array from before the kernel
is excluded to create teh DMAP region. In preperation for this pass in the
array length into regions_to_avail.
2018-05-22 14:26:58 +00:00
Andrew Turner
5a00bf535c Only set realmem based on memory where the EXFLAG_NOALLOC is unset. This
will allow us to query the maps at any time without disturbing this value.

Obtained from:	ABT Systems Ltd
Sponsored by:	Turing Robotic Industries
2018-05-22 13:21:44 +00:00
Andrew Turner
66971d57e9 Allow the 32-bit arm physmem code to work on arm64.
This will help simplify the arm64 code and allow us to properly exclude
memory that should never be mapped.

Obtained from:	ABT Systems Ltd
Sponsored by:	Turing Robotic Industries
2018-05-22 10:31:06 +00:00
Andrew Turner
89ae4d7f7a Coalesce adjacent physical mappings.
This reduces the overhead when we have many small mappings, e.g. on some
EFI systems. This is to help use this code on arm64 where we may have a
large number of entries from the EFI firmware.

Obtained from:	ABT Systems Ltd
Sponsored by:	Turing Robotic Industries
Differential Revision:	https://reviews.freebsd.org/D15477
2018-05-22 10:14:20 +00:00
Andriy Gapon
c9c4d38aa8 followup to r332730/r332752: set kdb_why to "trap" for fatal traps
This change updates arm, arm64 and mips achitectures.  Additionally, it
removes redundant checks for kdb_active where it already results in
kdb_reenter() and adds kdb_reenter() calls where they were missing.

Some architectures check the return value of kdb_trap(), but some don't.
I haven't changed any of that.

Some trap handling routines have a return code.  I am not sure if I
provided correct ones for returns after kdb_reenter().  kdb_reenter
should never return unless kdb_jmpbufp is NULL for some reason.

Only compile tested for all affected architectures.  There can be bugs
resulting from my poor understanding of architecture specific details.

Reported by:	jhb
Reviewed by:	jhb, eadler
MFC after:	4 weeks
Differential Revision: https://reviews.freebsd.org/D15431
2018-05-16 06:52:08 +00:00
Mark Johnston
20f85b1ddd Print the dump progress indicator after calling dump_start().
Dumpers may wish to print messages from an initialization hook; this
change ensures that such messages aren't mixed with output from the
generic dump code.

MFC after:	1 week
2018-05-01 17:32:43 +00:00
Brooks Davis
9c11d8d483 Remove the unused fuwintr() and suiwintr() functions.
Half of implementations always failed (returned (-1)) and they were
previously used in only one place.

Reviewed by:	kib, andrew
Obtained from:	CheriBSD
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15102
2018-04-17 18:04:28 +00:00
Brooks Davis
427be5bb3a Remove unused implementations of copyoutstr().
Also remove the commented out documentation.  The documentation arrived
with the import of the copy.9 manpage.  I suspect the implementations
came from NetBSD while bootstrapping the Arm and MIPS ports.

Reviewed by:	andrew, jmallett
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D15108
2018-04-17 17:20:04 +00:00
Brooks Davis
6469bdcdb6 Move most of the contents of opt_compat.h to opt_global.h.
opt_compat.h is mentioned in nearly 180 files. In-progress network
driver compabibility improvements may add over 100 more so this is
closer to "just about everywhere" than "only some files" per the
guidance in sys/conf/options.

Keep COMPAT_LINUX32 in opt_compat.h as it is confined to a subset of
sys/compat/linux/*.c.  A fake _COMPAT_LINUX option ensure opt_compat.h
is created on all architectures.

Move COMPAT_LINUXKPI to opt_dontuse.h as it is only used to control the
set of compiled files.

Reviewed by:	kib, cem, jhb, jtl
Sponsored by:	DARPA, AFRL
Differential Revision:	https://reviews.freebsd.org/D14941
2018-04-06 17:35:35 +00:00
Andrew Turner
3a648f0472 Add the missing header for malloc(9). It was pulled in through header
pollution that doesn't seem to exist in some configurations.
2018-04-02 13:36:48 +00:00
Ed Maste
fc2a8776a2 Rename assym.s to assym.inc
assym is only to be included by other .s files, and should never
actually be assembled by itself.

Reviewed by:	imp, bdrewery (earlier)
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D14180
2018-03-20 17:58:51 +00:00
Ian Lepore
9c237f3a13 Add the header file needed for the recently-added call to pagedaemon_wakeup(). 2018-03-16 16:06:25 +00:00
Konstantin Belousov
8c8ee2ee1c Unify bulk free operations in several pmaps.
Submitted by:	Yoshihiro Ota
Reviewed by:	markj
MFC after:	2 weeks
Differential revision:	https://reviews.freebsd.org/D13485
2018-03-04 20:53:20 +00:00
Ian Lepore
665c7f6f1e Initialize all members of vm_page::md_page for armv4/5 systems. This fixes
a hang in SI_SUB_KMEM sysinit, and is apparently required after r323290.
Inspired by the commit message for r323676.

Reported by:	andreast@
2018-02-27 02:11:23 +00:00
Ian Lepore
e5c6bca6ba Add a hw.model sysctl oid for armv6/7 which reports the CPU model, similar
to what other arches (all except riscv and armv4/5) do.

Submitted by:	Hyun Hwang <hyun@caffeinated.codes>
Differential Revision:	https://reviews.freebsd.org/D14465
2018-02-26 23:58:56 +00:00
Ian Lepore
bf56e64a4c Add support for booting into kdb on arm platforms when the RB_KDB is set
(using "boot -d" at the loader propmt or setting boot_ddb in loader.conf).

Submitted by:	Thomas Skibo <thomasskibo@yahoo.com>
Differential Revision:	https://reviews.freebsd.org/D14428
2018-02-25 18:42:59 +00:00
Olivier Houchard
6cd04bcd8e Use NULL as a mtx type instead of "", as it otherwise confuses WITNESS. 2018-02-24 14:34:23 +00:00
Konstantin Belousov
2c0f13aa59 vm_wait() rework.
Make vm_wait() take the vm_object argument which specifies the domain
set to wait for the min condition pass.  If there is no object
associated with the wait, use curthread' policy domainset.  The
mechanics of the wait in vm_wait() and vm_wait_domain() is supplied by
the new helper vm_wait_doms(), which directly takes the bitmask of the
domains to wait for passing min condition.

Eliminate pagedaemon_wait().  vm_domain_clear() handles the same
operations.

Eliminate VM_WAIT and VM_WAITPFAULT macros, the direct functions calls
are enough.

Eliminate several control state variables from vm_domain, unneeded
after the vm_wait() conversion.

Scetched and reviewed by:	jeff
Tested by:	pho
Sponsored by:	The FreeBSD Foundation, Mellanox Technologies
Differential revision:	https://reviews.freebsd.org/D14384
2018-02-20 10:13:13 +00:00
Olivier Houchard
0511fa0efd Rename the ACPI variant of the gicv2m driver from "gicv2m" to "gicv2m_acpi".
The FDT variant is called "gicv2m" too, and as both would try to register
on gic, only one of them would succeed, while we want them both in a
GENERIC kernel.

Reviewed by:	andrew
2018-02-15 15:46:14 +00:00
Jeff Roberson
e958ad4cf3 Make v_wire_count a per-cpu counter(9) counter. This eliminates a
significant source of cache line contention from vm_page_alloc().  Use
accessors and vm_page_unwire_noq() so that the mechanism can be easily
changed in the future.

Reviewed by:	markj
Discussed with:	kib, glebius
Tested by:	pho (earlier version)
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D14273
2018-02-12 22:53:00 +00:00
Jeff Roberson
e2068d0bcd Use per-domain locks for vm page queue free. Move paging control from
global to per-domain state.  Protect reservations with the free lock
from the domain that they belong to.  Refactor to make vm domains more
of a first class object.

Reviewed by:    markj, kib, gallatin
Tested by:      pho
Sponsored by:   Netflix, Dell/EMC Isilon
Differential Revision:  https://reviews.freebsd.org/D14000
2018-02-06 22:10:07 +00:00
Michal Meloun
a36b6ec0f9 Implement mitigation for Spectre version 2 attacks on ARMv7.
Similarly as we already do for arm64, for mitigation is necessary to
flush branch predictor when we:
- do task switch
- receive prefetch abort on non-userspace address

The user can disable this mitigation by setting 'machdep.disable_bp_hardening'
sysctl variable, or it can check actual system status by reading
'machdep.spectre_v2_safe'

The situation is complicated by fact that:
- for Cortex-A8, the BPIALL instruction is effectively NOP until the IBE bit
  in ACTLR is set.
- for Cortex-A15, the BPIALL is always NOP. The branch predictor can be
  only flushed by doing ICIALLU with special bit (Enable invalidates  of BTB)
  set in ACTLR.

Since access to the ACTLR register is locked to secure monitor/firmware on
most boards, they will also need update of firmware / U-boot.
In worst case, when secure monitor is on-chip ROM (e.g. PandaBoard),
the board is unfixable.

MFC after:	2 weeks
Reviewed by:	imp, emaste
Differential Revision:	https://reviews.freebsd.org/D13931
2018-01-27 11:19:41 +00:00
Jeff Roberson
6f4acaf4c9 Add support for NUMA domains to bus dma tags. This causes all memory
allocated with a tag to come from the specified domain if it meets the
other constraints provided by the tag.  Automatically create a tag at
the root of each bus specifying the domain local to that bus if
available.

Reviewed by:	jhb, kib
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13545
2018-01-12 23:34:16 +00:00
Jeff Roberson
3f289c3fcf Implement 'domainset', a cpuset based NUMA policy mechanism. This allows
userspace to control NUMA policy administratively and programmatically.

Implement domainset based iterators in the page layer.

Remove the now legacy numa_* syscalls.

Cleanup some header polution created by having seq.h in proc.h.

Reviewed by:	markj, kib
Discussed with:	alc
Tested by:	pho
Sponsored by:	Netflix, Dell/EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D13403
2018-01-12 22:48:23 +00:00
Andrew Turner
ee42eb7e5c iAdd ACPI attachments the the GIC and GICv3 interrupt controller drivers.
For each we need to walk the MADT to find which we have, then add the
driver as needed. As each may have a child they will each walk the same
table to find these details.

Reviewed by:	mmel
Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8720
2018-01-11 17:23:24 +00:00
Andrew Turner
224c3776f6 Add the start of INTRNG support for ACPI.
This adds a new acpi_bus interface with a map_intr method. This is similar
to the Open Firmware map_intr method and allows us to create the needed
mapping from ACPI space to INTRNG space.

Obtained from:	ABT Systems Ltd
Sponsored by:	The FreeBSD Foundation
Differential Revision:	https://reviews.freebsd.org/D8617
2018-01-11 17:09:12 +00:00
Colin Percival
d5d7606c0c Use the TSLOG framework to record entry/exit timestamps for DELAY and
_vprintf; these functions are called in many places and can contribute
meaningfully to the total time spent booting.
2017-12-31 09:24:41 +00:00
Ian Lepore
2d09b07279 Make kernel option KERNVIRTADDR optional, remove it from std.<platform>
files that can use the default value.

It used to be required that the low-order bits of KERNVIRTADDR matched
the low-order bits of the physical load address for all arm platforms.
That hasn't been a requirement for armv6 platforms since FreeBSD 10.
There is no longer any relationship between load addr and KERNVIRTADDR
except that both must be aligned to a 2 MiB boundary.

This change makes the default KERNVIRTADDR value 0xc0000000, and removes the
options from all the platforms that can use the default value.  The default
is now defined in vmparam.h, and that file is now included in a few new
places that reference KERNVIRTADDR, since it may not come in via the
forced-include of opt_global.h on the compile command line.
2017-12-30 00:20:49 +00:00
Ian Lepore
513581749b Add a new ARM kernel option, LOCORE_MAP_MB, to control the size of the
kernel VA mapping in the temporary page tables set up by locore-v6.S.

The number used to be hard-coded to 64MB, which is still the default if
the kernel option is not specified.  However, 64MB is insufficient for
using a large mdroot filesystem.  The hard-coded number can't be safely
increased because too large a number may run into memory-mapped IO space
on some SoCs that must not be mapped as ordinary memory.
2017-12-26 19:02:56 +00:00
Alexander Kabaev
151ba7933a Do pass removing some write-only variables from the kernel.
This reduces noise when kernel is compiled by newer GCC versions,
such as one used by external toolchain ports.

Reviewed by: kib, andrew(sys/arm and sys/arm64), emaste(partial), erj(partial)
Reviewed by: jhb (sys/dev/pci/* sys/kern/vfs_aio.c and sys/kern/kern_synch.c)
Differential Revision: https://reviews.freebsd.org/D10385
2017-12-25 04:48:39 +00:00
Ian Lepore
68359587f6 If a temporary mapping is made to support EARLY_PRINTF, undo that mapping
after cninit() runs, otherwise we leave a bogus device-memory mapping in
userspace VA in the kernel pmap forever.

Pointed out by:	cognet
2017-12-20 22:19:11 +00:00
Ian Lepore
e3842da22f Allow pmap_kremove() to remove 1MB section mappings as well as 4K pages.
This will allow it to undo temporary device mappings such as those made
with pmap_preboot_map_attr().

Reviewed by:	cognet
2017-12-20 22:17:27 +00:00
Ian Lepore
df1e0a51ec Restore the ability to use EARLY_PRINTF support during most of initarm().
The real kernel page tables are set up much earlier in initarm() now than
they were when early printf support was first added, and they end up undoing
the mapping made in locore.S for early printf support.  This re-adds the
mapping after switching to the new/real kernel page tables, making early
printf work again right after switching to them.
2017-12-20 20:46:12 +00:00
Bruce Evans
fb3cc1c37d Move instantiation of msgbufp from 9 MD files to subr_prf.c.
This variable should be pure MI except possibly for reading it in MD
dump routines.  Its initialization was pure MD in 4.4BSD, but FreeBSD
changed this in r36441 in 1998.  There were many imperfections in
r36441.  This commit fixes only a small one, to simplify fixing the
others 1 arch at a time.  (r47678 added support for
special/early/multiple message buffer initialization which I want in
a more general form, but this was too fragile to use because hacking
on the msgbufp global corrupted it, and was only used for 5 hours in
-current...)
2017-12-07 07:55:38 +00:00
Pedro F. Giffuni
af3dc4a7ca sys/arm: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 2-Clause license, however the tool I
was using misidentified many licenses so this was mostly a manual - error
prone - task.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.
2017-11-27 15:04:10 +00:00
Andrew Turner
521018d379 Ensure we check the program state set in the trap frame on arm and arm64.
This value may be set by userspace so we need to check it before using it.
If this is not done correctly on exception return the kernel may continue
in kernel mode with all registers set to a userspace controlled value. Fix
this by moving the check into set_mcontext, and also add the missing
sanitisation from the arm64 set_regs.

Discussed with:	security-officer@
MFC after:	3 days
Sponsored by:	DARPA, AFRL
2017-11-23 17:40:40 +00:00
Pedro F. Giffuni
51369649b0 sys: further adoption of SPDX licensing ID tags.
Mainly focus on files that use BSD 3-Clause license.

The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.
2017-11-20 19:43:44 +00:00
Pedro F. Giffuni
df57947f08 spdx: initial adoption of licensing ID tags.
The Software Package Data Exchange (SPDX) group provides a specification
to make it easier for automated tools to detect and summarize well known
opensource licenses. We are gradually adopting the specification, noting
that the tags are considered only advisory and do not, in any way,
superceed or replace the license texts.

Special thanks to Wind River for providing access to "The Duke of
Highlander" tool: an older (2014) run over FreeBSD tree was useful as a
starting point.

Initially, only tag files that use BSD 4-Clause "Original" license.

RelNotes:	yes
Differential Revision:	https://reviews.freebsd.org/D13133
2017-11-18 14:26:50 +00:00
Michal Meloun
6adbcd6bd8 Remove support for ARMv6/v7 platform from elf_trampoline.
The elf_trampoline.c is not connected to build for ARMv6/v7 for long time and
it uses outdated low level cpu functions.
This blocks forthcoming cleanup of ARM code.

MFC after:	3 weeks
2017-11-05 17:14:44 +00:00
Svatopluk Kraus
d18f8e22ec Take into account race conditions in case of accessed or modified bit
emulation in fast path of data/prefetch abort common routine. Process
these bits only if related page table entries are consistent with
provided abort info. In case of inconsistency, do nothing and let
processor to signal new abort if still needed.

The mapping related to an abort may be a subject of change concurrently.
The situation is more evident on multicore machines. Mapping may be
removed on one core while being used on another one before TLB flush
happened. Memory swapping process may be an example. Or, two or more
aborts may be signaled for the same page on more cores concurrently.
While an abort on one core may cause a promotion of related mapping,
an abort on another core may be inconsistent then as related mapping
was promoted. A question is how much real the issue may be on single
core machine. However, it's better to play safe even for these machines.

This change may solve some "PT2MAP abort" panics reported rarely.
The revision of pmap_fault() was initiated thanks to stack backtrace
provided by Bob Prohaska (fbsd at www.zefox.net).

While here, INVARIANTS block was changed. The previous check had iffy
value as only one entry from many was checked from L2 page table.

Reviewed by:	mmel
MFC after:	3 weeks
2017-11-02 14:08:38 +00:00
Mark Johnston
5fca1d90c1 Fix the VM_NRESERVLEVEL == 0 build.
Add VM_NRESERVLEVEL guards in the pmaps that implement transparent
superpage promotion using reservations.

Reviewed by:	alc, kib
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D12764
2017-10-23 15:34:05 +00:00
Michal Meloun
32c48d07c6 Fix spelling.
Reported by:	lidl
MFC after:	1 month
2017-10-21 15:48:16 +00:00
Michal Meloun
0cbf724ed0 Fullify implementation of AT_HWCAP and AT_HWCAP2 for ARMv6,7.
This makes elf_aux_info(3) useable for ARM ports.

MFC after:	1 month
2017-10-21 12:16:21 +00:00
Mark Johnston
46fcd1af63 Move kernel dump offset tracking into MI code.
All of the kernel dump implementations keep track of the current offset
("dumplo") within the dump device. However, except for textdumps, they
all write the dump sequentially, so we can reduce code duplication by
having the MI code keep track of the current offset. The new
dump_append() API can be used to write at the current offset.

This is needed to implement support for kernel dump compression in the
MI kernel dump code.

Also simplify dump_encrypted_write() somewhat: use dump_write() instead
of duplicating its bounds checks, and get rid of the redundant offset
tracking.

Reviewed by:	cem
Sponsored by:	Dell EMC Isilon
Differential Revision:	https://reviews.freebsd.org/D11722
2017-10-18 15:38:05 +00:00