Commit Graph

790 Commits

Author SHA1 Message Date
Michal Meloun
ffeeb2ab3f Split overbloated machep.c to multiple files and do basic cleanup
of these fragments.
2017-03-11 07:07:41 +00:00
Warner Losh
43b96a4eb5 Revert prior commit to restore the files mangled by my "fixing" merge
conflicts for a git rebase I tried to do.
2017-03-01 02:10:40 +00:00
Warner Losh
fbbd9655e5 Renumber copyright clause 4
Renumber cluase 4 to 3, per what everybody else did when BSD granted
them permission to remove clause 3. My insistance on keeping the same
numbering for legal reasons is too pedantic, so give up on that point.

Submitted by:	Jan Schaumann <jschauma@stevens.edu>
Pull Request:	https://github.com/freebsd/freebsd/pull/96
2017-02-28 23:42:47 +00:00
Ian Lepore
9fc7a59f2a Stop including sys/types.h from arm's machine/atomic.h, fix the places
where atomic.h was being included without ensuring that types.h (via
param.h) was included first, as required by atomic(9).
2017-02-11 01:07:46 +00:00
Konstantin Belousov
3b7a388b3e Update arm and arm64 counters MD bits.
On arm64 use atomics.  Then, both arm and arm64 do not need a critical
section around update.  Replace all cpus loop by CPU_FOREACH().
This brings arm and arm64 counter(9) implementation closer to current
amd64, but being more RISC-y, arm* version cannot avoid atomics.

Reported by:	Alexandre Martins <alexandre.martins@stormshield.eu>
Reviewed by:	andrew
Tested by:	Alexandre Martins, andrew
Sponsored by:	The FreeBSD Foundation
MFC after:	2 weeks
2017-02-06 17:20:37 +00:00
Konstantin Belousov
9fb10d635e Define the vm_ooffset_t and vm_pindex_t types as machine-independend.
The types are for the byte offset and page index in vm object.  They
are similar to off_t, which is defined as 64bit MI integer.  Using MI
definitions will allow to provide consistent MD values of vm
object-related maximum sizes.

Reviewed by:	alc
Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2017-02-04 12:26:38 +00:00
Andrew Turner
c1a2c9314f Remove an old use of _ARM_ARCH_6, we are moving to using the standard
__ARM_ARCH >= 6 spelling.

Sponsored by:	ABT Systems Ltd
2017-02-03 11:47:57 +00:00
Andrew Turner
aa7dd0fb0e Only define atomic_fcmpset_long in the kernel. We may include
machine/atomic.h in userspace, however atomic_fcmpset_32 is unimplemented
there.

Sponsored by:	ABT Systems Ltd
2017-02-03 10:04:17 +00:00
Michal Meloun
93a065e749 Remake support for SMP kernel on UP cpu:
- Use new option SMP_ON_UP instead of (mis)using specific CPU type.
   By this, any SMP kernel can be compiled with SMP_ON_UP support.
 - Enable runtime detection of CPU multiprocessor extensions only
   if SMP_ON_UP option is used. In other cases (pure SMP or UP),
   statically compile only required variant.
 - Don't leak multiprocessor instructions to UP kernel.
 - Correctly handle data cache write back to point of unification.
   DCCMVAU is supported on all armv7 cpus.
 - For SMP_ON_UP kernels, detect proper TTB flags on runtime.

Differential Revision: https://reviews.freebsd.org/D9133
2017-02-02 06:14:44 +00:00
Olivier Houchard
4943dc2d2c Correct the IT instruction in atomic_fcmpset_64().
Reported by:	andrew
2017-01-29 13:31:56 +00:00
Olivier Houchard
8406f869f4 Remove useless labels. 2017-01-28 17:48:33 +00:00
Olivier Houchard
14dd14fc00 Use strexeq instead of needlessly branch.
Suggested by:	ian
2017-01-28 17:46:04 +00:00
Olivier Houchard
dc5f9fcdae Implement atomic_fcmpset_* for arm and arm64. 2017-01-28 16:24:06 +00:00
Jason A. Harmening
fc100b8af2 Further cleanup of per-CPU armv6 pmap data:
- Replace pcpu_find(curcpu) with get_pcpu(), which is much
  more direct.

- Remove armv4 pcpu fields which I added in r286296 but never
  needed to use.

- armv6 pc_qmap_addr was leftover from the old armv6 pmap
  implementation.  Rename it and put it to use in the new one.

Noted by:	skra
Reviewed by:	skra
MFC after: 	1 week
Differential Revision:	https://reviews.freebsd.org/D9312
2017-01-26 05:23:33 +00:00
Jason A. Harmening
e0b79e669b Like r310481 for i386, move the objects used to create temporary
mappings for armv6 pmap zero and copy operations to the MD PCPU region.
Change sysmap initialization to only allocate KVA pages for CPUs that
are actually present.

While here, collapse CMAP3 into CMAP2 (their use was mutually exclusive
anyway) and "recover" some space in PCPU padding that has always been
available due to 64-byte cacheline padding.

Reviewed by:	skra
MFC after:	1 week
Differential Revision:	https://reviews.freebsd.org/D9172
2017-01-22 00:46:04 +00:00
Ian Lepore
3b9a898298 Remove arm's cpuconf.h, and references to it, after moving a few lines from
it into pmap-v4.h where they are used.  Other than those few lines of
support for different MMU types, nothing in cpuconf.h has been used in our
code for quite a while.

The file existed to set up a variety of symbols to describe the
architecture.  Over the past few years we have converted all of our source
to use the new architecture symbols standardized by ARM Inc, and predefined
by both clang and gcc.

PR:		216104
2017-01-16 16:44:13 +00:00
Mark Johnston
33ce6ec1d9 Add some missing atomic_*_ptr #defines for arm.
MFC after:	1 week
2017-01-04 00:18:51 +00:00
Luiz Otavio O Souza
ff30498384 Fix the armv6 build after r309553.
Sponsored by:	Rubicon Communications, LLC (Netgate)
2016-12-06 06:15:28 +00:00
Michal Meloun
65d5084f54 Fix build breakage caused by r309531.
Reported by: andrew
MFC after: 2 weeks
X-MFC with: r309531
2016-12-05 15:55:51 +00:00
Michal Meloun
bdbc1b764d Implement fake pmap_mapdev_attr() for ARMv6.
This function is referenced, but never called from DRM2 code. Also,
real behavior of pmap_mapdev_attr() in ARM world is unclear as we don't
have any additional attribute for a device memory type.

MFC after: 2 weeks
2016-12-04 15:27:39 +00:00
Bryan Drewery
28323add09 Fix improper use of "its".
Sponsored by:	Dell EMC Isilon
2016-11-08 23:59:41 +00:00
Stanislav Galabov
c1bc87b161 Generate an error if machine/armreg.h is included without sys/cdefs.h
machine/armreg.h requires access to the __ARM_ARCH macro, which is not
always properly defined (especially by gcc 4.2.1). We should include
sys/cdefs.h in order to get the definitions in machine/acle-compat.h,
which would properly define the __ARM_ARCH macro in these cases.

So, in cases where machine/armreg.h is included without _SYS_CDEFS_H_
being defined - generate an #error.

Reviewed by:	andrew
Sponsored by:	Smartcom - Bulgaria AD
Differential Revision:	https://reviews.freebsd.org/D8460
2016-11-07 11:35:14 +00:00
Andrew Turner
e9524a3c2f Allow an SMP kernel to boot on Cortex-A8:
* Rename ARM_HAVE_MP_EXTENSIONS to ARM_USE_MP_EXTENSIONS and extend it to
   handle more cases, including when SMP is not enabled.
 * Check ARM_USE_MP_EXTENSIONS when building for ARMv7+, even if no SMP.
 * Use ARM_USE_MP_EXTENSIONS in pmap-v6.c to detect when to set PRRR_NS1.

With this we should be able to boot on all ARMv7+ Cortex-A cores with
32-bit support.

Reviewed by:	mmel, imp (earlier version)
Relnotes:	yes
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D8335
2016-11-02 13:10:08 +00:00
Andrew Turner
12f7add9f7 Remove the need for the delay to be zero when MULTIDELAY is undefined,
it may be useful to only enable this in some configs.

Sponsored by:	ABT Systems Ltd
2016-10-25 17:57:31 +00:00
Andrew Turner
bf715082c3 Remove armadaxp_idcache_wbinv_all, it's a static function in the ELF
trampoline and not used outside this.

Sponsored by:	ABT Systems Ltd
2016-10-25 16:33:05 +00:00
Andrew Turner
ff300d2316 Remove arm11x6_setttb and armv7_setttb as they are unused. While here
remove unneeded code from the ARMv7 cpu assembly code.

Sponsored by:	ABT Systems Ltd
2016-10-25 16:25:06 +00:00
Andrew Turner
ce9fa35b35 Update the armv6 tlb handling functions to detect if it is running on
hardware that supports the mp extensions. If so it should use the broadcast
tlb invalidate instructions as other CPUs or devices may need to know about
the invalidation.

To simplify the code have the compiler optimise out the else case when not
builing for Cortex-A8.

Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D8092
2016-10-25 13:45:59 +00:00
Warner Losh
b2a7ac4802 Fix building on i386 and arm. But 'public domain' headers on the files
with no creative content. Include "lost" changes from git:
o Use /dev/efi instead of /dev/efidev
o Remove redundant NULL checks.

Submitted by: kib@, dim@, zbb@, emaste@
2016-10-13 06:56:23 +00:00
Michal Meloun
ba0bb20635 ARM: Split identify_arm_cpu() into ARMv4 and ARMv6 variant.
On ARMv6, be more verbose about supported CPU features and/or
optional instructions.
2016-10-09 10:24:10 +00:00
Michal Meloun
7cc70732a3 ARM: SEV/WFE instructions are implemented starting from ARMv6K,
use it directly.

MFC after: 1 week
2016-10-06 13:18:18 +00:00
Michal Meloun
55e447c98c ARM: Add identifiers for ARM Cortex v8 and Marvell Sheeva v7 cores.
Not a functional change.

MFC after: 3 days
2016-10-06 12:01:10 +00:00
Andrew Turner
c5bf621b7a We don't use cpu_control on armv6, remove the macro there. 2016-10-05 14:00:05 +00:00
Michal Meloun
9567acca41 ARM: Remove next bunch of unused cpu_functions from ARMv6. 2016-10-05 12:19:09 +00:00
Andrew Turner
1834282de6 Split CPU_CORTEXA into CPU_CORTEXA8, for the Cortex-A8, and CPU_CORTEXA_MP,
for later Cortex-A CPUs that support the Multiprocessor Extensions. This
will be needed to support both in a single GENERIC kernel while still
being able to only build for a single SoC.

Reviewed by:	mmel
Relnotes:	yes
Sponsored by:	ABT Systems Ltd
Differential Revision:	https://reviews.freebsd.org/D8138
2016-10-04 12:25:44 +00:00
Andrew Turner
d029cb6152 Read the Multiprocessing Extensions bit from the Multiprocessor Affinity
register.

Sponsored by:	ABT Systems Ltd
X-Differential Revision:	https://reviews.freebsd.org/D8092
2016-10-04 12:01:08 +00:00
Michal Meloun
fe8151a0e3 ARM: Add mising early clobber modifier in atomic_swap_32().
MFC after: 2 weeks
2016-10-04 09:59:37 +00:00
Michal Meloun
6542d1a4e7 ARM: Add atomic_swap_64(). It's need by linuxkpi and drm-next-4.7.
MFC after: 2 weeks
2016-10-04 09:51:54 +00:00
Andrew Turner
905339874b Add the Cortex-A{53,57,72} ID register values. These can all run 32-bit
code so could run a 32-bit kernel.

Sponsored by:	ABT Systems Ltd
2016-10-03 18:00:10 +00:00
Andrew Turner
0e484d95ca Remove unused functions on armv6. Some of the cache handling code is still
used in the elf trampoline so add a macro to handle this.

Sponsored by:	ABT Systems Ltd
2016-10-03 16:10:38 +00:00
Andrew Turner
610d93d80a Remove the parts of cpu_functions from armv6 that are unused on that
architecture.

Sponsored by:	ABT Systems Ltd
2016-10-03 14:43:57 +00:00
Andrew Turner
7efd943a95 Revert an accedintal commit. 2016-09-30 14:00:23 +00:00
Andrew Turner
968e30efd5 Fix the spelling of Cortex. 2016-09-30 13:47:52 +00:00
Andrew Turner
6c78e95a80 Remove struct platform_data, it was never used.
Sponsored by:	ABT Systems Ltd
2016-09-29 08:49:12 +00:00
Andrew Turner
c35b5d8372 Remove bus_dma_get_range and bus_dma_get_range_nb on armv6. We only need
this on a few earlier arm SoCs.

Reviewed by:	manu (earlier version)
Sponsored by:	ABT Systems Ltd
2016-09-23 12:38:05 +00:00
Ed Schouten
fa878ec311 Make it possible to safely use TPIDRURW from userspace.
On amd64, arm64 and i386, we have the possibility to switch between TLS
areas in userspace. The nice thing about this is that it makes it easier
to do light-weight threading, if we ever feel like doing that. On armv6,
let's go into the same direction by making it possible to safely use the
TPIDRURW register, which is intended for this purpose.

Clean up the ARMv6 code to remove md_tp entirely. Simply add a dedicated
field to the PCB to hold the value of TPIDRURW across context switches,
like we do for any other register. As userspace currently uses the
read-only TPIDRURO register, simply ensure that we keep both values in
sync where possible. The system calls for modifying the read-only
register will simply write the intended value into both registers, so
that it lazily ends up in the PCB during the next context switch.

Reviewed by:	https://reviews.freebsd.org/D7951
Approved by:	andrew
Reviewed by:	imp
Differential Revision:	https://reviews.freebsd.org/D7951
2016-09-22 08:14:59 +00:00
Konstantin Belousov
559a7b209a Add a way for the architecture to specify the calling ABI for methods
in the EFI Runtime Services Table.  On amd64, the calling conventions
are MS.

Sponsored by:	The FreeBSD Foundation
MFC after:	1 week
2016-09-21 10:35:44 +00:00
Konstantin Belousov
1680854946 Implement userspace gettimeofday(2) with HPET timecounter.
Right now, userspace (fast) gettimeofday(2) on x86 only works for
RDTSC.  For older machines, like Core2, where RDTSC is not C2/C3
invariant, and which fall to HPET hardware, this means that the call
has both the penalty of the syscall and of the uncached hw behind the
QPI or PCIe connection to the sought bridge.  Nothing can me done
against the access latency, but the syscall overhead can be removed.
System already provides mappable /dev/hpetX devices, which gives
straight access to the HPET registers page.

Add yet another algorithm to the x86 'vdso' timehands. Libc is updated
to handle both RDTSC and HPET.  For HPET, the index of the hpet device
to mmap is passed from kernel to userspace, index might be changed and
libc invalidates its mapping as needed.

Remove cpu_fill_vdso_timehands() KPI, instead require that
timecounters which can be used from userspace, to provide
tc_fill_vdso_timehands{,32}() methods.  Merge i386 and amd64
libc/<arch>/sys/__vdso_gettc.c into one source file in the new
libc/x86/sys location.  __vdso_gettc() internal interface is changed
to move timecounter algorithm detection into the MD code.

Measurements show that RDTSC even with the syscall overhead is faster
than userspace HPET access.  But still, userspace HPET is three-four
times faster than syscall HPET on several Core2 and SandyBridge
machines.

Tested by:	Howard Su <howard0su@gmail.com>
Sponsored by:	The FreeBSD Foundation
MFC after:	1 month
Differential revision:	https://reviews.freebsd.org/D7473
2016-08-17 09:52:09 +00:00
Ed Maste
dd2be8cb6e Move/add ARM ELF PHDR types to elf_common.h
Accidentally missed in r303674
2016-08-02 20:26:04 +00:00
Andrey A. Chernov
239e5577a2 Undo r302601, WCHAR_MAX may not be a valid wchar value. 2016-07-12 04:20:44 +00:00
Andrey A. Chernov
713a61670c I don't know why unsigned int is choosed for wchar_t here, but WCHAR_MAX
should be <= WINT_MAX. It is bigger, __UINT_MAX > INT32_MAX
2016-07-12 00:37:48 +00:00