d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
9666cda976
freebsd-dev/sys/arm64/include/pte.h

158 lines
5.2 KiB
C
Raw Normal View History

Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
/*-
* Copyright (c) 2014 Andrew Turner
* Copyright (c) 2014-2015 The FreeBSD Foundation
* All rights reserved.
*
* This software was developed by Andrew Turner under
* sponsorship from the FreeBSD Foundation.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
* ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
* ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE
* FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
* DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
* OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
* HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
* LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
* OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
* SUCH DAMAGE.
*
* $FreeBSD$
*/
#ifndef _MACHINE_PTE_H_
#define _MACHINE_PTE_H_
#ifndef LOCORE
typedef uint64_t pd_entry_t; /* page directory entry */
typedef uint64_t pt_entry_t; /* page table entry */
#endif
/* Block and Page attributes */
Split out the stage 1 pte bits and add the stage 2 bits In preperation for adding bhyve support to arm64 we need to split the stage 1 and stage 2 pte fields to allow future changes to create stage 2 page tables. MFC after: 1 month Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D23669
2020-02-24 16:45:31 +00:00
#define ATTR_MASK_H UINT64_C(0xfffc000000000000)
#define ATTR_MASK_L UINT64_C(0x0000000000000fff)
#define ATTR_MASK (ATTR_MASK_H | ATTR_MASK_L)
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
/* Bits 58:55 are reserved for software */
Split out the stage 1 pte bits and add the stage 2 bits In preperation for adding bhyve support to arm64 we need to split the stage 1 and stage 2 pte fields to allow future changes to create stage 2 page tables. MFC after: 1 month Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D23669
2020-02-24 16:45:31 +00:00
#define ATTR_SW_UNUSED2 (1UL << 58)
#define ATTR_SW_UNUSED1 (1UL << 57)
#define ATTR_SW_MANAGED (1UL << 56)
#define ATTR_SW_WIRED (1UL << 55)
#define ATTR_S1_UXN (1UL << 54)
#define ATTR_S1_PXN (1UL << 53)
#define ATTR_S1_XN (ATTR_S1_PXN | ATTR_S1_UXN)
#define ATTR_S2_XN(x) ((x) << 53)
Support creating and using arm64 pmap at stage 2 Add minimal support for creating stage 2 IPA -> PA mappings. For this we need to: - Create a new vmid set to allocate a vmid for each Virtual Machine - Add the missing stage 2 attributes - Use these in pmap_enter to create a new mapping - Handle stage 2 faults The vmid set is based on the current asid set that was generalised in r358328. It adds a function pointer for bhyve to use when the kernel needs to reset the vmid set. This will need to call into EL2 and invalidate the TLB. The stage 2 attributes have been added. To simplify setting these fields two new functions are added to get the memory type and protection fields. These are slightly different on stage 1 and stage 2 tables. We then use them in pmap_enter to set the new level 3 entry to be stored. The D-cache on all entries is cleaned to the point of coherency. This is to allow the data to be visible to the VM. To allow for userspace to load code when creating a new executable entry an invalid entry is created. When the VM tried to use it the I-cache is invalidated. As the D-cache has already been cleaned this will ensure the I-cache is synchronised with the D-cache. When the hardware implements a VPIPT I-cache we need to either have the correct VMID set or invalidate it from EL2. As the host kernel will have the wrong VMID set we need to call into EL2 to clean it. For this a second function pointer is added that is called when this invalidation is needed. Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D23875
2020-05-27 08:00:38 +00:00
#define ATTR_S2_XN_MASK ATTR_S2_XN(3UL)
#define ATTR_S2_XN_NONE 0UL /* Allow execution at EL0 & EL1 */
#define ATTR_S2_XN_EL1 1UL /* Allow execution at EL0 */
#define ATTR_S2_XN_ALL 2UL /* No execution */
#define ATTR_S2_XN_EL0 3UL /* Allow execution at EL1 */
Split out the stage 1 pte bits and add the stage 2 bits In preperation for adding bhyve support to arm64 we need to split the stage 1 and stage 2 pte fields to allow future changes to create stage 2 page tables. MFC after: 1 month Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D23669
2020-02-24 16:45:31 +00:00
#define ATTR_CONTIGUOUS (1UL << 52)
#define ATTR_DBM (1UL << 51)
#define ATTR_S1_nG (1 << 11)
#define ATTR_AF (1 << 10)
#define ATTR_SH(x) ((x) << 8)
#define ATTR_SH_MASK ATTR_SH(3)
#define ATTR_SH_NS 0 /* Non-shareable */
#define ATTR_SH_OS 2 /* Outer-shareable */
#define ATTR_SH_IS 3 /* Inner-shareable */
#define ATTR_S1_AP_RW_BIT (1 << 7)
#define ATTR_S1_AP(x) ((x) << 6)
#define ATTR_S1_AP_MASK ATTR_S1_AP(3)
#define ATTR_S1_AP_RW (0 << 1)
#define ATTR_S1_AP_RO (1 << 1)
#define ATTR_S1_AP_USER (1 << 0)
#define ATTR_S1_NS (1 << 5)
#define ATTR_S1_IDX(x) ((x) << 2)
#define ATTR_S1_IDX_MASK (7 << 2)
#define ATTR_S2_S2AP(x) ((x) << 6)
Support creating and using arm64 pmap at stage 2 Add minimal support for creating stage 2 IPA -> PA mappings. For this we need to: - Create a new vmid set to allocate a vmid for each Virtual Machine - Add the missing stage 2 attributes - Use these in pmap_enter to create a new mapping - Handle stage 2 faults The vmid set is based on the current asid set that was generalised in r358328. It adds a function pointer for bhyve to use when the kernel needs to reset the vmid set. This will need to call into EL2 and invalidate the TLB. The stage 2 attributes have been added. To simplify setting these fields two new functions are added to get the memory type and protection fields. These are slightly different on stage 1 and stage 2 tables. We then use them in pmap_enter to set the new level 3 entry to be stored. The D-cache on all entries is cleaned to the point of coherency. This is to allow the data to be visible to the VM. To allow for userspace to load code when creating a new executable entry an invalid entry is created. When the VM tried to use it the I-cache is invalidated. As the D-cache has already been cleaned this will ensure the I-cache is synchronised with the D-cache. When the hardware implements a VPIPT I-cache we need to either have the correct VMID set or invalidate it from EL2. As the host kernel will have the wrong VMID set we need to call into EL2 to clean it. For this a second function pointer is added that is called when this invalidation is needed. Sponsored by: Innovate UK Differential Revision: https://reviews.freebsd.org/D23875
2020-05-27 08:00:38 +00:00
#define ATTR_S2_S2AP_MASK 3
#define ATTR_S2_S2AP_READ 1
#define ATTR_S2_S2AP_WRITE 2
#define ATTR_S2_MEMATTR(x) ((x) << 2)
#define ATTR_S2_MEMATTR_MASK ATTR_S2_MEMATTR(0xf)
#define ATTR_S2_MEMATTR_DEVICE_nGnRnE 0x0
#define ATTR_S2_MEMATTR_NC 0xf
#define ATTR_S2_MEMATTR_WT 0xa
#define ATTR_S2_MEMATTR_WB 0xf
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
Set memory to be inner-sharable. This isn't needed on device memory as the MMU will ignore the attribute there, howeverit simplifies to code to alwas set it. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
2015-07-14 12:37:47 +00:00
#define ATTR_DEFAULT (ATTR_AF | ATTR_SH(ATTR_SH_IS))
Ordinarily, during a superpage promotion or demotion within a pmap, the pmap's lock ensures that other operations on the pmap don't observe the old mapping being broken before the new mapping is established. However, pmap_kextract() doesn't acquire the kernel pmap's lock, so it may observe the broken mapping. And, if it does, it returns an incorrect result. This revision implements a lock-free solution to this problem in pmap_update_entry() and pmap_kextract() because pmap_kextract() can't acquire the kernel pmap's lock. Reported by: andrew, greg_unrelenting.technology Reviewed by: andrew, markj Tested by: greg_unrelenting.technology X-MFC with: r350579 Differential Revision: https://reviews.freebsd.org/D21169
2019-08-08 06:26:34 +00:00
#define ATTR_DESCR_MASK 3
#define ATTR_DESCR_VALID 1
#define ATTR_DESCR_TYPE_MASK 2
#define ATTR_DESCR_TYPE_TABLE 2
#define ATTR_DESCR_TYPE_PAGE 2
#define ATTR_DESCR_TYPE_BLOCK 0
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
/* Level 0 table, 512GiB per entry */
#define L0_SHIFT 39
Add support for 4 level pagetables. The userland address space has been increased to 256TiB. The kernel address space can also be increased to be the same size, but this will be performed in a later change. To help work with an extra level of page tables two new functions have been added, one to file the lowest level table entry, and one to find the block/page level. Both of these find the entry for a given pmap and virtual address. This has been tested with a combination of buildworld, stress2 tests, and by using sort to consume a large amount of memory by sorting /dev/zero. No new issues are known to be present from this change. Reviewed by: kib Obtained from: ABT Systems Ltd Relnotes: yes Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D5720
2016-03-31 11:07:24 +00:00
#define L0_SIZE (1ul << L0_SHIFT)
#define L0_OFFSET (L0_SIZE - 1ul)
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
#define L0_INVAL 0x0 /* An invalid address */
Remove the invalid L0_BLOCK definition. ARMv8 doesn't support block translation in the level 0 descriptor. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
2016-03-18 10:01:25 +00:00
/* 0x1 Level 0 doesn't support block translation */
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
/* 0x2 also marks an invalid address */
#define L0_TABLE 0x3 /* A next-level table */
/* Level 1 table, 1GiB per entry */
#define L1_SHIFT 30
#define L1_SIZE (1 << L1_SHIFT)
#define L1_OFFSET (L1_SIZE - 1)
#define L1_INVAL L0_INVAL
Remove the invalid L0_BLOCK definition. ARMv8 doesn't support block translation in the level 0 descriptor. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
2016-03-18 10:01:25 +00:00
#define L1_BLOCK 0x1
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
#define L1_TABLE L0_TABLE
/* Level 2 table, 2MiB per entry */
#define L2_SHIFT 21
#define L2_SIZE (1 << L2_SHIFT)
#define L2_OFFSET (L2_SIZE - 1)
Remove the invalid L0_BLOCK definition. ARMv8 doesn't support block translation in the level 0 descriptor. Obtained from: ABT Systems Ltd Sponsored by: The FreeBSD Foundation
2016-03-18 10:01:25 +00:00
#define L2_INVAL L1_INVAL
#define L2_BLOCK L1_BLOCK
#define L2_TABLE L1_TABLE
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
#define L2_BLOCK_MASK UINT64_C(0xffffffe00000)
/* Level 3 table, 4KiB per entry */
#define L3_SHIFT 12
#define L3_SIZE (1 << L3_SHIFT)
#define L3_OFFSET (L3_SIZE - 1)
#define L3_INVAL 0x0
/* 0x1 is reserved */
/* 0x2 also marks an invalid address */
#define L3_PAGE 0x3
arm64: Raise again L3 table for early devmap The initial raise in r336519 wasn't enough for using big resolution (1920 x 1200 for example). Raise it again. Reported by: bob prohaska <fbsd@www.zefox.net> Tested by: bob prohaska <fbsd@www.zefox.net> Approved by: re (gjb@)
2018-10-01 14:27:53 +00:00
#define PMAP_MAPDEV_EARLY_SIZE (L2_SIZE * 8)
Raise the size of L3 table for early devmap on arm64 Some driver (like efifb) needs to map more than the current L2_SIZE Raise the size so we can map the framebuffer setup by the bootloader. Reviewed by: cognet
2018-07-19 21:58:06 +00:00
Add support for 4 level pagetables. The userland address space has been increased to 256TiB. The kernel address space can also be increased to be the same size, but this will be performed in a later change. To help work with an extra level of page tables two new functions have been added, one to file the lowest level table entry, and one to find the block/page level. Both of these find the entry for a given pmap and virtual address. This has been tested with a combination of buildworld, stress2 tests, and by using sort to consume a large amount of memory by sorting /dev/zero. No new issues are known to be present from this change. Reviewed by: kib Obtained from: ABT Systems Ltd Relnotes: yes Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D5720
2016-03-31 11:07:24 +00:00
#define L0_ENTRIES_SHIFT 9
#define L0_ENTRIES (1 << L0_ENTRIES_SHIFT)
#define L0_ADDR_MASK (L0_ENTRIES - 1)
#define Ln_ENTRIES_SHIFT 9
#define Ln_ENTRIES (1 << Ln_ENTRIES_SHIFT)
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
#define Ln_ADDR_MASK (Ln_ENTRIES - 1)
#define Ln_TABLE_MASK ((1 << 12) - 1)
Move the pmap_l0_index, etc. macros to pte.h. These will be used by the EFI Runtime Services code. Sponsored by: DARPA, AFRL
2017-10-10 12:54:36 +00:00
#define pmap_l0_index(va) (((va) >> L0_SHIFT) & L0_ADDR_MASK)
#define pmap_l1_index(va) (((va) >> L1_SHIFT) & Ln_ADDR_MASK)
#define pmap_l2_index(va) (((va) >> L2_SHIFT) & Ln_ADDR_MASK)
#define pmap_l3_index(va) (((va) >> L3_SHIFT) & Ln_ADDR_MASK)
Add the start of the arm64 machine headers. This is the subset needed to start getting userland libraries building. Reviewed by: imp Sponsored by: The FreeBSD Foundation
2015-03-23 11:54:56 +00:00
#endif /* !_MACHINE_PTE_H_ */
/* End of pte.h */
Reference in New Issue Copy Permalink