d/freebsd-dev
1
0
Fork 0
Code Issues Pull Requests Packages Projects Releases Wiki Activity
ae5a522cae
freebsd-dev/sys/powerpc/booke/pmap.c

3117 lines
78 KiB
C
Raw Normal View History

Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*-
sys/powerpc: further adoption of SPDX licensing ID tags. Mainly focus on files that use BSD 2-Clause license, however the tool I was using misidentified many licenses so this was mostly a manual - error prone - task. The Software Package Data Exchange (SPDX) group provides a specification to make it easier for automated tools to detect and summarize well known opensource licenses. We are gradually adopting the specification, noting that the tags are considered only advisory and do not, in any way, superceed or replace the license texts.
2017-11-27 15:09:59 +00:00
* SPDX-License-Identifier: BSD-2-Clause-FreeBSD
*
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
* Copyright (C) 2007-2009 Semihalf, Rafal Jaworowski <raj@semihalf.com>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* Copyright (C) 2006 Semihalf, Marian Balakowicz <m8@semihalf.com>
* All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* 1. Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* 2. Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
*
* THIS SOFTWARE IS PROVIDED BY THE AUTHOR ``AS IS'' AND ANY EXPRESS OR
* IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES
* OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN
* NO EVENT SHALL THE AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
* SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
* TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF
* LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING
* NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
* SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*
* Some hw specific parts of this pmap were derived or influenced
* by NetBSD's ibm4xx pmap module. More generic code is shared with
* a few other pmap modules from the FreeBSD tree.
*/
/*
* VM layout notes:
*
* Kernel and user threads run within one common virtual address space
* defined by AS=0.
*
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 32-bit pmap:
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* Virtual address space layout:
* -----------------------------
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 0x0000_0000 - 0x7fff_ffff : user process
* 0x8000_0000 - 0xbfff_ffff : pmap_mapdev()-ed area (PCI/PCIE etc.)
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* 0xc000_0000 - 0xc0ff_ffff : kernel reserved
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
* 0xc000_0000 - data_end : kernel code+data, env, metadata etc.
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* 0xc100_0000 - 0xffff_ffff : KVA
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* 0xc100_0000 - 0xc100_3fff : reserved for page zero/copy
* 0xc100_4000 - 0xc200_3fff : reserved for ptbl bufs
* 0xc200_4000 - 0xc200_8fff : guard page + kstack0
* 0xc200_9000 - 0xfeef_ffff : actual free KVA space
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
*
* 64-bit pmap:
* Virtual address space layout:
* -----------------------------
* 0x0000_0000_0000_0000 - 0xbfff_ffff_ffff_ffff : user process
* 0x0000_0000_0000_0000 - 0x8fff_ffff_ffff_ffff : text, data, heap, maps, libraries
* 0x9000_0000_0000_0000 - 0xafff_ffff_ffff_ffff : mmio region
* 0xb000_0000_0000_0000 - 0xbfff_ffff_ffff_ffff : stack
* 0xc000_0000_0000_0000 - 0xcfff_ffff_ffff_ffff : kernel reserved
* 0xc000_0000_0000_0000 - endkernel-1 : kernel code & data
* endkernel - msgbufp-1 : flat device tree
powerpc/pmap: Simplify Book-E 64-bit page table management There is no need for the 64-bit pmap to have a fixed number of page table buffers. Since the 64-bit pmap has a DMAP, we can effectively have user page tables limited only by total RAM size.
2019-08-06 03:16:06 +00:00
* msgbufp - kernel_pdir-1 : message buffer
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
* kernel_pdir - kernel_pp2d-1 : kernel page directory
* kernel_pp2d - . : kernel pointers to page directory
* pmap_zero_copy_min - crashdumpmap-1 : reserved for page zero/copy
* crashdumpmap - ptbl_buf_pool_vabase-1 : reserved for ptbl bufs
* ptbl_buf_pool_vabase - virtual_avail-1 : user page directories and page tables
* virtual_avail - 0xcfff_ffff_ffff_ffff : actual free KVA space
* 0xd000_0000_0000_0000 - 0xdfff_ffff_ffff_ffff : coprocessor region
* 0xe000_0000_0000_0000 - 0xefff_ffff_ffff_ffff : mmio region
* 0xf000_0000_0000_0000 - 0xffff_ffff_ffff_ffff : direct map
* 0xf000_0000_0000_0000 - +Maxmem : physmem map
* - 0xffff_ffff_ffff_ffff : device direct map
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
#include <sys/cdefs.h>
__FBSDID("$FreeBSD$");
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
#include "opt_ddb.h"
Fix KSTACK_PAGES issue when the default value was changed in KERNCONF If KSTACK_PAGES was changed to anything alse than the default, the value from param.h was taken instead in some places and the value from KENRCONF in some others. This resulted in inconsistency which caused corruption in SMP envorinment. Ensure all places where KSTACK_PAGES are used the opt_kstack_pages.h is included. The file opt_kstack_pages.h could not be included in param.h because was breaking the toolchain compilation. Reviewed by: kib Obtained from: Semihalf Sponsored by: The FreeBSD Foundation Differential Revision: https://reviews.freebsd.org/D3094
2015-07-16 10:46:52 +00:00
#include "opt_kstack_pages.h"
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/param.h>
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
#include <sys/conf.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/malloc.h>
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
#include <sys/ktr.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/proc.h>
#include <sys/user.h>
#include <sys/queue.h>
#include <sys/systm.h>
#include <sys/kernel.h>
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
#include <sys/kerneldump.h>
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
#include <sys/linker.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/msgbuf.h>
#include <sys/lock.h>
#include <sys/mutex.h>
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
#include <sys/rwlock.h>
MFC
2011-05-17 22:03:01 +00:00
#include <sys/sched.h>
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
#include <sys/smp.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <sys/vmmeter.h>
#include <vm/vm.h>
Avoid dump_avail[] redefinition. Move dump_avail[] extern declaration and inlines into a new header vm/vm_dumpset.h. This fixes default gcc build for mips. Reviewed by: alc, scottph Tested by: kevans (previous version) Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D26741
2020-10-14 22:51:40 +00:00
#include <vm/vm_param.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <vm/vm_page.h>
#include <vm/vm_kern.h>
#include <vm/vm_pageout.h>
#include <vm/vm_extern.h>
#include <vm/vm_object.h>
#include <vm/vm_map.h>
#include <vm/vm_pager.h>
Fix powerpc Book-E build post-331018/331048. pagedaemon_wakeup() was moved from vm_pageout.h to vm_pagequeue.h.
2018-03-20 01:07:22 +00:00
#include <vm/vm_phys.h>
#include <vm/vm_pagequeue.h>
Avoid dump_avail[] redefinition. Move dump_avail[] extern declaration and inlines into a new header vm/vm_dumpset.h. This fixes default gcc build for mips. Reviewed by: alc, scottph Tested by: kevans (previous version) Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D26741
2020-10-14 22:51:40 +00:00
#include <vm/vm_dumpset.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <vm/uma.h>
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#include <machine/_inttypes.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <machine/cpu.h>
#include <machine/pcb.h>
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
#include <machine/platform.h>
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#include <machine/tlb.h>
#include <machine/spr.h>
#include <machine/md_var.h>
#include <machine/mmuvar.h>
#include <machine/pmap.h>
#include <machine/pte.h>
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
#include <ddb/ddb.h>
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#define SPARSE_MAPDEV
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
/* Use power-of-two mappings in mmu_booke_mapdev(), to save entries. */
#define POW2_MAPPINGS
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#ifdef DEBUG
#define debugf(fmt, args...) printf(fmt, ##args)
#else
#define debugf(fmt, args...)
#endif
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
#define PRI0ptrX "016lx"
#else
#define PRI0ptrX "08x"
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
#define TODO panic("%s: not implemented", __func__);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
extern unsigned char _etext[];
extern unsigned char _end[];
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
extern uint32_t *bootinfo;
vm_paddr_t kernload;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
vm_offset_t kernstart;
vm_size_t kernsize;
/* Message buffer and tables. */
static vm_offset_t data_start;
static vm_size_t data_end;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
/* Phys/avail memory regions. */
static struct mem_region *availmem_regions;
static int availmem_regions_sz;
static struct mem_region *physmem_regions;
static int physmem_regions_sz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Clean up pmap a little for 64-bit 64-bit Book-E pmap doesn't need copy and zero bounce pages, nor the mutex. Don't initialize them or reserve space for them.
2019-08-25 20:11:35 +00:00
#ifndef __powerpc64__
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Reserved KVA space and mutex for mmu_booke_zero_page. */
static vm_offset_t zero_page_va;
static struct mtx zero_page_mutex;
/* Reserved KVA space and mutex for mmu_booke_copy_page. */
static vm_offset_t copy_page_src_va;
static vm_offset_t copy_page_dst_va;
static struct mtx copy_page_mutex;
powerpc/booke: Clean up pmap a little for 64-bit 64-bit Book-E pmap doesn't need copy and zero bounce pages, nor the mutex. Don't initialize them or reserve space for them.
2019-08-25 20:11:35 +00:00
#endif
static struct mtx tlbivax_mutex;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* PMAP */
/**************************************************************************/
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static int mmu_booke_enter_locked(pmap_t, vm_offset_t, vm_page_t,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t, u_int flags, int8_t psind);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
unsigned int kptbl_min; /* Index of the first kernel ptbl. */
powerpc: Unify pmap definitions between AIM and Book-E This is part 2 of r347078, pulling the page directory out of the Book-E pmap. This breaks KBI for anything that uses struct pmap (such as vm_map) so any modules that access this must be rebuilt.
2019-08-12 03:03:56 +00:00
static uma_zone_t ptbl_root_zone;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* If user pmap is processed with mmu_booke_remove and the resident count
* drops to 0, there are no more pages to remove, so we need not continue.
*/
#define PMAP_REMOVE_DONE(pmap) \
((pmap) != kernel_pmap && (pmap)->pm_stats.resident_count == 0)
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#if defined(COMPAT_FREEBSD32) || !defined(__powerpc64__)
Missed local diff.
2015-03-01 21:47:38 +00:00
extern int elf32_nxstack;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* TLB and TID handling */
/**************************************************************************/
/* Translation ID busy table */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static volatile pmap_t tidbusy[MAXCPU][TID_MAX + 1];
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* TLB0 capabilities (entry, way numbers etc.). These can vary between e500
* core revisions and should be read from h/w registers during early config.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t tlb0_entries;
uint32_t tlb0_ways;
uint32_t tlb0_entries_per_way;
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
uint32_t tlb1_entries;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
#define TLB0_ENTRIES (tlb0_entries)
#define TLB0_WAYS (tlb0_ways)
#define TLB0_ENTRIES_PER_WAY (tlb0_entries_per_way)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
#define TLB1_ENTRIES (tlb1_entries)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static tlbtid_t tid_alloc(struct pmap *);
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
#ifdef DDB
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#ifdef __powerpc64__
static void tlb_print_entry(int, uint32_t, uint64_t, uint32_t, uint32_t);
#else
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static void tlb_print_entry(int, uint32_t, uint32_t, uint32_t, uint32_t);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
static void tlb1_read_entry(tlb_entry_t *, unsigned int);
static void tlb1_write_entry(tlb_entry_t *, unsigned int);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static int tlb1_iomapped(int, vm_paddr_t, vm_size_t, vm_offset_t *);
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
static vm_size_t tlb1_mapin_region(vm_offset_t, vm_paddr_t, vm_size_t, int);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
static __inline uint32_t tlb_calc_wimg(vm_paddr_t pa, vm_memattr_t ma);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static vm_size_t tsize2size(unsigned int);
static unsigned int size2tsize(vm_size_t);
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
static unsigned long ilog2(unsigned long);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void set_mas4_defaults(void);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static inline void tlb0_flush_entry(vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static inline unsigned int tlb0_tableidx(vm_offset_t, unsigned int);
/**************************************************************************/
/* Page table management */
/**************************************************************************/
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
static struct rwlock_padalign pvh_global_lock;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Data for the pv entry allocation mechanism */
static uma_zone_t pvzone;
static int pv_entry_count = 0, pv_entry_max = 0, pv_entry_high_water = 0;
#define PV_ENTRY_ZONE_MIN 2048 /* min pv entries in uma zone */
#ifndef PMAP_SHPGPERPROC
#define PMAP_SHPGPERPROC 200
#endif
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static vm_paddr_t pte_vatopa(pmap_t, vm_offset_t);
static int pte_enter(pmap_t, vm_page_t, vm_offset_t, uint32_t, boolean_t);
static int pte_remove(pmap_t, vm_offset_t, uint8_t);
static pte_t *pte_find(pmap_t, vm_offset_t);
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
static void kernel_pte_alloc(vm_offset_t, vm_offset_t);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static pv_entry_t pv_alloc(void);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void pv_free(pv_entry_t);
static void pv_insert(pmap_t, vm_offset_t, vm_page_t);
static void pv_remove(pmap_t, vm_offset_t, vm_page_t);
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static void booke_pmap_init_qpages(void);
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
static inline void tlb_miss_lock(void);
static inline void tlb_miss_unlock(void);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#ifdef SMP
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
extern tlb_entry_t __boot_tlb1[];
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
void pmap_bootstrap_ap(volatile uint32_t *);
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#endif
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Kernel MMU interface
*/
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_clear_modify(vm_page_t);
static void mmu_booke_copy(pmap_t, pmap_t, vm_offset_t,
Fix Book-E/MPC85XX build. Some prototypes were wrong and got revealed with the recent kobj signature checking.
2009-06-13 08:57:04 +00:00
vm_size_t, vm_offset_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_copy_page(vm_page_t, vm_page_t);
static void mmu_booke_copy_pages(vm_page_t *,
Add pmap function pmap_copy_pages(), which copies the content of the pages around, taking array of vm_page_t both for source and destination. Starting offsets and total transfer size are specified. The function implements optimal algorithm for copying using the platform-specific optimizations. For instance, on the architectures were the direct map is available, no transient mappings are created, for i386 the per-cpu ephemeral page frame is used. The code was typically borrowed from the pmap_copy_page() for the same architecture. Only i386/amd64, powerpc aim and arm/arm-v6 implementations were tested at the time of commit. High-level code, not committed yet to the tree, ensures that the use of the function is only allowed after explicit enablement. For sparc64, the existing code has known issues and a stab is added instead, to allow the kernel linking. Sponsored by: The FreeBSD Foundation Tested by: pho (i386, amd64), scottl (amd64), ian (arm and arm-v6) MFC after: 2 weeks
2013-03-14 20:18:12 +00:00
vm_offset_t, vm_page_t *, vm_offset_t, int);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static int mmu_booke_enter(pmap_t, vm_offset_t, vm_page_t,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t, u_int flags, int8_t psind);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_enter_object(pmap_t, vm_offset_t, vm_offset_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_page_t, vm_prot_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_enter_quick(pmap_t, vm_offset_t, vm_page_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static vm_paddr_t mmu_booke_extract(pmap_t, vm_offset_t);
static vm_page_t mmu_booke_extract_and_hold(pmap_t, vm_offset_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_init(void);
static boolean_t mmu_booke_is_modified(vm_page_t);
static boolean_t mmu_booke_is_prefaultable(pmap_t, vm_offset_t);
static boolean_t mmu_booke_is_referenced(vm_page_t);
static int mmu_booke_ts_referenced(vm_page_t);
static vm_offset_t mmu_booke_map(vm_offset_t *, vm_paddr_t, vm_paddr_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static int mmu_booke_mincore(pmap_t, vm_offset_t,
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
vm_paddr_t *);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_object_init_pt(pmap_t, vm_offset_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_object_t, vm_pindex_t, vm_size_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static boolean_t mmu_booke_page_exists_quick(pmap_t, vm_page_t);
static void mmu_booke_page_init(vm_page_t);
static int mmu_booke_page_wired_mappings(vm_page_t);
static int mmu_booke_pinit(pmap_t);
static void mmu_booke_pinit0(pmap_t);
static void mmu_booke_protect(pmap_t, vm_offset_t, vm_offset_t,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_qenter(vm_offset_t, vm_page_t *, int);
static void mmu_booke_qremove(vm_offset_t, int);
static void mmu_booke_release(pmap_t);
static void mmu_booke_remove(pmap_t, vm_offset_t, vm_offset_t);
static void mmu_booke_remove_all(vm_page_t);
static void mmu_booke_remove_write(vm_page_t);
static void mmu_booke_unwire(pmap_t, vm_offset_t, vm_offset_t);
static void mmu_booke_zero_page(vm_page_t);
static void mmu_booke_zero_page_area(vm_page_t, int, int);
static void mmu_booke_activate(struct thread *);
static void mmu_booke_deactivate(struct thread *);
static void mmu_booke_bootstrap(vm_offset_t, vm_offset_t);
static void *mmu_booke_mapdev(vm_paddr_t, vm_size_t);
static void *mmu_booke_mapdev_attr(vm_paddr_t, vm_size_t, vm_memattr_t);
static void mmu_booke_unmapdev(vm_offset_t, vm_size_t);
static vm_paddr_t mmu_booke_kextract(vm_offset_t);
static void mmu_booke_kenter(vm_offset_t, vm_paddr_t);
static void mmu_booke_kenter_attr(vm_offset_t, vm_paddr_t, vm_memattr_t);
static void mmu_booke_kremove(vm_offset_t);
static boolean_t mmu_booke_dev_direct_mapped(vm_paddr_t, vm_size_t);
static void mmu_booke_sync_icache(pmap_t, vm_offset_t,
o Introduce vm_sync_icache() for making the I-cache coherent with the memory or D-cache, depending on the semantics of the platform. vm_sync_icache() is basically a wrapper around pmap_sync_icache(), that translates the vm_map_t argumument to pmap_t. o Introduce pmap_sync_icache() to all PMAP implementation. For powerpc it replaces the pmap_page_executable() function, added to solve the I-cache problem in uiomove_fromphys(). o In proc_rwmem() call vm_sync_icache() when writing to a page that has execute permissions. This assures that when breakpoints are written, the I-cache will be coherent and the process will actually hit the breakpoint. o This also fixes the Book-E PMAP implementation that was missing necessary locking while trying to deal with the I-cache coherency in pmap_enter() (read: mmu_booke_enter_locked). The key property of this change is that the I-cache is made coherent *after* writes have been done. Doing it in the PMAP layer when adding or changing a mapping means that the I-cache is made coherent *before* any writes happen. The difference is key when the I-cache prefetches.
2009-10-21 18:38:02 +00:00
vm_size_t);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_dumpsys_map(vm_paddr_t pa, size_t,
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
void **);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_dumpsys_unmap(vm_paddr_t pa, size_t,
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
void *);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_scan_init(void);
static vm_offset_t mmu_booke_quick_enter_page(vm_page_t m);
static void mmu_booke_quick_remove_page(vm_offset_t addr);
static int mmu_booke_change_attr(vm_offset_t addr,
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
vm_size_t sz, vm_memattr_t mode);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static int mmu_booke_decode_kernel_ptr(vm_offset_t addr,
Remove hard-coded trap-handling logic involving the segmented memory model used with hashed page tables on AIM and place it into a new, modular pmap function called pmap_decode_kernel_ptr(). This function is the inverse of pmap_map_user_ptr(). With POWER9 radix tables, which mapping to use becomes more complex than just AIM/BOOKE and it is best to have it in the same place as pmap_map_user_ptr(). Reviewed by: jhibbits
2018-01-29 04:33:41 +00:00
int *is_user, vm_offset_t *decoded_addr);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static void mmu_booke_page_array_startup(long);
static boolean_t mmu_booke_page_is_mapped(vm_page_t m);
Clean up a couple of MD warts in vm_fault_populate(): --Eliminate a big ifdef that encompassed all currently-supported architectures except mips and powerpc32. This applied to the case in which we've allocated a superpage but the pager-populated range is insufficient for a superpage mapping. For platforms that don't support superpages the check should be inexpensive as we shouldn't get a superpage in the first place. Make the normal-page fallback logic identical for all platforms and provide a simple implementation of pmap_ps_enabled() for MIPS and Book-E/AIM32 powerpc. --Apply the logic for handling pmap_enter() failure if a superpage mapping can't be supported due to additional protection policy. Use KERN_PROTECTION_FAILURE instead of KERN_FAILURE for this case, and note Intel PKU on amd64 as the first example of such protection policy. Reviewed by: kib, markj, bdragon Differential Revision: https://reviews.freebsd.org/D29439
2021-03-27 03:10:46 +00:00
static bool mmu_booke_ps_enabled(pmap_t pmap);
Move the pmap-specific code in copyinout.c that gets pointers to userland buffers into a new pmap-module function pmap_map_user_ptr() that can be implemented by the respective modules. This is required to implement non-segment-based AIM-ish MMU systems such as the radix-tree page tables introduced by POWER ISA 3.0 and present on POWER9. Reviewed by: jhibbits
2018-01-15 06:46:33 +00:00
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
static struct pmap_funcs mmu_booke_methods = {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* pmap dispatcher interface */
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
.clear_modify = mmu_booke_clear_modify,
.copy = mmu_booke_copy,
.copy_page = mmu_booke_copy_page,
.copy_pages = mmu_booke_copy_pages,
.enter = mmu_booke_enter,
.enter_object = mmu_booke_enter_object,
.enter_quick = mmu_booke_enter_quick,
.extract = mmu_booke_extract,
.extract_and_hold = mmu_booke_extract_and_hold,
.init = mmu_booke_init,
.is_modified = mmu_booke_is_modified,
.is_prefaultable = mmu_booke_is_prefaultable,
.is_referenced = mmu_booke_is_referenced,
.ts_referenced = mmu_booke_ts_referenced,
.map = mmu_booke_map,
.mincore = mmu_booke_mincore,
.object_init_pt = mmu_booke_object_init_pt,
.page_exists_quick = mmu_booke_page_exists_quick,
.page_init = mmu_booke_page_init,
.page_wired_mappings = mmu_booke_page_wired_mappings,
.pinit = mmu_booke_pinit,
.pinit0 = mmu_booke_pinit0,
.protect = mmu_booke_protect,
.qenter = mmu_booke_qenter,
.qremove = mmu_booke_qremove,
.release = mmu_booke_release,
.remove = mmu_booke_remove,
.remove_all = mmu_booke_remove_all,
.remove_write = mmu_booke_remove_write,
.sync_icache = mmu_booke_sync_icache,
.unwire = mmu_booke_unwire,
.zero_page = mmu_booke_zero_page,
.zero_page_area = mmu_booke_zero_page_area,
.activate = mmu_booke_activate,
.deactivate = mmu_booke_deactivate,
.quick_enter_page = mmu_booke_quick_enter_page,
.quick_remove_page = mmu_booke_quick_remove_page,
.page_array_startup = mmu_booke_page_array_startup,
.page_is_mapped = mmu_booke_page_is_mapped,
Clean up a couple of MD warts in vm_fault_populate(): --Eliminate a big ifdef that encompassed all currently-supported architectures except mips and powerpc32. This applied to the case in which we've allocated a superpage but the pager-populated range is insufficient for a superpage mapping. For platforms that don't support superpages the check should be inexpensive as we shouldn't get a superpage in the first place. Make the normal-page fallback logic identical for all platforms and provide a simple implementation of pmap_ps_enabled() for MIPS and Book-E/AIM32 powerpc. --Apply the logic for handling pmap_enter() failure if a superpage mapping can't be supported due to additional protection policy. Use KERN_PROTECTION_FAILURE instead of KERN_FAILURE for this case, and note Intel PKU on amd64 as the first example of such protection policy. Reviewed by: kib, markj, bdragon Differential Revision: https://reviews.freebsd.org/D29439
2021-03-27 03:10:46 +00:00
.ps_enabled = mmu_booke_ps_enabled,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Internal interfaces */
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
.bootstrap = mmu_booke_bootstrap,
.dev_direct_mapped = mmu_booke_dev_direct_mapped,
.mapdev = mmu_booke_mapdev,
.mapdev_attr = mmu_booke_mapdev_attr,
.kenter = mmu_booke_kenter,
.kenter_attr = mmu_booke_kenter_attr,
.kextract = mmu_booke_kextract,
.kremove = mmu_booke_kremove,
.unmapdev = mmu_booke_unmapdev,
.change_attr = mmu_booke_change_attr,
.decode_kernel_ptr = mmu_booke_decode_kernel_ptr,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
/* dumpsys() support */
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
.dumpsys_map_chunk = mmu_booke_dumpsys_map,
.dumpsys_unmap_chunk = mmu_booke_dumpsys_unmap,
.dumpsys_pa_init = mmu_booke_scan_init,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
};
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
MMU_DEF(booke_mmu, MMU_TYPE_BOOKE, mmu_booke_methods);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
#ifdef __powerpc64__
#include "pmap_64.c"
#else
#include "pmap_32.c"
#endif
static vm_offset_t tlb1_map_base = VM_MAPDEV_BASE;
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
static __inline uint32_t
Use the correct type for physical addresses. On Book-E, physical addresses are actually 36-bits, not 32-bits. This is currently worked around by ignoring the top bits. However, in some cases, the boot loader configures CCSR to something above the 32-bit mark. This is stage 1 in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
tlb_calc_wimg(vm_paddr_t pa, vm_memattr_t ma)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
uint32_t attrib;
int i;
if (ma != VM_MEMATTR_DEFAULT) {
switch (ma) {
case VM_MEMATTR_UNCACHEABLE:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_I | MAS2_G);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
case VM_MEMATTR_WRITE_COMBINING:
case VM_MEMATTR_WRITE_BACK:
case VM_MEMATTR_PREFETCHABLE:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_I);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
case VM_MEMATTR_WRITE_THROUGH:
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
return (MAS2_W | MAS2_M);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
case VM_MEMATTR_CACHEABLE:
return (MAS2_M);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
}
}
/*
* Assume the page is cache inhibited and access is guarded unless
* it's in our available memory array.
*/
attrib = _TLB_ENTRY_IO;
for (i = 0; i < physmem_regions_sz; i++) {
if ((pa >= physmem_regions[i].mr_start) &&
(pa < (physmem_regions[i].mr_start +
physmem_regions[i].mr_size))) {
attrib = _TLB_ENTRY_MEM;
break;
}
}
return (attrib);
}
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
static inline void
tlb_miss_lock(void)
{
#ifdef SMP
struct pcpu *pc;
if (!smp_started)
return;
On multi-core, multi-threaded PPC systems, it is important that the threads be brought up in the order they are enumerated in the device tree (in particular, that thread 0 on each core be brought up first). The SLIST through which we loop to start the CPUs has all of its entries added with SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration and so AP startup would always fail in such situations (causing a machine check or RTAS failure). Fix this by changing the SLIST into an STAILQ, and inserting new CPUs at the end. Reviewed by: jhb
2011-05-31 15:11:43 +00:00
STAILQ_FOREACH(pc, &cpuhead, pc_allcpu) {
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
if (pc != pcpup) {
CTR3(KTR_PMAP, "%s: tlb miss LOCK of CPU=%d, "
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
"tlb_lock=%p", __func__, pc->pc_cpuid, pc->pc_booke.tlb_lock);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
KASSERT((pc->pc_cpuid != PCPU_GET(cpuid)),
("tlb_miss_lock: tried to lock self"));
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
tlb_lock(pc->pc_booke.tlb_lock);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
CTR1(KTR_PMAP, "%s: locked", __func__);
}
}
#endif
}
static inline void
tlb_miss_unlock(void)
{
#ifdef SMP
struct pcpu *pc;
if (!smp_started)
return;
On multi-core, multi-threaded PPC systems, it is important that the threads be brought up in the order they are enumerated in the device tree (in particular, that thread 0 on each core be brought up first). The SLIST through which we loop to start the CPUs has all of its entries added with SLIST_INSERT_HEAD(), which means it is in reverse order of enumeration and so AP startup would always fail in such situations (causing a machine check or RTAS failure). Fix this by changing the SLIST into an STAILQ, and inserting new CPUs at the end. Reviewed by: jhb
2011-05-31 15:11:43 +00:00
STAILQ_FOREACH(pc, &cpuhead, pc_allcpu) {
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
if (pc != pcpup) {
CTR2(KTR_PMAP, "%s: tlb miss UNLOCK of CPU=%d",
__func__, pc->pc_cpuid);
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
tlb_unlock(pc->pc_booke.tlb_lock);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
CTR1(KTR_PMAP, "%s: unlocked", __func__);
}
}
#endif
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Return number of entries in TLB0. */
static __inline void
tlb0_get_tlbconf(void)
{
uint32_t tlb0_cfg;
tlb0_cfg = mfspr(SPR_TLB0CFG);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_entries = tlb0_cfg & TLBCFG_NENTRY_MASK;
tlb0_ways = (tlb0_cfg & TLBCFG_ASSOC_MASK) >> TLBCFG_ASSOC_SHIFT;
tlb0_entries_per_way = tlb0_entries / tlb0_ways;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* Return number of entries in TLB1. */
static __inline void
tlb1_get_tlbconf(void)
{
uint32_t tlb1_cfg;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
tlb1_cfg = mfspr(SPR_TLB1CFG);
tlb1_entries = tlb1_cfg & TLBCFG_NENTRY_MASK;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/**************************************************************************/
/* Page table related */
/**************************************************************************/
powerpc/booke: Do as much work outside of TLB locks as possible Reduce the surface area of the TLB locks. Unfortunately the same trick for serializing the tlbie instruction on OEA64 cannot be used here to reduce the scope of the tlbivax mutex to the tlbsync only, as the mutex also serializes the TLB miss lock as a side effect, so contention on this lock may not be reducible any further.
2019-05-08 16:05:18 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* Allocate pv_entry structure. */
pv_entry_t
pv_alloc(void)
{
pv_entry_t pv;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
pv_entry_count++;
if (pv_entry_count > pv_entry_high_water)
pagedaemon_wakeup(0); /* XXX powerpc NUMA */
pv = uma_zalloc(pvzone, M_NOWAIT);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
return (pv);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* Free pv_entry structure. */
static __inline void
pv_free(pv_entry_t pve)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
pv_entry_count--;
uma_zfree(pvzone, pve);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* Allocate and initialize pv_entry structure. */
static void
pv_insert(pmap_t pmap, vm_offset_t va, vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
pv_entry_t pve;
//int su = (pmap == kernel_pmap);
//debugf("pv_insert: s (su = %d pmap = 0x%08x va = 0x%08x m = 0x%08x)\n", su,
// (u_int32_t)pmap, va, (u_int32_t)m);
pve = pv_alloc();
if (pve == NULL)
panic("pv_insert: no pv entries!");
pve->pv_pmap = pmap;
pve->pv_va = va;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* add to pv_list */
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
rw_assert(&pvh_global_lock, RA_WLOCKED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
TAILQ_INSERT_TAIL(&m->md.pv_list, pve, pv_link);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
//debugf("pv_insert: e\n");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* Destroy pv entry. */
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
static void
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
pv_remove(pmap_t pmap, vm_offset_t va, vm_page_t m)
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
{
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
pv_entry_t pve;
//int su = (pmap == kernel_pmap);
//debugf("pv_remove: s (su = %d pmap = 0x%08x va = 0x%08x)\n", su, (u_int32_t)pmap, va);
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
rw_assert(&pvh_global_lock, RA_WLOCKED);
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
/* find pv entry */
TAILQ_FOREACH(pve, &m->md.pv_list, pv_link) {
if ((pmap == pve->pv_pmap) && (va == pve->pv_va)) {
/* remove from pv_list */
TAILQ_REMOVE(&m->md.pv_list, pve, pv_link);
if (TAILQ_EMPTY(&m->md.pv_list))
vm_page_aflag_clear(m, PGA_WRITEABLE);
/* free pv entry struct */
pv_free(pve);
break;
}
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
}
powerpc/booke: Split out 32- and 64- bit pmap details from main body Summary: This is largely a straight-forward cleave of the 32-bit and 64-bit page table specifics, along with the mmu_booke_*() functions that are wholely different between the two implementations. The ultimate goal of this is to make it easier to reason about and update a specific implementation without wading through the other implementation details. This is in support of further changes to the 64-bit pmap. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D23983
2020-03-10 03:30:11 +00:00
//debugf("pv_remove: e\n");
Hide most of the PTE initialization and management. By confining the page table management to a handful of functions it'll be easier to modify the page table scheme without affecting other functions. This will be necessary when 64-bit support is added, and page tables become much larger.
2016-01-19 03:07:25 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* PMAP related */
/**************************************************************************/
/*
Better support different kernel hand-offs. When loaded directly from U-Boot, the kernel is passed a standard argc/argv pair. The Juniper loader passes the metadata pointer as the second argument and passes 0 in the first. The FreeBSD loader passes the metadata pointer in the first argument. As such, have locore preserve the first 2 arguments in registers r30 & r31. Change e500_init() to accept these arguments. Don't pass global offsets (i.e. kernel_text and _end) as arguments to e500_init(). We can reference those directly. Rename e500_init() to booke_init() now that we're changing the prototype. In booke_init(), "decode" arg1 and arg2 to obtain the metadata pointer correctly. For the U-Boot case, clear SBSS and BSS and bank on having a static FDT for now. This allows loading the ELF kernel and jumping to the entry point without trampoline.
2011-05-28 04:10:44 +00:00
* This is called during booke_init, before the system is really initialized.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_bootstrap(vm_offset_t start, vm_offset_t kernelend)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_paddr_t phys_kernelend;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
struct mem_region *mp, *mp1;
int cnt, i, j;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_paddr_t s, e, sz;
vm_paddr_t physsz, hwphyssz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
u_int phys_avail_count;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
vm_size_t kstack0_sz;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
vm_paddr_t kstack0_phys;
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
vm_offset_t kstack0;
Implement a facility for dynamic per-cpu variables. - Modules and kernel code alike may use DPCPU_DEFINE(), DPCPU_GET(), DPCPU_SET(), etc. akin to the statically defined PCPU_*. Requires only one extra instruction more than PCPU_* and is virtually the same as __thread for builtin and much faster for shared objects. DPCPU variables can be initialized when defined. - Modules are supported by relocating the module's per-cpu linker set over space reserved in the kernel. Modules may fail to load if there is insufficient space available. - Track space available for modules with a one-off extent allocator. Free may block for memory to allocate space for an extent. Reviewed by: jhb, rwatson, kan, sam, grehan, marius, marcel, stas
2009-06-23 22:42:39 +00:00
void *dpcpu;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("mmu_booke_bootstrap: entered\n");
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
/* Set interesting system properties */
Add support for a pmap direct map for 64-bit Book-E As with AIM64, map the DMAP at the beginning of the fourth "quadrant" of memory, and move the KERNBASE to the the start of KVA. Eventually we may run the kernel out of the DMAP, but for now, continue booting as it has been.
2018-04-03 00:45:38 +00:00
#ifdef __powerpc64__
hw_direct_map = 1;
#else
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
hw_direct_map = 0;
Add support for a pmap direct map for 64-bit Book-E As with AIM64, map the DMAP at the beginning of the fourth "quadrant" of memory, and move the KERNBASE to the the start of KVA. Eventually we may run the kernel out of the DMAP, but for now, continue booting as it has been.
2018-04-03 00:45:38 +00:00
#endif
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#if defined(COMPAT_FREEBSD32) || !defined(__powerpc64__)
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
elf32_nxstack = 1;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Initialize invalidation mutex */
mtx_init(&tlbivax_mutex, "tlbivax", NULL, MTX_SPIN);
/* Read TLB0 size and associativity. */
tlb0_get_tlbconf();
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
/*
* Align kernel start and end address (kernel image).
* Note that kernel end does not necessarily relate to kernsize.
* kernsize is the size of the kernel that is actually mapped.
*/
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_start = round_page(kernelend);
data_end = data_start;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
/* Allocate the dynamic per-cpu area. */
dpcpu = (void *)data_end;
data_end += DPCPU_SIZE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate space for the message buffer. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
msgbufp = (struct msgbuf *)data_end;
Make MSGBUF_SIZE kernel option a loader tunable kern.msgbufsize. Submitted by: perryh pluto.rain.com (previous version) Reviewed by: jhb Approved by: kib (mentor) Tested by: universe
2011-01-21 10:26:26 +00:00
data_end += msgbufsize;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" msgbufp at 0x%"PRI0ptrX" end = 0x%"PRI0ptrX"\n",
(uintptr_t)msgbufp, data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
data_end = round_page(data_end);
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
data_end = round_page(mmu_booke_alloc_kernel_pgtables(data_end));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/pmap: NUMA-ize vm_page_array on powerpc Summary: This matches r351198 from amd64. This only applies to AIM64 and Book-E. On AIM64 it short-circuits with one domain, to behave similar to existing. Otherwise it will allocate 16MB huge pages to hold the page array, across all NUMA domains. On the first domain it will shift the page array base up, to "upper-align" the page array in that domain, so as to reduce the number of pages from the next domain appearing in this domain. After the first domain, subsequent domains will be allocated in full 16MB pages, until the final domain, which can be short. This means some inner domains may have pages accounted in earlier domains. On Book-E the page array is setup at MMU bootstrap time so that it's always mapped in TLB1, on both 32-bit and 64-bit. This reduces the TLB0 overhead for touching the vm_page_array, which reduces up to one TLB miss per array access. Since page_range (vm_page_startup()) is no longer used on Book-E but is on 32-bit AIM, mark the variable as potentially unused, rather than using a nasty #if defined() list. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21449
2019-12-07 03:34:03 +00:00
/* Retrieve phys/avail mem regions */
mem_regions(&physmem_regions, &physmem_regions_sz,
&availmem_regions, &availmem_regions_sz);
if (PHYS_AVAIL_ENTRIES < availmem_regions_sz)
panic("mmu_booke_bootstrap: phys_avail too small");
data_end = round_page(data_end);
vm_page_array = (vm_page_t)data_end;
/*
* Get a rough idea (upper bound) on the size of the page array. The
* vm_page_array will not handle any more pages than we have in the
* avail_regions array, and most likely much less.
*/
sz = 0;
for (mp = availmem_regions; mp->mr_size; mp++) {
sz += mp->mr_size;
}
sz = (round_page(sz) / (PAGE_SIZE + sizeof(struct vm_page)));
data_end += round_page(sz * sizeof(struct vm_page));
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
/* Pre-round up to 1MB. This wastes some space, but saves TLB entries */
data_end = roundup2(data_end, 1 << 20);
powerpc/pmap: NUMA-ize vm_page_array on powerpc Summary: This matches r351198 from amd64. This only applies to AIM64 and Book-E. On AIM64 it short-circuits with one domain, to behave similar to existing. Otherwise it will allocate 16MB huge pages to hold the page array, across all NUMA domains. On the first domain it will shift the page array base up, to "upper-align" the page array in that domain, so as to reduce the number of pages from the next domain appearing in this domain. After the first domain, subsequent domains will be allocated in full 16MB pages, until the final domain, which can be short. This means some inner domains may have pages accounted in earlier domains. On Book-E the page array is setup at MMU bootstrap time so that it's always mapped in TLB1, on both 32-bit and 64-bit. This reduces the TLB0 overhead for touching the vm_page_array, which reduces up to one TLB miss per array access. Since page_range (vm_page_startup()) is no longer used on Book-E but is on 32-bit AIM, mark the variable as potentially unused, rather than using a nasty #if defined() list. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21449
2019-12-07 03:34:03 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" data_end: 0x%"PRI0ptrX"\n", data_end);
powerpc/booke: Fix some formatting errors in debug printfs Use the right formats for the types given (vm_offset_t and vm_size_t are both uint32_t on 32-bit platforms, and uint64_t on 64-bit platforms, and match size_t in size, so we can use the size_t format as we do in other similar code). These were found by clang.
2019-12-04 03:51:30 +00:00
debugf(" kernstart: %#zx\n", kernstart);
debugf(" kernsize: %#zx\n", kernsize);
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
if (data_end - kernstart > kernsize) {
kernsize += tlb1_mapin_region(kernstart + kernsize,
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
kernload + kernsize, (data_end - kernstart) - kernsize,
_TLB_ENTRY_MEM);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
}
data_end = kernstart + kernsize;
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf(" updated data_end: 0x%"PRI0ptrX"\n", data_end);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
/*
* Clear the structures - note we can only do it safely after the
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* possible additional TLB1 translations are in place (above) so that
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
* all range up to the currently calculated 'data_end' is covered.
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
*/
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
bzero((void *)data_start, data_end - data_start);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
dpcpu_init(dpcpu, 0);
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Set the start and end of kva. */
/*******************************************************/
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
virtual_avail = round_page(data_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
virtual_end = VM_MAX_KERNEL_ADDRESS;
powerpc/booke: Clean up pmap a little for 64-bit 64-bit Book-E pmap doesn't need copy and zero bounce pages, nor the mutex. Don't initialize them or reserve space for them.
2019-08-25 20:11:35 +00:00
#ifndef __powerpc64__
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Allocate KVA space for page zero/copy operations. */
zero_page_va = virtual_avail;
virtual_avail += PAGE_SIZE;
copy_page_src_va = virtual_avail;
virtual_avail += PAGE_SIZE;
copy_page_dst_va = virtual_avail;
virtual_avail += PAGE_SIZE;
Print the full-width pointer values in hex. PRI0ptrX is used to print a zero-padded hex value of the architecture's bitness, so on 64-bit architectures it'll print the full 64 bit address.
2018-05-28 00:19:08 +00:00
debugf("zero_page_va = 0x%"PRI0ptrX"\n", zero_page_va);
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
debugf("copy_page_src_va = 0x%"PRI0ptrX"\n", copy_page_src_va);
debugf("copy_page_dst_va = 0x%"PRI0ptrX"\n", copy_page_dst_va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Initialize page zero/copy mutexes. */
mtx_init(&zero_page_mutex, "mmu_booke_zero_page", NULL, MTX_DEF);
mtx_init(&copy_page_mutex, "mmu_booke_copy_page", NULL, MTX_DEF);
/* Allocate KVA space for ptbl bufs. */
ptbl_buf_pool_vabase = virtual_avail;
virtual_avail += PTBL_BUFS * PTBL_PAGES * PAGE_SIZE;
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
debugf("ptbl_buf_pool_vabase = 0x%"PRI0ptrX" end = 0x%"PRI0ptrX"\n",
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
ptbl_buf_pool_vabase, virtual_avail);
powerpc/pmap: Simplify Book-E 64-bit page table management There is no need for the 64-bit pmap to have a fixed number of page table buffers. Since the 64-bit pmap has a DMAP, we can effectively have user page tables limited only by total RAM size.
2019-08-06 03:16:06 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Calculate corresponding physical addresses for the kernel region. */
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
phys_kernelend = kernload + kernsize;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("kernel image and allocated data:\n");
Add support for building Book-E kernels with clang/lld. This involved several changes: * Since lld does not like text relocations, replace SMP boot page text relocs in booke/locore.S with position-independent math, and track the virtual base in the SMP boot page header. * As some SPRs are interpreted differently on clang due to the way it handles platform-specific SPRs, switch m*dear and m*esr mnemonics out for regular m*spr. Add both forms of SPR_DEAR to spr.h so the correct encoding is selected. * Change some hardcoded 32 bit things in the boot page to be pointer-sized, and fix alignment. * Fix 64-bit build of booke/pmap.c when enabling pmap debugging. Additionally, I took the opportunity to document how the SMP boot page works. Approved by: jhibbits (mentor) Differential Revision: https://reviews.freebsd.org/D21999
2019-11-02 21:15:56 +00:00
debugf(" kernload = 0x%09jx\n", (uintmax_t)kernload);
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
debugf(" kernstart = 0x%"PRI0ptrX"\n", kernstart);
debugf(" kernsize = 0x%"PRI0ptrX"\n", kernsize);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Remove kernel physical address range from avail regions list. Page
* align all regions. Non-page aligned memory isn't very interesting
* to us. Also, sort the entries for ascending addresses.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Factor out platform dependent things unrelated to device drivers into a new platform module. These are probed in early boot, and have the responsibility of determining the layout of physical memory, determining the CPU timebase frequency, and handling the zoo of SMP mechanisms found on PowerPC. Reviewed by: marcel, raj Book-E parts by: raj
2009-05-14 00:34:26 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
sz = 0;
cnt = availmem_regions_sz;
debugf("processing avail regions:\n");
for (mp = availmem_regions; mp->mr_size; mp++) {
s = mp->mr_start;
e = mp->mr_start + mp->mr_size;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
debugf(" %09jx-%09jx -> ", (uintmax_t)s, (uintmax_t)e);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Check whether this region holds all of the kernel. */
if (s < kernload && e > phys_kernelend) {
availmem_regions[cnt].mr_start = phys_kernelend;
availmem_regions[cnt++].mr_size = e - phys_kernelend;
e = kernload;
}
/* Look whether this regions starts within the kernel. */
if (s >= kernload && s < phys_kernelend) {
if (e <= phys_kernelend)
goto empty;
s = phys_kernelend;
}
/* Now look whether this region ends within the kernel. */
if (e > kernload && e <= phys_kernelend) {
if (s >= kernload)
goto empty;
e = kernload;
}
/* Now page align the start and size of the region. */
s = round_page(s);
e = trunc_page(e);
if (e < s)
e = s;
sz = e - s;
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
debugf("%09jx-%09jx = %jx\n",
(uintmax_t)s, (uintmax_t)e, (uintmax_t)sz);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Check whether some memory is left here. */
if (sz == 0) {
empty:
memmove(mp, mp + 1,
(cnt - (mp - availmem_regions)) * sizeof(*mp));
cnt--;
mp--;
continue;
}
/* Do an insertion sort. */
for (mp1 = availmem_regions; mp1 < mp; mp1++)
if (s < mp1->mr_start)
break;
if (mp1 < mp) {
memmove(mp1 + 1, mp1, (char *)mp - (char *)mp1);
mp1->mr_start = s;
mp1->mr_size = sz;
} else {
mp->mr_start = s;
mp->mr_size = sz;
}
}
availmem_regions_sz = cnt;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
/*******************************************************/
/* Steal physical memory for kernel stack from the end */
/* of the first avail region */
/*******************************************************/
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
kstack0_sz = kstack_pages * PAGE_SIZE;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
kstack0_phys = availmem_regions[0].mr_start +
availmem_regions[0].mr_size;
kstack0_phys -= kstack0_sz;
availmem_regions[0].mr_size -= kstack0_sz;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Fill in phys_avail table, based on availmem_regions */
/*******************************************************/
phys_avail_count = 0;
physsz = 0;
hwphyssz = 0;
TUNABLE_ULONG_FETCH("hw.physmem", (u_long *) &hwphyssz);
debugf("fill in phys_avail:\n");
for (i = 0, j = 0; i < availmem_regions_sz; i++, j += 2) {
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
debugf(" region: 0x%jx - 0x%jx (0x%jx)\n",
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
(uintmax_t)availmem_regions[i].mr_start,
(uintmax_t)availmem_regions[i].mr_start +
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
availmem_regions[i].mr_size,
Extend Book-E to support >4GB RAM Summary: With some additional changes for AIM, that could also support much larger physmem sizes. Given that 32-bit AIM is more or less obsolete, though, it's not worth it at this time. Differential Revision: https://reviews.freebsd.org/D4345
2015-12-24 04:30:15 +00:00
(uintmax_t)availmem_regions[i].mr_size);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
if (hwphyssz != 0 &&
(physsz + availmem_regions[i].mr_size) >= hwphyssz) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf(" hw.physmem adjust\n");
if (physsz < hwphyssz) {
phys_avail[j] = availmem_regions[i].mr_start;
Move initialization of tlb0, ptbl_bufs and kernel_pdir regions after we are 100% sure that TLB1 mapping covers for them; previously we could lock the CPU with an untranslated references. Obtained from: Semihalf
2008-08-28 07:38:08 +00:00
phys_avail[j + 1] =
availmem_regions[i].mr_start +
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
hwphyssz - physsz;
physsz = hwphyssz;
phys_avail_count++;
powerpc/booke: Fix Book-E boot post-minidump r353489 added minidump support for powerpc64, but it added a dependency on the dump_avail array. Leaving it uninitialized caused breakage in late boot. Initialize dump_avail, even though the 64-bit booke pmap doesn't yet support minidumps, but will in the future.
2019-10-23 00:31:19 +00:00
dump_avail[j] = phys_avail[j];
dump_avail[j + 1] = phys_avail[j + 1];
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
break;
}
phys_avail[j] = availmem_regions[i].mr_start;
phys_avail[j + 1] = availmem_regions[i].mr_start +
availmem_regions[i].mr_size;
phys_avail_count++;
physsz += availmem_regions[i].mr_size;
powerpc/booke: Fix Book-E boot post-minidump r353489 added minidump support for powerpc64, but it added a dependency on the dump_avail array. Leaving it uninitialized caused breakage in late boot. Initialize dump_avail, even though the 64-bit booke pmap doesn't yet support minidumps, but will in the future.
2019-10-23 00:31:19 +00:00
dump_avail[j] = phys_avail[j];
dump_avail[j + 1] = phys_avail[j + 1];
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
physmem = btoc(physsz);
/* Calculate the last available physical address. */
for (i = 0; phys_avail[i + 2] != 0; i += 2)
;
Maxmem = powerpc_btop(phys_avail[i + 1]);
debugf("Maxmem = 0x%08lx\n", Maxmem);
debugf("phys_avail_count = %d\n", phys_avail_count);
Fix a debug printf(). Somehow this printf() was missed in the conversion of vm_paddr_t to 64-bit, and made it through until now. Sponsored by: Alex Perez/Inertial Computing
2016-01-26 03:52:14 +00:00
debugf("physsz = 0x%09jx physmem = %jd (0x%09jx)\n",
(uintmax_t)physsz, (uintmax_t)physmem, (uintmax_t)physmem);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add support for a pmap direct map for 64-bit Book-E As with AIM64, map the DMAP at the beginning of the fourth "quadrant" of memory, and move the KERNBASE to the the start of KVA. Eventually we may run the kernel out of the DMAP, but for now, continue booting as it has been.
2018-04-03 00:45:38 +00:00
#ifdef __powerpc64__
/*
* Map the physical memory contiguously in TLB1.
* Round so it fits into a single mapping.
*/
tlb1_mapin_region(DMAP_BASE_ADDRESS, 0,
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
phys_avail[i + 1], _TLB_ENTRY_MEM);
Add support for a pmap direct map for 64-bit Book-E As with AIM64, map the DMAP at the beginning of the fourth "quadrant" of memory, and move the KERNBASE to the the start of KVA. Eventually we may run the kernel out of the DMAP, but for now, continue booting as it has been.
2018-04-03 00:45:38 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Initialize (statically allocated) kernel pmap. */
/*******************************************************/
PMAP_LOCK_INIT(kernel_pmap);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("kernel_pmap = 0x%"PRI0ptrX"\n", (uintptr_t)kernel_pmap);
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
kernel_pte_alloc(virtual_avail, kernstart);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
for (i = 0; i < MAXCPU; i++) {
kernel_pmap->pm_tid[i] = TID_KERNEL;
/* Initialize each CPU's tidbusy entry 0 with kernel_pmap */
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
tidbusy[i][TID_KERNEL] = kernel_pmap;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
}
Fill PTEs covering kernel code and data. Without this fix pte_vatopa() was not able to retrieve physical address of data structures inside kernel, for example EFAULT was reported while acessing /dev/kmem ('netstat -nr'). Submitted by: Piotr Ziecik Obtained from: Semihalf
2009-06-05 09:09:46 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Mark kernel_pmap active on all CPUs */
Add the powerpc support. Note that there is a dirty hack for calling openpic_write(), but nwhitehorn approved it. Discussed with: nwhitehorn
2011-05-09 16:16:15 +00:00
CPU_FILL(&kernel_pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
/*
* Initialize the global pv list lock.
*/
rw_init(&pvh_global_lock, "pmap pv global");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*******************************************************/
/* Final setup */
/*******************************************************/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
/* Enter kstack0 into kernel map, provide guard page */
kstack0 = virtual_avail + KSTACK_GUARD_PAGES * PAGE_SIZE;
thread0.td_kstack = kstack0;
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
thread0.td_kstack_pages = kstack_pages;
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
Add support for building Book-E kernels with clang/lld. This involved several changes: * Since lld does not like text relocations, replace SMP boot page text relocs in booke/locore.S with position-independent math, and track the virtual base in the SMP boot page header. * As some SPRs are interpreted differently on clang due to the way it handles platform-specific SPRs, switch m*dear and m*esr mnemonics out for regular m*spr. Add both forms of SPR_DEAR to spr.h so the correct encoding is selected. * Change some hardcoded 32 bit things in the boot page to be pointer-sized, and fix alignment. * Fix 64-bit build of booke/pmap.c when enabling pmap debugging. Additionally, I took the opportunity to document how the SMP boot page works. Approved by: jhibbits (mentor) Differential Revision: https://reviews.freebsd.org/D21999
2019-11-02 21:15:56 +00:00
debugf("kstack_sz = 0x%08jx\n", (uintmax_t)kstack0_sz);
debugf("kstack0_phys at 0x%09jx - 0x%09jx\n",
(uintmax_t)kstack0_phys, (uintmax_t)kstack0_phys + kstack0_sz);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("kstack0 at 0x%"PRI0ptrX" - 0x%"PRI0ptrX"\n",
kstack0, kstack0 + kstack0_sz);
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
virtual_avail += KSTACK_GUARD_PAGES * PAGE_SIZE + kstack0_sz;
Make kstack_pages a tunable on arm, x86, and powepc. On i386, the initial thread stack is not adjusted by the tunable, the stack is allocated too early to get access to the kernel environment. See TD0_KSTACK_PAGES for the thread0 stack sizing on i386. The tunable was tested on x86 only. From the visual inspection, it seems that it might work on arm and powerpc. The arm USPACE_SVC_STACK_TOP and powerpc USPACE macros seems to be already incorrect for the threads with non-default kstack size. I only changed the macros to use variable instead of constant, since I cannot test. On arm64, mips and sparc64, some static data structures are sized by KSTACK_PAGES, so the tunable is disabled. Sponsored by: The FreeBSD Foundation MFC after: 2 week
2015-08-10 17:18:21 +00:00
for (i = 0; i < kstack_pages; i++) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter(kstack0, kstack0_phys);
Improve kernel stack handling on e500. - Allocate thread0.td_kstack in pmap_bootstrap(), provide guard page - Switch to thread0.td_kstack as soon as possible i.e. right after return from e500_init() and before mi_startup() happens - Clean up temp stack area - Other minor cosmetics in machdep.c Obtained from: Semihalf
2008-08-26 17:07:37 +00:00
kstack0 += PAGE_SIZE;
kstack0_phys += PAGE_SIZE;
}
Initialize NX stack capabilities and direct map status in pmap like on AIM.
2015-03-01 21:23:23 +00:00
pmap_bootstrapped = 1;
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
debugf("virtual_avail = %"PRI0ptrX"\n", virtual_avail);
debugf("virtual_end = %"PRI0ptrX"\n", virtual_end);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
debugf("mmu_booke_bootstrap: exit\n");
}
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#ifdef SMP
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
void
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_ap_prep(void)
{
tlb_entry_t *e, tmp;
unsigned int i;
/* Prepare TLB1 image for AP processors */
e = __boot_tlb1;
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&tmp, i);
if ((tmp.mas1 & MAS1_VALID) && (tmp.mas2 & _TLB_ENTRY_SHARED))
memcpy(e++, &tmp, sizeof(tmp));
}
}
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
void
pmap_bootstrap_ap(volatile uint32_t *trcp __unused)
{
int i;
/*
* Finish TLB1 configuration: the BSP already set up its TLB1 and we
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
* have the snapshot of its contents in the s/w __boot_tlb1[] table
* created by tlb1_ap_prep(), so use these values directly to
* (re)program AP's TLB1 hardware.
*
* Start at index 1 because index 0 has the kernel map.
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 1; i < TLB1_ENTRIES; i++) {
if (__boot_tlb1[i].mas1 & MAS1_VALID)
tlb1_write_entry(&__boot_tlb1[i], i);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
}
set_mas4_defaults();
}
Wrap #ifdef guards around pmap_bootstrap ap. It's only used in SMP, and building without SMP causes a build failure. MFC after: 1 month
2015-03-28 21:39:42 +00:00
#endif
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
Add two new pmap functions: vm_offset_t pmap_quick_enter_page(vm_page_t m) void pmap_quick_remove_page(vm_offset_t kva) These will create and destroy a temporary, CPU-local KVA mapping of a specified page. Guarantees: --Will not sleep and will not fail. --Safe to call under a non-sleepable lock or from an ithread Restrictions: --Not guaranteed to be safe to call from an interrupt filter or under a spin mutex on all platforms --Current implementation does not guarantee more than one page of mapping space across all platforms. MI code should not make nested calls to pmap_quick_enter_page. --MI code should not perform locking while holding onto a mapping created by pmap_quick_enter_page The idea is to use this in busdma, for bounce buffer copies as well as virtually-indexed cache maintenance on mips and arm. NOTE: the non-i386, non-amd64 implementations of these functions still need review and testing. Reviewed by: kib Approved by: kib (mentor) Differential Revision: http://reviews.freebsd.org/D3013
2015-08-04 19:46:13 +00:00
static void
booke_pmap_init_qpages(void)
{
struct pcpu *pc;
int i;
CPU_FOREACH(i) {
pc = pcpu_find(i);
pc->pc_qmap_addr = kva_alloc(PAGE_SIZE);
if (pc->pc_qmap_addr == 0)
panic("pmap_init_qpages: unable to allocate KVA");
}
}
SYSINIT(qpages_init, SI_SUB_CPU, SI_ORDER_ANY, booke_pmap_init_qpages, NULL);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Get the physical page address for the given pmap/virtual address.
*/
static vm_paddr_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_extract(pmap_t pmap, vm_offset_t va)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_paddr_t pa;
PMAP_LOCK(pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pa = pte_vatopa(pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
return (pa);
}
/*
* Extract the physical page address associated with the given
* kernel virtual address.
*/
static vm_paddr_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kextract(vm_offset_t va)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Only check the page tables if within the KVA. Devices aren't mapped within the KVA, and with the way 64-bit hashes the addresses pte_vatopa() may not return a 0 physical address for a device. MFC after: 1 week
2017-11-29 01:26:07 +00:00
vm_paddr_t p = 0;
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
int i;
powerpc/pmap: Enable UMA_MD_SMALL_ALLOC for 64-bit booke The only thing blocking UMA_MD_SMALL_ALLOC from working on 64-bit booke powerpc was a missing check in pmap_kextract(). Adding DMAP handling into pmap_kextract(), we can now use UMA_MD_SMALL_ALLOC. This should improve performance and stability a bit, since DMAP is always mapped in TLB1, so this relieves pressure on TLB0. MFC after: 3 weeks
2019-08-15 03:42:15 +00:00
#ifdef __powerpc64__
if (va >= DMAP_BASE_ADDRESS && va <= DMAP_MAX_ADDRESS)
return (DMAP_TO_PHYS(va));
#endif
Only check the page tables if within the KVA. Devices aren't mapped within the KVA, and with the way 64-bit hashes the addresses pte_vatopa() may not return a 0 physical address for a device. MFC after: 1 week
2017-11-29 01:26:07 +00:00
if (va >= VM_MIN_KERNEL_ADDRESS && va <= VM_MAX_KERNEL_ADDRESS)
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
p = pte_vatopa(kernel_pmap, va);
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Check the page table before TLB1 in pmap_kextract() The vast majority of pmap_kextract() calls are looking for a physical memory address, not a device address. By checking the page table first this saves the formerly inevitable 64 (on e500mc and derivatives) iteration loop through TLB1 in the most common cases. Benchmarking this on the P5020 (e5500 core) yields a 300% throughput improvement on dtsec(4) (115Mbit/s -> 460Mbit/s) measured with iperf. Benchmarked on the P1022 (e500v2 core, 16 TLB1 entries) yields a 50% throughput improvement on tsec(4) (~93Mbit/s -> 165Mbit/s) measured with iperf. MFC after: 1 week Relnotes: Maybe (significant performance improvement)
2017-11-21 03:12:16 +00:00
if (p == 0) {
/* Check TLB1 mappings */
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
if (va >= e.virt && va < e.virt + e.size)
return (e.phys + (va - e.virt));
}
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Check the page table before TLB1 in pmap_kextract() The vast majority of pmap_kextract() calls are looking for a physical memory address, not a device address. By checking the page table first this saves the formerly inevitable 64 (on e500mc and derivatives) iteration loop through TLB1 in the most common cases. Benchmarking this on the P5020 (e5500 core) yields a 300% throughput improvement on dtsec(4) (115Mbit/s -> 460Mbit/s) measured with iperf. Benchmarked on the P1022 (e500v2 core, 16 TLB1 entries) yields a 50% throughput improvement on tsec(4) (~93Mbit/s -> 165Mbit/s) measured with iperf. MFC after: 1 week Relnotes: Maybe (significant performance improvement)
2017-11-21 03:12:16 +00:00
return (p);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Initialize the pmap module.
* Called by vm_init, to initialize any structures that the pmap
* system needs to map virtual memory.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_init()
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
int shpgperproc = PMAP_SHPGPERPROC;
/*
* Initialize the address space (zone) for the pv entries. Set a
* high water mark so that the system can recover from excessive
* numbers of pv entries.
*/
pvzone = uma_zcreate("PV ENTRY", sizeof(struct pv_entry), NULL, NULL,
NULL, NULL, UMA_ALIGN_PTR, UMA_ZONE_VM | UMA_ZONE_NOFREE);
TUNABLE_INT_FETCH("vm.pmap.shpgperproc", &shpgperproc);
Rename global cnt to vm_cnt to avoid shadowing. To reduce the diff struct pcu.cnt field was not renamed, so PCPU_OP(cnt.field) is still used. pc_cnt and pcpu are also used in kvm(3) and vmstat(8). The goal was to not affect externally used KPI. Bump __FreeBSD_version_ in case some out-of-tree module/code relies on the the global cnt variable. Exp-run revealed no ports using it directly. No objection from: arch@ Sponsored by: EMC / Isilon Storage Division
2014-03-22 10:26:09 +00:00
pv_entry_max = shpgperproc * maxproc + vm_cnt.v_page_count;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TUNABLE_INT_FETCH("vm.pmap.pv_entries", &pv_entry_max);
pv_entry_high_water = 9 * (pv_entry_max / 10);
Merge from vmc-playground branch: Replace the sub-optimal uma_zone_set_obj() primitive with more modern uma_zone_reserve_kva(). The new primitive reserves before hand the necessary KVA space to cater the zone allocations and allocates pages with ALLOC_NOOBJ. More specifically: - uma_zone_reserve_kva() does not need an object to cater the backend allocator. - uma_zone_reserve_kva() can cater M_WAITOK requests, in order to serve zones which need to do uma_prealloc() too. - When possible, uma_zone_reserve_kva() uses directly the direct-mapping by uma_small_alloc() rather than relying on the KVA / offset combination. The removal of the object attribute allows 2 further changes: 1) _vm_object_allocate() becomes static within vm_object.c 2) VM_OBJECT_LOCK_INIT() is removed. This function is replaced by direct calls to mtx_init() as there is no need to export it anymore and the calls aren't either homogeneous anymore: there are now small differences between arguments passed to mtx_init(). Sponsored by: EMC / Isilon storage division Reviewed by: alc (which also offered almost all the comments) Tested by: pho, jhb, davide
2013-02-26 23:35:27 +00:00
uma_zone_reserve_kva(pvzone, pv_entry_max);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Pre-fill pvzone with initial number of pv entries. */
uma_prealloc(pvzone, PV_ENTRY_ZONE_MIN);
powerpc: Unify pmap definitions between AIM and Book-E This is part 2 of r347078, pulling the page directory out of the Book-E pmap. This breaks KBI for anything that uses struct pmap (such as vm_map) so any modules that access this must be rebuilt.
2019-08-12 03:03:56 +00:00
/* Create a UMA zone for page table roots. */
ptbl_root_zone = uma_zcreate("pmap root", PMAP_ROOT_SIZE,
NULL, NULL, NULL, NULL, UMA_ALIGN_CACHE, UMA_ZONE_VM);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Initialize ptbl allocation. */
ptbl_init();
}
/*
* Map a list of wired pages into kernel virtual address space. This is
* intended for temporary mappings which do not need page modification or
* references recorded. Existing mappings in the region are overwritten.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_qenter(vm_offset_t sva, vm_page_t *m, int count)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_offset_t va;
va = sva;
while (count-- > 0) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter(va, VM_PAGE_TO_PHYS(*m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
va += PAGE_SIZE;
m++;
}
}
/*
* Remove page mappings from kernel virtual address space. Intended for
* temporary mappings entered by mmu_booke_qenter.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_qremove(vm_offset_t sva, int count)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_offset_t va;
va = sva;
while (count-- > 0) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kremove(va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
va += PAGE_SIZE;
}
}
/*
* Map a wired page into kernel virtual address space.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter(vm_offset_t va, vm_paddr_t pa)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter_attr(va, pa, VM_MEMATTR_DEFAULT);
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
}
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter_attr(vm_offset_t va, vm_paddr_t pa, vm_memattr_t ma)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pte_t *pte;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= VM_MIN_KERNEL_ADDRESS) &&
(va <= VM_MAX_KERNEL_ADDRESS)), ("mmu_booke_kenter: invalid va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
flags = PTE_SR | PTE_SW | PTE_SX | PTE_WIRED | PTE_VALID;
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
flags |= tlb_calc_wimg(pa, ma) << PTE_MAS2_SHIFT;
flags |= PTE_PS_4KB;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
KASSERT((pte != NULL), ("mmu_booke_kenter: invalid va. NULL PTE"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: replacing entry!", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush entry from TLB0 */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = PTE_RPN_FROM_PA(pa) | flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
//debugf("mmu_booke_kenter: pdir_idx = %d ptbl_idx = %d va=0x%08x "
// "pa=0x%08x rpn=0x%08x flags=0x%08x\n",
// pdir_idx, ptbl_idx, va, pa, pte->rpn, pte->flags);
/* Flush the real memory from the instruction cache. */
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if ((flags & (PTE_I | PTE_G)) == 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
__syncicache((void *)va, PAGE_SIZE);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Remove a page from kernel page table.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kremove(vm_offset_t va)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
CTR2(KTR_PMAP,"%s: s (va = 0x%"PRI0ptrX")\n", __func__, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
KASSERT(((va >= VM_MIN_KERNEL_ADDRESS) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
("mmu_booke_kremove: invalid va"));
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: invalid pte", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return;
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Invalidate entry in TLB0, update PTE. */
tlb0_flush_entry(va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte = 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Remove hard-coded trap-handling logic involving the segmented memory model used with hashed page tables on AIM and place it into a new, modular pmap function called pmap_decode_kernel_ptr(). This function is the inverse of pmap_map_user_ptr(). With POWER9 radix tables, which mapping to use becomes more complex than just AIM/BOOKE and it is best to have it in the same place as pmap_map_user_ptr(). Reviewed by: jhibbits
2018-01-29 04:33:41 +00:00
/*
* Figure out where a given kernel pointer (usually in a fault) points
* to from the VM's perspective, potentially remapping into userland's
* address space.
*/
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_decode_kernel_ptr(vm_offset_t addr, int *is_user,
Remove hard-coded trap-handling logic involving the segmented memory model used with hashed page tables on AIM and place it into a new, modular pmap function called pmap_decode_kernel_ptr(). This function is the inverse of pmap_map_user_ptr(). With POWER9 radix tables, which mapping to use becomes more complex than just AIM/BOOKE and it is best to have it in the same place as pmap_map_user_ptr(). Reviewed by: jhibbits
2018-01-29 04:33:41 +00:00
vm_offset_t *decoded_addr)
{
powerpc: Link Book-E kernels at the same address as AIM kernels Summary: Reduce the diff between AIM and Book-E even more. This also cleans up vmparam.h significantly. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21301
2019-08-20 01:26:02 +00:00
if (trunc_page(addr) <= VM_MAXUSER_ADDRESS)
Remove hard-coded trap-handling logic involving the segmented memory model used with hashed page tables on AIM and place it into a new, modular pmap function called pmap_decode_kernel_ptr(). This function is the inverse of pmap_map_user_ptr(). With POWER9 radix tables, which mapping to use becomes more complex than just AIM/BOOKE and it is best to have it in the same place as pmap_map_user_ptr(). Reviewed by: jhibbits
2018-01-29 04:33:41 +00:00
*is_user = 1;
else
*is_user = 0;
*decoded_addr = addr;
return (0);
}
powerpc64: Implement Radix MMU for POWER9 CPUs Summary: POWER9 supports two MMU formats: traditional hashed page tables, and Radix page tables, similar to what's presesnt on most other architectures. The PowerISA also specifies a process table -- a table of page table pointers-- which on the POWER9 is only available with the Radix MMU, so we can take advantage of it with the Radix MMU driver. Written by Matt Macy. Differential Revision: https://reviews.freebsd.org/D19516
2020-05-11 02:33:37 +00:00
static boolean_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_page_is_mapped(vm_page_t m)
powerpc64: Implement Radix MMU for POWER9 CPUs Summary: POWER9 supports two MMU formats: traditional hashed page tables, and Radix page tables, similar to what's presesnt on most other architectures. The PowerISA also specifies a process table -- a table of page table pointers-- which on the POWER9 is only available with the Radix MMU, so we can take advantage of it with the Radix MMU driver. Written by Matt Macy. Differential Revision: https://reviews.freebsd.org/D19516
2020-05-11 02:33:37 +00:00
{
return (!TAILQ_EMPTY(&(m)->md.pv_list));
}
Clean up a couple of MD warts in vm_fault_populate(): --Eliminate a big ifdef that encompassed all currently-supported architectures except mips and powerpc32. This applied to the case in which we've allocated a superpage but the pager-populated range is insufficient for a superpage mapping. For platforms that don't support superpages the check should be inexpensive as we shouldn't get a superpage in the first place. Make the normal-page fallback logic identical for all platforms and provide a simple implementation of pmap_ps_enabled() for MIPS and Book-E/AIM32 powerpc. --Apply the logic for handling pmap_enter() failure if a superpage mapping can't be supported due to additional protection policy. Use KERN_PROTECTION_FAILURE instead of KERN_FAILURE for this case, and note Intel PKU on amd64 as the first example of such protection policy. Reviewed by: kib, markj, bdragon Differential Revision: https://reviews.freebsd.org/D29439
2021-03-27 03:10:46 +00:00
static bool
mmu_booke_ps_enabled(pmap_t pmap __unused)
{
return (false);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Initialize pmap associated with process 0.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_pinit0(pmap_t pmap)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Revert r254501. Instead, reuse the type stability of the struct pmap which is the part of struct vmspace, allocated from UMA_ZONE_NOFREE zone. Initialize the pmap lock in the vmspace zone init function, and remove pmap lock initialization and destruction from pmap_pinit() and pmap_release(). Suggested and reviewed by: alc (previous version) Tested by: pho Sponsored by: The FreeBSD Foundation
2013-08-22 18:12:24 +00:00
PMAP_LOCK_INIT(pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_pinit(pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PCPU_SET(curpmap, pmap);
}
/*
* Insert the given physical page at the specified virtual address in the
* target physical map with the protection requested. If specified the page
* will be wired down.
*/
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter(pmap_t pmap, vm_offset_t va, vm_page_t m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t prot, u_int flags, int8_t psind)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
int error;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
error = mmu_booke_enter_locked(pmap, va, m, prot, flags, psind);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
rw_wunlock(&pvh_global_lock);
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
return (error);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter_locked(pmap_t pmap, vm_offset_t va, vm_page_t m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
vm_prot_t prot, u_int pmap_flags, int8_t psind __unused)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
vm_paddr_t pa;
powerpc/booke: Change Book-E 64-bit pmap to 4-level table Summary: The existing page table is fraught with errors, since it creates a hole in the address space bits. Fix this by taking a cue from the POWER9 radix pmap, and make the page table 4 levels, 52 bits. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24220
2020-04-11 00:12:34 +00:00
pte_t flags;
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
int error, su, sync;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pa = VM_PAGE_TO_PHYS(m);
su = (pmap == kernel_pmap);
sync = 0;
//debugf("mmu_booke_enter_locked: s (pmap=0x%08x su=%d tid=%d m=0x%08x va=0x%08x "
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
// "pa=0x%08x prot=0x%08x flags=%#x)\n",
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
// (u_int32_t)pmap, su, pmap->pm_tid,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
// (u_int32_t)m, va, pa, prot, flags);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (su) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= virtual_avail) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
("mmu_booke_enter_locked: kernel pmap, non kernel va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
KASSERT((va <= VM_MAXUSER_ADDRESS),
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
("mmu_booke_enter_locked: user pmap, non user va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Fix assert in PowerPC pmaps after introduction of object busy. The VM_PAGE_OBJECT_BUSY_ASSERT() in pmap_enter() implementation should be only asserted when the code is executed as result of pmap_enter(), not when the same code is entered from e.g. pmap_enter_quick(). This is relevant for all PowerPC pmap variants, because mmu_*_enter() is used as the backend, and assert is located there. Add a PowerPC private pmap_enter() PMAP_ENTER_QUICK_LOCKED flag to indicate that the call is not from pmap_enter(). For non-quick-locked calls, assert that the object is locked. Reported and tested by: bdragon Reviewed by: alc, bdragon, markj Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D22041
2019-10-16 07:09:15 +00:00
if ((m->oflags & VPO_UNMANAGED) == 0) {
if ((pmap_flags & PMAP_ENTER_QUICK_LOCKED) == 0)
VM_PAGE_OBJECT_BUSY_ASSERT(m);
else
VM_OBJECT_ASSERT_LOCKED(m->object);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK_ASSERT(pmap, MA_OWNED);
/*
* If there is an existing mapping, and the physical address has not
* changed, must be protection or wiring change.
*/
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if (((pte = pte_find(pmap, va)) != NULL) &&
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
(PTE_ISVALID(pte)) && (PTE_PA(pte) == pa)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* Before actually updating pte->flags we calculate and
* prepare its new value in a helper var.
*/
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
flags = *pte;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~(PTE_UW | PTE_UX | PTE_SW | PTE_SX | PTE_MODIFIED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Wiring change, just update stats. */
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
if ((pmap_flags & PMAP_ENTER_WIRED) != 0) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!PTE_ISWIRED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_WIRED;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap->pm_stats.wired_count++;
}
} else {
if (PTE_ISWIRED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~PTE_WIRED;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap->pm_stats.wired_count--;
}
}
if (prot & VM_PROT_WRITE) {
/* Add write permissions. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_SW;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_UW;
Set PG_WRITEABLE in Book-E pmap_enter[_locked] if it creates a mapping that permits write access. This is similar to r192671. Pointed out and reviewed by: alc
2009-05-26 06:24:50 +00:00
Don't set PG_WRITEABLE in pmap_enter() unless the page is managed. Correct a typo in a nearby comment on sparc64.
2010-06-05 18:20:09 +00:00
if ((flags & PTE_MANAGED) != 0)
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_set(m, PGA_WRITEABLE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
/* Handle modified pages, sense modify status. */
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* The PTE_MODIFIED flag could be set by underlying
* TLB misses since we last read it (above), possibly
* other CPUs could update it so we check in the PTE
* directly rather than rely on that saved local flags
* copy.
*/
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (prot & VM_PROT_EXECUTE) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_SX;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_UX;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* Check existing flags for execute permissions: if we
* are turning execute permissions on, icache should
* be flushed.
*/
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if ((*pte & (PTE_UX | PTE_SX)) == 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
sync++;
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags &= ~PTE_REFERENCED;
/*
* The new flags value is all calculated -- only now actually
* update the PTE.
*/
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Fix a panic bug that cropped up in the PTE rewrite. PTE was getting overwritten by just the flags. Pointy-hat to: jhibbits
2016-02-16 02:13:55 +00:00
*pte &= ~PTE_FLAGS_MASK;
*pte |= flags;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* If there is an existing mapping, but it's for a different
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* physical address, pte_enter() will delete the old mapping.
*/
//if ((pte != NULL) && PTE_ISVALID(pte))
// debugf("mmu_booke_enter_locked: replace\n");
//else
// debugf("mmu_booke_enter_locked: new\n");
/* Now set up the flags and install the new mapping. */
flags = (PTE_SR | PTE_VALID);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
flags |= PTE_M;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (!su)
flags |= PTE_UR;
if (prot & VM_PROT_WRITE) {
flags |= PTE_SW;
if (!su)
flags |= PTE_UW;
Set PG_WRITEABLE in Book-E pmap_enter[_locked] if it creates a mapping that permits write access. This is similar to r192671. Pointed out and reviewed by: alc
2009-05-26 06:24:50 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
if ((m->oflags & VPO_UNMANAGED) == 0)
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_set(m, PGA_WRITEABLE);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (prot & VM_PROT_EXECUTE) {
flags |= PTE_SX;
if (!su)
flags |= PTE_UX;
}
/* If its wired update stats. */
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
if ((pmap_flags & PMAP_ENTER_WIRED) != 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
flags |= PTE_WIRED;
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
error = pte_enter(pmap, m, va, flags,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
(pmap_flags & PMAP_ENTER_NOSLEEP) != 0);
if (error != 0)
return (KERN_RESOURCE_SHORTAGE);
if ((flags & PMAP_ENTER_WIRED) != 0)
pmap->pm_stats.wired_count++;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush the real memory from the instruction cache. */
if (prot & VM_PROT_EXECUTE)
sync++;
}
if (sync && (su || pmap == PCPU_GET(curpmap))) {
__syncicache((void *)va, PAGE_SIZE);
sync = 0;
}
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
return (KERN_SUCCESS);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Maps a sequence of resident pages belonging to the same object.
* The sequence begins with the given page m_start. This page is
* mapped at the given virtual address start. Each subsequent page is
* mapped at a virtual address that is offset from start by the same
* amount as the page is offset from m_start within the object. The
* last page in the sequence is the page with the largest offset from
* m_start that can be mapped at a virtual address less than the given
* virtual address end. Not every virtual page between start and end
* is mapped; only those for which a resident page exists with the
* corresponding offset from m_start are mapped.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter_object(pmap_t pmap, vm_offset_t start,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_offset_t end, vm_page_t m_start, vm_prot_t prot)
{
vm_page_t m;
vm_pindex_t diff, psize;
o Relax locking assertions for vm_page_find_least() o Relax locking assertions for pmap_enter_object() and add them also to architectures that currently don't have any o Introduce VM_OBJECT_LOCK_DOWNGRADE() which is basically a downgrade operation on the per-object rwlock o Use all the mechanisms above to make vm_map_pmap_enter() to work mostl of the times only with readlocks. Sponsored by: EMC / Isilon storage division Reviewed by: alc
2013-05-21 20:38:19 +00:00
VM_OBJECT_ASSERT_LOCKED(m_start->object);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
psize = atop(end - start);
m = m_start;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
while (m != NULL && (diff = m->pindex - m_start->pindex) < psize) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter_locked(pmap, start + ptoa(diff), m,
Change pmap_enter(9) interface to take flags parameter and superpage mapping size (currently unused). The flags includes the fault access bits, wired flag as PMAP_ENTER_WIRED, and a new flag PMAP_ENTER_NOSLEEP to indicate that pmap should not sleep. For powerpc aim both 32 and 64 bit, fix implementation to ensure that the requested mapping is created when PMAP_ENTER_NOSLEEP is not specified, in particular, wait for the available memory required to proceed. In collaboration with: alc Tested by: nwhitehorn (ppc aim32 and booke) Sponsored by: The FreeBSD Foundation and EMC / Isilon Storage Division MFC after: 2 weeks
2014-08-08 17:12:03 +00:00
prot & (VM_PROT_READ | VM_PROT_EXECUTE),
Fix assert in PowerPC pmaps after introduction of object busy. The VM_PAGE_OBJECT_BUSY_ASSERT() in pmap_enter() implementation should be only asserted when the code is executed as result of pmap_enter(), not when the same code is entered from e.g. pmap_enter_quick(). This is relevant for all PowerPC pmap variants, because mmu_*_enter() is used as the backend, and assert is located there. Add a PowerPC private pmap_enter() PMAP_ENTER_QUICK_LOCKED flag to indicate that the call is not from pmap_enter(). For non-quick-locked calls, assert that the object is locked. Reported and tested by: bdragon Reviewed by: alc, bdragon, markj Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D22041
2019-10-16 07:09:15 +00:00
PMAP_ENTER_NOSLEEP | PMAP_ENTER_QUICK_LOCKED, 0);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = TAILQ_NEXT(m, listq);
}
PMAP_UNLOCK(pmap);
powerpc/booke: Fix handling of pvh_global_lock and pmap lock ptbl_alloc() is expected to return with the pvh_global_lock and pmap lock held. However, it will return with them unlocked if nosleep is specified. Along with this, fix lock ordering of pvh_global_lock with respect to the pmap lock in other places. Differential Revision: https://reviews.freebsd.org/D23692
2020-02-22 01:31:06 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter_quick(pmap_t pmap, vm_offset_t va, vm_page_t m,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t prot)
{
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_enter_locked(pmap, va, m,
Fix assert in PowerPC pmaps after introduction of object busy. The VM_PAGE_OBJECT_BUSY_ASSERT() in pmap_enter() implementation should be only asserted when the code is executed as result of pmap_enter(), not when the same code is entered from e.g. pmap_enter_quick(). This is relevant for all PowerPC pmap variants, because mmu_*_enter() is used as the backend, and assert is located there. Add a PowerPC private pmap_enter() PMAP_ENTER_QUICK_LOCKED flag to indicate that the call is not from pmap_enter(). For non-quick-locked calls, assert that the object is locked. Reported and tested by: bdragon Reviewed by: alc, bdragon, markj Sponsored by: The FreeBSD Foundation Differential revision: https://reviews.freebsd.org/D22041
2019-10-16 07:09:15 +00:00
prot & (VM_PROT_READ | VM_PROT_EXECUTE), PMAP_ENTER_NOSLEEP |
PMAP_ENTER_QUICK_LOCKED, 0);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pmap);
powerpc/booke: Fix handling of pvh_global_lock and pmap lock ptbl_alloc() is expected to return with the pvh_global_lock and pmap lock held. However, it will return with them unlocked if nosleep is specified. Along with this, fix lock ordering of pvh_global_lock with respect to the pmap lock in other places. Differential Revision: https://reviews.freebsd.org/D23692
2020-02-22 01:31:06 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Remove the given range of addresses from the specified map.
*
* It is assumed that the start and end are properly rounded to the page size.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_remove(pmap_t pmap, vm_offset_t va, vm_offset_t endva)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint8_t hold_flag;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
int su = (pmap == kernel_pmap);
//debugf("mmu_booke_remove: s (su = %d pmap=0x%08x tid=%d va=0x%08x endva=0x%08x)\n",
// su, (u_int32_t)pmap, pmap->pm_tid, va, endva);
if (su) {
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
KASSERT(((va >= virtual_avail) &&
(va <= VM_MAX_KERNEL_ADDRESS)),
("mmu_booke_remove: kernel pmap, non kernel va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
} else {
KASSERT((va <= VM_MAXUSER_ADDRESS),
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
("mmu_booke_remove: user pmap, non user va"));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
if (PMAP_REMOVE_DONE(pmap)) {
//debugf("mmu_booke_remove: e (empty)\n");
return;
}
hold_flag = PTBL_HOLD_FLAG(pmap);
//debugf("mmu_booke_remove: hold_flag = %d\n", hold_flag);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pmap);
for (; va < endva; va += PAGE_SIZE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find_next(pmap, &va);
powerpc/booke: Add pte_find_next() to find the next in-use PTE Summary: Iterating over VM_MIN_ADDRESS->VM_MAXUSER_ADDRESS can take a very long time iterating one page at a time (2**(log_2(SIZE)-12) operations), yielding possibly several days or even weeks on 64-bit Book-E, even for a largely empty, which can happen when swapping out a process by vmdaemon. Speed this up by instead finding the next PTE at or equal to the given VA. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24238
2020-04-11 00:16:50 +00:00
if ((pte == NULL) || !PTE_ISVALID(pte))
break;
if (va >= endva)
break;
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte_remove(pmap, va, hold_flag);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
PMAP_UNLOCK(pmap);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
//debugf("mmu_booke_remove: e\n");
}
/*
* Remove physical page from all pmaps in which it resides.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_remove_all(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pv_entry_t pv, pvn;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint8_t hold_flag;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
powerpc/pmap: Replace a logical TAILQ_FOREACH_SAFE with the real thing No functional change, just cleanup.
2020-03-30 16:32:55 +00:00
TAILQ_FOREACH_SAFE(pv, &m->md.pv_list, pv_link, pvn) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_LOCK(pv->pv_pmap);
hold_flag = PTBL_HOLD_FLAG(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte_remove(pv->pv_pmap, pv->pv_va, hold_flag);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pv->pv_pmap);
}
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_clear(m, PGA_WRITEABLE);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Map a range of physical addresses into kernel virtual address space.
*/
static vm_offset_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_map(vm_offset_t *virt, vm_paddr_t pa_start,
Fix physical address type to vm_paddr_t.
2012-05-24 21:13:24 +00:00
vm_paddr_t pa_end, int prot)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
vm_offset_t sva = *virt;
vm_offset_t va = sva;
powerpc/booke: Use the DMAP if possible in pmap_map() This avoids unnecessary TLB usage for statically mapped regions, such as vm_page_array.
2019-08-25 20:08:48 +00:00
#ifdef __powerpc64__
/* XXX: Handle memory not starting at 0x0. */
if (pa_end < ctob(Maxmem))
return (PHYS_TO_DMAP(pa_start));
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
while (pa_start < pa_end) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_kenter(va, pa_start);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
va += PAGE_SIZE;
pa_start += PAGE_SIZE;
}
*virt = va;
return (sva);
}
/*
* The pmap must be activated before it's address space can be accessed in any
* way.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_activate(struct thread *td)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pmap_t pmap;
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
u_int cpuid;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
pmap = &td->td_proc->p_vmspace->vm_pmap;
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
CTR5(KTR_PMAP, "%s: s (td = %p, proc = '%s', id = %d, pmap = 0x%"PRI0ptrX")",
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__func__, td, td->td_proc->p_comm, td->td_proc->p_pid, pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pmap != kernel_pmap), ("mmu_booke_activate: kernel_pmap!"));
Follow up r223485, which made AIM use the ABI thread pointer instead of PCPU fields for curthread, by doing the same to Book-E. This closes some potential races switching between CPUs. As a side effect, it turns out the AIM and Book-E swtch.S implementations were the same to within a few registers, so move that to powerpc/powerpc. MFC after: 3 months
2013-11-11 17:37:50 +00:00
sched_pin();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
cpuid = PCPU_GET(cpuid);
CPU_SET_ATOMIC(cpuid, &pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PCPU_SET(curpmap, pmap);
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
if (pmap->pm_tid[cpuid] == TID_NONE)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
tid_alloc(pmap);
/* Load PID0 register with pmap tid value. */
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
mtspr(SPR_PID0, pmap->pm_tid[cpuid]);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Move Book-E/AIM dependent bits for setting user PMAP during thread switch out of cpu_switch() and into pmap_activate() where they belong. This also removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
mtspr(SPR_DBCR0, td->td_pcb->pcb_cpu.booke.dbcr0);
Follow up r223485, which made AIM use the ABI thread pointer instead of PCPU fields for curthread, by doing the same to Book-E. This closes some potential races switching between CPUs. As a side effect, it turns out the AIM and Book-E swtch.S implementations were the same to within a few registers, so move that to powerpc/powerpc. MFC after: 3 months
2013-11-11 17:37:50 +00:00
sched_unpin();
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR3(KTR_PMAP, "%s: e (tid = %d for '%s')", __func__,
pmap->pm_tid[PCPU_GET(cpuid)], td->td_proc->p_comm);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Deactivate the specified process's address space.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_deactivate(struct thread *td)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pmap_t pmap;
pmap = &td->td_proc->p_vmspace->vm_pmap;
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
powerpc/booke: Fix debug printfs in pmap Add missing '%'s so printf formats are actually handled.
2018-11-28 04:02:26 +00:00
CTR5(KTR_PMAP, "%s: td=%p, proc = '%s', id = %d, pmap = 0x%"PRI0ptrX,
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__func__, td, td->td_proc->p_comm, td->td_proc->p_pid, pmap);
Move Book-E/AIM dependent bits for setting user PMAP during thread switch out of cpu_switch() and into pmap_activate() where they belong. This also removes all the #ifdef from cpu_switch().
2015-03-04 16:45:31 +00:00
td->td_pcb->pcb_cpu.booke.dbcr0 = mfspr(SPR_DBCR0);
Remove pc_other_cpus and pc_cpumask usage from powerpc support. Tested and reviewed by: andreast
2011-06-16 07:27:13 +00:00
CPU_CLR_ATOMIC(PCPU_GET(cpuid), &pmap->pm_active);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PCPU_SET(curpmap, NULL);
}
/*
* Copy the range specified by src_addr/len
* from the source map to the range dst_addr/len
* in the destination map.
*
* This routine is only advisory and need not do anything.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_copy(pmap_t dst_pmap, pmap_t src_pmap,
Fix Book-E/MPC85XX build. Some prototypes were wrong and got revealed with the recent kobj signature checking.
2009-06-13 08:57:04 +00:00
vm_offset_t dst_addr, vm_size_t len, vm_offset_t src_addr)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
}
/*
* Set the physical protection on the specified range of this map as requested.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_protect(pmap_t pmap, vm_offset_t sva, vm_offset_t eva,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t prot)
{
vm_offset_t va;
vm_page_t m;
pte_t *pte;
if ((prot & VM_PROT_READ) == VM_PROT_NONE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_remove(pmap, sva, eva);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return;
}
if (prot & VM_PROT_WRITE)
return;
PMAP_LOCK(pmap);
for (va = sva; va < eva; va += PAGE_SIZE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pmap, va)) != NULL) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISVALID(pte)) {
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Handle modified pages. */
MFamd64/i386 r207205 Clearing a page table entry's accessed bit and setting the page's PG_REFERENCED flag in pmap_protect() can't really be justified, so don't do it. Additionally, two changes that make this pmap behave like the others do: Change pmap_protect() such that it calls vm_page_dirty() only if the page is managed. Change pmap_remove_write() such that it doesn't clear a page table entry's accessed bit.
2010-04-30 15:22:52 +00:00
if (PTE_ISMODIFIED(pte) && PTE_ISMANAGED(pte))
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_UW | PTE_SW | PTE_MODIFIED);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
}
}
PMAP_UNLOCK(pmap);
}
/*
* Clear the write and modified bits in each of the given page's mappings.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_remove_write(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pv_entry_t pv;
pte_t *pte;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
On entry to pmap_enter(), assert that the page is busy. While I'm here, make the style of assertion used by pmap_enter() consistent across all architectures. On entry to pmap_remove_write(), assert that the page is neither unmanaged nor fictitious, since we cannot remove write access to either kind of page. With the push down of the page queues lock, pmap_remove_write() cannot condition its behavior on the state of the PG_WRITEABLE flag if the page is busy. Assert that the object containing the page is locked. This allows us to know that the page will neither become busy nor will PG_WRITEABLE be set on it while pmap_remove_write() is running. Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly do copy-on-write-based zero-copy transmit on unmanaged or fictitious pages, so don't even try. Previously, the call to pmap_remove_write() would have failed silently.
2010-05-16 23:45:10 +00:00
("mmu_booke_remove_write: page %p is not managed", m));
(6/6) Convert pmap to expect busy in write related operations now that all callers hold it. This simplifies pmap code and removes a dependency on the object lock. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21596
2019-10-15 03:51:46 +00:00
vm_page_assert_busied(m);
On entry to pmap_enter(), assert that the page is busy. While I'm here, make the style of assertion used by pmap_enter() consistent across all architectures. On entry to pmap_remove_write(), assert that the page is neither unmanaged nor fictitious, since we cannot remove write access to either kind of page. With the push down of the page queues lock, pmap_remove_write() cannot condition its behavior on the state of the PG_WRITEABLE flag if the page is busy. Assert that the object containing the page is locked. This allows us to know that the page will neither become busy nor will PG_WRITEABLE be set on it while pmap_remove_write() is running. Correct a long-standing bug in vm_page_cowsetup(). We cannot possibly do copy-on-write-based zero-copy transmit on unmanaged or fictitious pages, so don't even try. Previously, the call to pmap_remove_write() would have failed silently.
2010-05-16 23:45:10 +00:00
(6/6) Convert pmap to expect busy in write related operations now that all callers hold it. This simplifies pmap code and removes a dependency on the object lock. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21596
2019-10-15 03:51:46 +00:00
if (!pmap_page_is_write_mapped(m))
return;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISVALID(pte)) {
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Handle modified pages. */
Eliminate track_modified_needed(), better known as pmap_track_modified() on other platforms. We no longer need it because we do not create managed mappings within the clean submap. Pointed out by: alc
2008-04-27 21:04:54 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Flush mapping from TLB0. */
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_UW | PTE_SW | PTE_MODIFIED);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
}
PMAP_UNLOCK(pv->pv_pmap);
}
Split the vm_page flags PG_WRITEABLE and PG_REFERENCED into atomic flags field. Updates to the atomic flags are performed using the atomic ops on the containing word, do not require any vm lock to be held, and are non-blocking. The vm_page_aflag_set(9) and vm_page_aflag_clear(9) functions are provided to modify afalgs. Document the changes to flags field to only require the page lock. Introduce vm_page_reference(9) function to provide a stable KPI and KBI for filesystems like tmpfs and zfs which need to mark a page as referenced. Reviewed by: alc, attilio Tested by: marius, flo (sparc64); andreast (powerpc, powerpc64) Approved by: re (bz)
2011-09-06 10:30:11 +00:00
vm_page_aflag_clear(m, PGA_WRITEABLE);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Atomically extract and hold the physical page with the given
* pmap and virtual address pair if that mapping permits the given
* protection.
*/
static vm_page_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_extract_and_hold(pmap_t pmap, vm_offset_t va,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_prot_t prot)
{
pte_t *pte;
vm_page_t m;
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t pte_wbit;
Change synchonization rules for vm_page reference counting. There are several mechanisms by which a vm_page reference is held, preventing the page from being freed back to the page allocator. In particular, holding the page's object lock is sufficient to prevent the page from being freed; holding the busy lock or a wiring is sufficent as well. These references are protected by the page lock, which must therefore be acquired for many per-page operations. This results in false sharing since the page locks are external to the vm_page structures themselves and each lock protects multiple structures. Transition to using an atomically updated per-page reference counter. The object's reference is counted using a flag bit in the counter. A second flag bit is used to atomically block new references via pmap_extract_and_hold() while removing managed mappings of a page. Thus, the reference count of a page is guaranteed not to increase if the page is unbusied, unmapped, and the object's write lock is held. As a consequence of this, the page lock no longer protects a page's identity; operations which move pages between objects are now synchronized solely by the objects' locks. The vm_page_wire() and vm_page_unwire() KPIs are changed. The former requires that either the object lock or the busy lock is held. The latter no longer has a return value and may free the page if it releases the last reference to that page. vm_page_unwire_noq() behaves the same as before; the caller is responsible for checking its return value and freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is introduced for use in pmap_extract_and_hold(). It fails if the page is concurrently being unmapped, typically triggering a fallback to the fault handler. vm_page_wire() no longer requires the page lock and vm_page_unwire() now internally acquires the page lock when releasing the last wiring of a page (since the page lock still protects a page's queue state). In particular, synchronization details are no longer leaked into the caller. The change excises the page lock from several frequently executed code paths. In particular, vm_object_terminate() no longer bounces between page locks as it releases an object's pages, and direct I/O and sendfile(SF_NOCACHE) completions no longer require the page lock. In these latter cases we now get linear scalability in the common scenario where different threads are operating on different files. __FreeBSD_version is bumped. The DRM ports have been updated to accomodate the KPI changes. Reviewed by: jeff (earlier version) Tested by: gallatin (earlier version), pho Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20486
2019-09-09 21:32:42 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = NULL;
PMAP_LOCK(pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(pmap, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if ((pte != NULL) && PTE_ISVALID(pte)) {
if (pmap == kernel_pmap)
pte_wbit = PTE_SW;
else
pte_wbit = PTE_UW;
Change synchonization rules for vm_page reference counting. There are several mechanisms by which a vm_page reference is held, preventing the page from being freed back to the page allocator. In particular, holding the page's object lock is sufficient to prevent the page from being freed; holding the busy lock or a wiring is sufficent as well. These references are protected by the page lock, which must therefore be acquired for many per-page operations. This results in false sharing since the page locks are external to the vm_page structures themselves and each lock protects multiple structures. Transition to using an atomically updated per-page reference counter. The object's reference is counted using a flag bit in the counter. A second flag bit is used to atomically block new references via pmap_extract_and_hold() while removing managed mappings of a page. Thus, the reference count of a page is guaranteed not to increase if the page is unbusied, unmapped, and the object's write lock is held. As a consequence of this, the page lock no longer protects a page's identity; operations which move pages between objects are now synchronized solely by the objects' locks. The vm_page_wire() and vm_page_unwire() KPIs are changed. The former requires that either the object lock or the busy lock is held. The latter no longer has a return value and may free the page if it releases the last reference to that page. vm_page_unwire_noq() behaves the same as before; the caller is responsible for checking its return value and freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is introduced for use in pmap_extract_and_hold(). It fails if the page is concurrently being unmapped, typically triggering a fallback to the fault handler. vm_page_wire() no longer requires the page lock and vm_page_unwire() now internally acquires the page lock when releasing the last wiring of a page (since the page lock still protects a page's queue state). In particular, synchronization details are no longer leaked into the caller. The change excises the page lock from several frequently executed code paths. In particular, vm_object_terminate() no longer bounces between page locks as it releases an object's pages, and direct I/O and sendfile(SF_NOCACHE) completions no longer require the page lock. In these latter cases we now get linear scalability in the common scenario where different threads are operating on different files. __FreeBSD_version is bumped. The DRM ports have been updated to accomodate the KPI changes. Reviewed by: jeff (earlier version) Tested by: gallatin (earlier version), pho Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20486
2019-09-09 21:32:42 +00:00
if ((*pte & pte_wbit) != 0 || (prot & VM_PROT_WRITE) == 0) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
m = PHYS_TO_VM_PAGE(PTE_PA(pte));
Change synchonization rules for vm_page reference counting. There are several mechanisms by which a vm_page reference is held, preventing the page from being freed back to the page allocator. In particular, holding the page's object lock is sufficient to prevent the page from being freed; holding the busy lock or a wiring is sufficent as well. These references are protected by the page lock, which must therefore be acquired for many per-page operations. This results in false sharing since the page locks are external to the vm_page structures themselves and each lock protects multiple structures. Transition to using an atomically updated per-page reference counter. The object's reference is counted using a flag bit in the counter. A second flag bit is used to atomically block new references via pmap_extract_and_hold() while removing managed mappings of a page. Thus, the reference count of a page is guaranteed not to increase if the page is unbusied, unmapped, and the object's write lock is held. As a consequence of this, the page lock no longer protects a page's identity; operations which move pages between objects are now synchronized solely by the objects' locks. The vm_page_wire() and vm_page_unwire() KPIs are changed. The former requires that either the object lock or the busy lock is held. The latter no longer has a return value and may free the page if it releases the last reference to that page. vm_page_unwire_noq() behaves the same as before; the caller is responsible for checking its return value and freeing or enqueuing the page as appropriate. vm_page_wire_mapped() is introduced for use in pmap_extract_and_hold(). It fails if the page is concurrently being unmapped, typically triggering a fallback to the fault handler. vm_page_wire() no longer requires the page lock and vm_page_unwire() now internally acquires the page lock when releasing the last wiring of a page (since the page lock still protects a page's queue state). In particular, synchronization details are no longer leaked into the caller. The change excises the page lock from several frequently executed code paths. In particular, vm_object_terminate() no longer bounces between page locks as it releases an object's pages, and direct I/O and sendfile(SF_NOCACHE) completions no longer require the page lock. In these latter cases we now get linear scalability in the common scenario where different threads are operating on different files. __FreeBSD_version is bumped. The DRM ports have been updated to accomodate the KPI changes. Reviewed by: jeff (earlier version) Tested by: gallatin (earlier version), pho Sponsored by: Netflix Differential Revision: https://reviews.freebsd.org/D20486
2019-09-09 21:32:42 +00:00
if (!vm_page_wire_mapped(m))
m = NULL;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
}
PMAP_UNLOCK(pmap);
return (m);
}
/*
* Initialize a vm_page's machine-dependent fields.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_page_init(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Properly initialize the full md_page structure
2017-11-10 04:23:58 +00:00
m->md.pv_tracked = 0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_INIT(&m->md.pv_list);
}
/*
* Return whether or not the specified physical page was modified
* in any of physical maps.
*/
static boolean_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_is_modified(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
pv_entry_t pv;
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
boolean_t rv;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
("mmu_booke_is_modified: page %p is not managed", m));
rv = FALSE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
/*
(6/6) Convert pmap to expect busy in write related operations now that all callers hold it. This simplifies pmap code and removes a dependency on the object lock. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21596
2019-10-15 03:51:46 +00:00
* If the page is not busied then this check is racy.
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
*/
(6/6) Convert pmap to expect busy in write related operations now that all callers hold it. This simplifies pmap code and removes a dependency on the object lock. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21596
2019-10-15 03:51:46 +00:00
if (!pmap_page_is_write_mapped(m))
return (FALSE);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL &&
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
PTE_ISVALID(pte)) {
if (PTE_ISMODIFIED(pte))
rv = TRUE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
PMAP_UNLOCK(pv->pv_pmap);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
if (rv)
break;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
return (rv);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* Return whether or not the specified virtual address is eligible
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
* for prefault.
*/
static boolean_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_is_prefaultable(pmap_t pmap, vm_offset_t addr)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
return (FALSE);
}
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
/*
* Return whether or not the specified physical page was referenced
* in any physical maps.
*/
static boolean_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_is_referenced(vm_page_t m)
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
{
pte_t *pte;
pv_entry_t pv;
boolean_t rv;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Push down page queues lock acquisition in pmap_enter_object() and pmap_is_referenced(). Eliminate the corresponding page queues lock acquisitions from vm_map_pmap_enter() and mincore(), respectively. In mincore(), this allows some additional cases to complete without ever acquiring the page queues lock. Assert that the page is managed in pmap_is_referenced(). On powerpc/aim, push down the page queues lock acquisition from moea*_is_modified() and moea*_is_referenced() into moea*_query_bit(). Again, this will allow some additional cases to complete without ever acquiring the page queues lock. Reorder a few statements in vm_page_dontneed() so that a race can't lead to an old reference persisting. This scenario is described in detail by a comment. Correct a spelling error in vm_page_dontneed(). Assert that the object is locked in vm_page_clear_dirty(), and restrict the page queues lock assertion to just those cases in which the page is currently writeable. Add object locking to vnode_pager_generic_putpages(). This was the one and only place where vm_page_clear_dirty() was being called without the object being locked. Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call to vm_page_clear_dirty(). Change vnode_pager_generic_putpages() to the modern-style of function definition. Also, change the name of one of the parameters to follow virtual memory system naming conventions. Reviewed by: kib
2010-05-26 18:00:44 +00:00
("mmu_booke_is_referenced: page %p is not managed", m));
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
rv = FALSE;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL &&
Push down page queues lock acquisition in pmap_enter_object() and pmap_is_referenced(). Eliminate the corresponding page queues lock acquisitions from vm_map_pmap_enter() and mincore(), respectively. In mincore(), this allows some additional cases to complete without ever acquiring the page queues lock. Assert that the page is managed in pmap_is_referenced(). On powerpc/aim, push down the page queues lock acquisition from moea*_is_modified() and moea*_is_referenced() into moea*_query_bit(). Again, this will allow some additional cases to complete without ever acquiring the page queues lock. Reorder a few statements in vm_page_dontneed() so that a race can't lead to an old reference persisting. This scenario is described in detail by a comment. Correct a spelling error in vm_page_dontneed(). Assert that the object is locked in vm_page_clear_dirty(), and restrict the page queues lock assertion to just those cases in which the page is currently writeable. Add object locking to vnode_pager_generic_putpages(). This was the one and only place where vm_page_clear_dirty() was being called without the object being locked. Eliminate an unnecessary vm_page_lock() around vnode_pager_setsize()'s call to vm_page_clear_dirty(). Change vnode_pager_generic_putpages() to the modern-style of function definition. Also, change the name of one of the parameters to follow virtual memory system naming conventions. Reviewed by: kib
2010-05-26 18:00:44 +00:00
PTE_ISVALID(pte)) {
if (PTE_ISREFERENCED(pte))
rv = TRUE;
}
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
PMAP_UNLOCK(pv->pv_pmap);
if (rv)
break;
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Resurrect pmap_is_referenced() and use it in mincore(). Essentially, pmap_ts_referenced() is not always appropriate for checking whether or not pages have been referenced because it clears any reference bits that it encounters. For example, in mincore(), clearing the reference bits has two negative consequences. First, it throws off the activity count calculations performed by the page daemon. Specifically, a page on which mincore() has called pmap_ts_referenced() looks less active to the page daemon than it should. Consequently, the page could be deactivated prematurely by the page daemon. Arguably, this problem could be fixed by having mincore() duplicate the activity count calculation on the page. However, there is a second problem for which that is not a solution. In order to clear a reference on a 4KB page, it may be necessary to demote a 2/4MB page mapping. Thus, a mincore() by one process can have the side effect of demoting a superpage mapping within another process!
2010-04-24 17:32:52 +00:00
return (rv);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Clear the modify bits on the specified physical page.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_clear_modify(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
pv_entry_t pv;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
("mmu_booke_clear_modify: page %p is not managed", m));
(6/6) Convert pmap to expect busy in write related operations now that all callers hold it. This simplifies pmap code and removes a dependency on the object lock. Reviewed by: kib, markj Tested by: pho Sponsored by: Netflix, Intel Differential Revision: https://reviews.freebsd.org/D21596
2019-10-15 03:51:46 +00:00
vm_page_assert_busied(m);
if (!pmap_page_is_write_mapped(m))
return;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL &&
Roughly half of a typical pmap_mincore() implementation is machine- independent code. Move this code into mincore(), and eliminate the page queues lock from pmap_mincore(). Push down the page queues lock into pmap_clear_modify(), pmap_clear_reference(), and pmap_is_modified(). Assert that these functions are never passed an unmanaged page. Eliminate an inaccurate comment from powerpc/powerpc/mmu_if.m: Contrary to what the comment says, pmap_mincore() is not simply an optimization. Without a complete pmap_mincore() implementation, mincore() cannot return either MINCORE_MODIFIED or MINCORE_REFERENCED because only the pmap can provide this information. Eliminate the page queues lock from vfs_setdirty_locked_object(), vm_pageout_clean(), vm_object_page_collect_flush(), and vm_object_page_clean(). Generally speaking, these are all accesses to the page's dirty field, which are synchronized by the containing vm object's lock. Reduce the scope of the page queues lock in vm_object_madvise() and vm_page_dontneed(). Reviewed by: kib (an earlier version)
2010-05-24 14:26:57 +00:00
PTE_ISVALID(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
if (*pte & (PTE_SW | PTE_UW | PTE_MODIFIED)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(pv->pv_va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~(PTE_SW | PTE_UW | PTE_MODIFIED |
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PTE_REFERENCED);
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return a count of reference bits for a page, clearing those bits.
* It is not necessary for every reference bit to be cleared, but it
* is necessary that 0 only be returned when there are truly no
* reference bits set.
*
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
* As an optimization, update the page's dirty field if a modified bit is
* found while counting reference bits. This opportunistic update can be
* performed at low cost and can eliminate the need for some future calls
* to pmap_is_modified(). However, since this function stops after
* finding PMAP_TS_REFERENCED_MAX reference bits, it may not detect some
* dirty pages. Those dirty pages will only be detected by a future call
* to pmap_is_modified().
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_ts_referenced(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pte_t *pte;
pv_entry_t pv;
int count;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
("mmu_booke_ts_referenced: page %p is not managed", m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
count = 0;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL &&
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
PTE_ISVALID(pte)) {
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
if (PTE_ISMODIFIED(pte))
vm_page_dirty(m);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISREFERENCED(pte)) {
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_lock_spin(&tlbivax_mutex);
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_lock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tlb0_flush_entry(pv->pv_va);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~PTE_REFERENCED;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
tlb_miss_unlock();
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_unlock_spin(&tlbivax_mutex);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Various changes to pmap_ts_referenced() Move PMAP_TS_REFERENCED_MAX out of the various pmap implementations and into vm/pmap.h, and describe what its purpose is. Eliminate the archaic "XXX" comment about its value. I don't believe that its exact value, e.g., 5 versus 6, matters. Update the arm64 and riscv pmap implementations of pmap_ts_referenced() to opportunistically update the page's dirty field. On amd64, use the PDE value already cached in a local variable rather than dereferencing a pointer again and again. Reviewed by: kib, markj MFC after: 2 weeks Differential Revision: https://reviews.freebsd.org/D7836
2016-09-10 16:49:25 +00:00
if (++count >= PMAP_TS_REFERENCED_MAX) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
PMAP_UNLOCK(pv->pv_pmap);
break;
}
}
}
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
}
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
/*
* Clear the wired attribute from the mappings for the specified range of
* addresses in the given pmap. Every valid mapping within that range must
* have the wired attribute set. In contrast, invalid mappings cannot have
* the wired attribute set, so they are ignored.
*
* The wired attribute of the page table entry is not a hardware feature, so
* there is no need to invalidate any TLB entries.
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_unwire(pmap_t pmap, vm_offset_t sva, vm_offset_t eva)
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
{
vm_offset_t va;
pte_t *pte;
PMAP_LOCK(pmap);
for (va = sva; va < eva; va += PAGE_SIZE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pmap, va)) != NULL &&
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
PTE_ISVALID(pte)) {
if (!PTE_ISWIRED(pte))
panic("mmu_booke_unwire: pte %p isn't wired",
pte);
Migrate the PTE format for book-e to standardize on the 'indirect PTE' format Summary: The revised Book-E spec, adding the specification for the MMUv2 and e6500, includes a hardware PTE layout for indirect page tables. In order to support this in the future, migrate the PTE format to match the MMUv2 hardware PTE format. Test Plan: Boot tested on a P5020 board. Booted to multiuser mode. Differential Revision: https://reviews.freebsd.org/D5224
2016-02-11 13:15:37 +00:00
*pte &= ~PTE_WIRED;
Implement pmap_unwire(). See r268327 for the motivation behind this change.
2014-07-13 16:27:57 +00:00
pmap->pm_stats.wired_count--;
}
}
PMAP_UNLOCK(pmap);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Return true if the pmap's pv is one of the first 16 pvs linked to from this
* page. This count may be changed upwards or downwards in the future; it is
* only necessary that true be returned for a small subset of pmaps for proper
* page aging.
*/
static boolean_t
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_page_exists_quick(pmap_t pmap, vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pv_entry_t pv;
int loops;
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
boolean_t rv;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
KASSERT((m->oflags & VPO_UNMANAGED) == 0,
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
("mmu_booke_page_exists_quick: page %p is not managed", m));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
loops = 0;
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
rv = FALSE;
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
if (pv->pv_pmap == pmap) {
rv = TRUE;
break;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (++loops >= 16)
break;
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Reduce the scope of the page queues lock and the number of PG_REFERENCED changes in vm_pageout_object_deactivate_pages(). Simplify this function's inner loop using TAILQ_FOREACH(), and shorten some of its overly long lines. Update a stale comment. Assert that PG_REFERENCED may be cleared only if the object containing the page is locked. Add a comment documenting this. Assert that a caller to vm_page_requeue() holds the page queues lock, and assert that the page is on a page queue. Push down the page queues lock into pmap_ts_referenced() and pmap_page_exists_quick(). (As of now, there are no longer any pmap functions that expect to be called with the page queues lock held.) Neither pmap_ts_referenced() nor pmap_page_exists_quick() should ever be passed an unmanaged page. Assert this rather than returning "0" and "FALSE" respectively. ARM: Simplify pmap_page_exists_quick() by switching to TAILQ_FOREACH(). Push down the page queues lock inside of pmap_clearbit(), simplifying pmap_clear_modify(), pmap_clear_reference(), and pmap_remove_write(). Additionally, this allows for avoiding the acquisition of the page queues lock in some cases. PowerPC/AIM: moea*_page_exits_quick() and moea*_page_wired_mappings() will never be called before pmap initialization is complete. Therefore, the check for moea_initialized can be eliminated. Push down the page queues lock inside of moea*_clear_bit(), simplifying moea*_clear_modify() and moea*_clear_reference(). The last parameter to moea*_clear_bit() is never used. Eliminate it. PowerPC/BookE: Simplify mmu_booke_page_exists_quick()'s control flow. Reviewed by: kib@
2010-06-10 16:56:35 +00:00
return (rv);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return the number of managed mappings to the given physical page that are
* wired.
*/
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_page_wired_mappings(vm_page_t m)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
pv_entry_t pv;
pte_t *pte;
int count = 0;
- Move the PG_UNMANAGED flag from m->flags to m->oflags, renaming the flag to VPO_UNMANAGED (and also making the flag protected by the vm object lock, instead of vm page queue lock). - Mark the fake pages with both PG_FICTITIOUS (as it is now) and VPO_UNMANAGED. As a consequence, pmap code now can use use just VPO_UNMANAGED to decide whether the page is unmanaged. Reviewed by: alc Tested by: pho (x86, previous version), marius (sparc64), marcel (arm, ia64, powerpc), ray (mips) Sponsored by: The FreeBSD Foundation Approved by: re (bz)
2011-08-09 21:01:36 +00:00
if ((m->oflags & VPO_UNMANAGED) != 0)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
TAILQ_FOREACH(pv, &m->md.pv_list, pv_link) {
PMAP_LOCK(pv->pv_pmap);
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
if ((pte = pte_find(pv->pv_pmap, pv->pv_va)) != NULL)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (PTE_ISVALID(pte) && PTE_ISWIRED(pte))
count++;
PMAP_UNLOCK(pv->pv_pmap);
}
Replace all uses of the page queues lock by a R/W lock that is private to this pmap. Eliminate two redundant #include's. Tested by: marcel
2012-11-03 23:22:49 +00:00
rw_wunlock(&pvh_global_lock);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (count);
}
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_dev_direct_mapped(vm_paddr_t pa, vm_size_t size)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
int i;
vm_offset_t va;
/*
* This currently does not work for entries that
* overlap TLB1 entries.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i ++) {
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (tlb1_iomapped(i, pa, size, &va) == 0)
return (0);
}
return (EFAULT);
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_dumpsys_map(vm_paddr_t pa, size_t sz, void **va)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
vm_paddr_t ppa;
vm_offset_t ofs;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
vm_size_t gran;
/* Minidumps are based on virtual memory addresses. */
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
if (do_minidump) {
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
*va = (void *)(vm_offset_t)pa;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
return;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
/* Raw physical memory dumps don't have a virtual address. */
/* We always map a 256MB page at 256M. */
gran = 256 * 1024 * 1024;
sys: use our roundup2/rounddown2() macros when param.h is available. rounddown2 tends to produce longer lines than the original code and when the code has a high indentation level it was not really advantageous to do the replacement. This tries to strike a balance between readability using the macros and flexibility of having the expressions, so not everything is converted.
2016-04-21 19:57:40 +00:00
ppa = rounddown2(pa, gran);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
ofs = pa - ppa;
*va = (void *)gran;
tlb1_set_entry((vm_offset_t)va, ppa, gran, _TLB_ENTRY_IO);
if (sz > (gran - ofs))
tlb1_set_entry((vm_offset_t)(va + gran), ppa + gran, gran,
_TLB_ENTRY_IO);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_dumpsys_unmap(vm_paddr_t pa, size_t sz, void *va)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
vm_paddr_t ppa;
vm_offset_t ofs;
vm_size_t gran;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
int i;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
/* Minidumps are based on virtual memory addresses. */
/* Nothing to do... */
if (do_minidump)
return;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
break;
}
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
/* Raw physical memory dumps don't have a virtual address. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
i--;
e.mas1 = 0;
e.mas2 = 0;
e.mas3 = 0;
tlb1_write_entry(&e, i);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
gran = 256 * 1024 * 1024;
sys: use our roundup2/rounddown2() macros when param.h is available. rounddown2 tends to produce longer lines than the original code and when the code has a high indentation level it was not really advantageous to do the replacement. This tries to strike a balance between readability using the macros and flexibility of having the expressions, so not everything is converted.
2016-04-21 19:57:40 +00:00
ppa = rounddown2(pa, gran);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
ofs = pa - ppa;
if (sz > (gran - ofs)) {
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
i--;
e.mas1 = 0;
e.mas2 = 0;
e.mas3 = 0;
tlb1_write_entry(&e, i);
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
extern struct dump_pa dump_map[PHYS_AVAIL_SZ + 1];
void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_scan_init()
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
{
vm_offset_t va;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
pte_t *pte;
int i;
if (!do_minidump) {
/* Initialize phys. segments for dumpsys(). */
memset(&dump_map, 0, sizeof(dump_map));
mem_regions(&physmem_regions, &physmem_regions_sz, &availmem_regions,
&availmem_regions_sz);
for (i = 0; i < physmem_regions_sz; i++) {
dump_map[i].pa_start = physmem_regions[i].mr_start;
dump_map[i].pa_size = physmem_regions[i].mr_size;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
return;
}
/* Virtual segments for minidumps: */
memset(&dump_map, 0, sizeof(dump_map));
/* 1st: kernel .data and .bss. */
dump_map[0].pa_start = trunc_page((uintptr_t)_etext);
dump_map[0].pa_size =
round_page((uintptr_t)_end) - dump_map[0].pa_start;
/* 2nd: msgbuf and tables (see pmap_bootstrap()). */
dump_map[1].pa_start = data_start;
dump_map[1].pa_size = data_end - data_start;
/* 3rd: kernel VM. */
va = dump_map[1].pa_start + dump_map[1].pa_size;
/* Find start of next chunk (from va). */
while (va < virtual_end) {
/* Don't dump the buffer cache. */
if (va >= kmi.buffer_sva && va < kmi.buffer_eva) {
va = kmi.buffer_eva;
continue;
}
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
if (pte != NULL && PTE_ISVALID(pte))
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
break;
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
va += PAGE_SIZE;
}
if (va < virtual_end) {
dump_map[2].pa_start = va;
va += PAGE_SIZE;
/* Find last page in chunk. */
while (va < virtual_end) {
/* Don't run into the buffer cache. */
if (va == kmi.buffer_sva)
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
break;
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
if (pte == NULL || !PTE_ISVALID(pte))
break;
va += PAGE_SIZE;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
Factor out duplicated code from dumpsys() on each architecture into generic code in sys/kern/kern_dump.c. Most dumpsys() implementations are nearly identical and simply redefine a number of constants and helper subroutines; a generic implementation will make it easier to implement features around kernel core dumps. This change does not alter any minidump code and should have no functional impact. PR: 193873 Differential Revision: https://reviews.freebsd.org/D904 Submitted by: Conrad Meyer <conrad.meyer@isilon.com> Reviewed by: jhibbits (earlier version) Sponsored by: EMC / Isilon Storage Division
2015-01-07 01:01:39 +00:00
dump_map[2].pa_size = va - dump_map[2].pa_start;
Implement kernel core dump support for Book-E processors. Both raw physical memory dumps and virtual minidumps are supported. The default being minidumps. Obtained from: Juniper Networks
2009-04-04 22:01:43 +00:00
}
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Map a set of physical memory pages into the kernel virtual address space.
* Return a pointer to where it is mapped. This routine is intended to be used
* for mapping device memory, NOT real memory.
*/
static void *
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_mapdev(vm_paddr_t pa, vm_size_t size)
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
{
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
return (mmu_booke_mapdev_attr(pa, size, VM_MEMATTR_DEFAULT));
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
}
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
static int
tlb1_find_pa(vm_paddr_t pa, tlb_entry_t *e)
{
int i;
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(e, i);
if ((e->mas1 & MAS1_VALID) == 0)
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
continue;
if (e->phys == pa)
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
return (i);
}
return (-1);
}
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
static void *
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_mapdev_attr(vm_paddr_t pa, vm_size_t size, vm_memattr_t ma)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
vm_paddr_t tmppa;
powerpc/booke: Use a pseudo-DMAP for the device mappings on booke64 Since powerpc64 has such a large virtual address space, significantly larger than its physical address space, take advantage of this, and create yet another DMAP-like instance for the device mappings. In this case, the device mapping "DMAP" is in the 0x8000000000000000 - 0xc000000000000000 range, so as not to overlap the physical memory DMAP. This will allow us to add TLB1 entry coalescing in the future, especially useful for things like the radeonkms driver, which maps parts of the GPU at a time, but eventually maps all of it, using up a lot of TLB1 entries (~40).
2020-02-25 01:40:22 +00:00
#ifndef __powerpc64__
uintptr_t tmpva;
#endif
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
uintptr_t va, retva;
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
vm_size_t sz;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
int i;
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
int wimge;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
/*
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
* Check if this is premapped in TLB1.
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
*/
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
sz = size;
tmppa = pa;
va = ~0;
wimge = tlb_calc_wimg(pa, ma);
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
if (wimge != (e.mas2 & (MAS2_WIMGE_MASK & ~_TLB_ENTRY_SHARED)))
continue;
if (tmppa >= e.phys && tmppa < e.phys + e.size) {
va = e.virt + (pa - e.phys);
tmppa = e.phys + e.size;
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
sz -= MIN(sz, e.size - (pa - e.phys));
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
while (sz > 0 && (i = tlb1_find_pa(tmppa, &e)) != -1) {
if (wimge != (e.mas2 & (MAS2_WIMGE_MASK & ~_TLB_ENTRY_SHARED)))
break;
sz -= MIN(sz, e.size);
tmppa = e.phys + e.size;
}
if (sz != 0)
break;
return ((void *)va);
}
1. Have the APs initialize the TLB1 entries from what has been programmed on the BSP during (early) boot. This makes sure that the APs get configured the same as the BSP, irrspective of how FreeBSD was loaded. 2. Make sure to flush the dcache after writing the TLB1 entries to the boot page. The APs aren't part of the coherency domain just yet. 3. Set pmap_bootstrapped after calling pmap_bootstrap(). The FDT code now maps the devices (like OF), and this resulted in a panic. 4. Since we pre-wire the CCSR, make sure not to map chunks of it in pmap_mapdev().
2012-11-03 22:02:12 +00:00
}
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size = roundup(size, PAGE_SIZE);
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
powerpc/booke: Use a pseudo-DMAP for the device mappings on booke64 Since powerpc64 has such a large virtual address space, significantly larger than its physical address space, take advantage of this, and create yet another DMAP-like instance for the device mappings. In this case, the device mapping "DMAP" is in the 0x8000000000000000 - 0xc000000000000000 range, so as not to overlap the physical memory DMAP. This will allow us to add TLB1 entry coalescing in the future, especially useful for things like the radeonkms driver, which maps parts of the GPU at a time, but eventually maps all of it, using up a lot of TLB1 entries (~40).
2020-02-25 01:40:22 +00:00
#ifdef __powerpc64__
KASSERT(pa < VM_MAPDEV_PA_MAX,
("Unsupported physical address! %lx", pa));
va = VM_MAPDEV_BASE + pa;
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
retva = va;
#ifdef POW2_MAPPINGS
/*
* Align the mapping to a power of 2 size, taking into account that we
* may need to increase the size multiple times to satisfy the size and
* alignment requirements.
*
* This works in the general case because it's very rare (near never?)
* to have different access properties (WIMG) within a single
* power-of-two region. If a design does call for that, POW2_MAPPINGS
* can be undefined, and exact mappings will be used instead.
*/
sz = size;
size = roundup2(size, 1 << ilog2(size));
while (rounddown2(va, size) + size < va + sz)
size <<= 1;
va = rounddown2(va, size);
pa = rounddown2(pa, size);
#endif
powerpc/booke: Use a pseudo-DMAP for the device mappings on booke64 Since powerpc64 has such a large virtual address space, significantly larger than its physical address space, take advantage of this, and create yet another DMAP-like instance for the device mappings. In this case, the device mapping "DMAP" is in the 0x8000000000000000 - 0xc000000000000000 range, so as not to overlap the physical memory DMAP. This will allow us to add TLB1 entry coalescing in the future, especially useful for things like the radeonkms driver, which maps parts of the GPU at a time, but eventually maps all of it, using up a lot of TLB1 entries (~40).
2020-02-25 01:40:22 +00:00
#else
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
/*
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
* The device mapping area is between VM_MAXUSER_ADDRESS and
* VM_MIN_KERNEL_ADDRESS. This gives 1GB of device addressing.
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
*/
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#ifdef SPARSE_MAPDEV
/*
* With a sparse mapdev, align to the largest starting region. This
* could feasibly be optimized for a 'best-fit' alignment, but that
* calculation could be very costly.
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
* Align to the smaller of:
* - first set bit in overlap of (pa & size mask)
* - largest size envelope
*
* It's possible the device mapping may start at a PA that's not larger
* than the size mask, so we need to offset in to maximize the TLB entry
* range and minimize the number of used TLB entries.
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
*/
do {
tmpva = tlb1_map_base;
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
sz = ffsl((~((1 << flsl(size-1)) - 1)) & pa);
Book-E pmap_mapdev_attr() improvements * Check TLB1 in all mapdev cases, in case the memattr matches an existing mapping (doesn't need to be MAP_DEFAULT). * Fix mapping where the starting address is not a multiple of the widest size base. For instance, it will now properly map 0xffffef000, size 0x11000 using 2 TLB entries, basing it at 0x****f000, instead of 0x***00000. MFC after: 2 weeks
2017-11-10 04:14:48 +00:00
sz = sz ? min(roundup(sz + 3, 4), flsl(size) - 1) : flsl(size) - 1;
va = roundup(tlb1_map_base, 1 << sz) | (((1 << sz) - 1) & pa);
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
} while (!atomic_cmpset_int(&tlb1_map_base, tmpva, va + size));
Introduce 64-bit PowerPC Book-E support Extend the Book-E pmap to support 64-bit operation. Much of this was taken from Juniper's Junos FreeBSD port. It uses a 3-level page table (page directory list -- PP2D, page directory, page table), but has gaps in the page directory list where regions will repeat, due to the design of the PP2D hash (a 20-bit gap between the two parts of the index). In practice this may not be a problem given the expanded address space. However, an alternative to this would be to use a 4-level page table, like Linux, and possibly reduce the available address space; Linux appears to use a 46-bit address space. Alternatively, a cache of page directory pointers could be used to keep the overall design as-is, but remove the gaps in the address space. This includes a new kernel config for 64-bit QorIQ SoCs, based on MPC85XX, with the following notes: * The DPAA driver has not yet been ported to 64-bit so is not included in the kernel config. * This has been tested on the AmigaOne X5000, using a MD_ROOT compiled in (total size kernel+mdroot must be under 64MB). * This can run both 32-bit and 64-bit processes, and has even been tested to run a 32-bit init with 64-bit children. Many thanks to stevek and marcel for getting Juniper's FreeBSD patches open sourced to be used here, and to stevek for reviewing, and providing some historical contexts on quirks of the code. Reviewed by: stevek Obtained from: Juniper (in part) MFC after: 2 months Relnotes: yes Differential Revision: https://reviews.freebsd.org/D9433
2017-03-17 21:40:14 +00:00
#endif
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
va = atomic_fetchadd_int(&tlb1_map_base, size);
retva = va;
Restructure device mappings for Book-E. Summary: There is currently a 1GB hole between user and kernel address spaces into which direct (1:1 PA:VA) device mappings go. This appears to go largely unused, leaving all devices to contend with the 128MB block at the end of the 32-bit space (0xf8000000-0xffffffff). This easily fills up, and needs to be densely packed. However, dense packing wastes precious TLB1 space, of which there are only 16 (e500v2) or 64(e5500) entries available. Change this by using the 1GB space for all device mappings, and allow the kernel to use the entire upper 1GB for KVA. This also allows us to use sparse device mappings, freeing up TLB entries. Test Plan: Boot tested on p5020. Differential Revision: https://reviews.freebsd.org/D5832
2016-04-10 15:48:09 +00:00
#endif
Add pmap_mapdev_attr() and pmap_kenter_attr() interfaces. pmap_set_memattr() is slightly more complicated and is left unimplemented for now. Also prevent pmap_mapdev() from mapping over the kernel and KVA regions if devices happen to have high physical addresses. MFC after: 2 weeks
2013-10-26 14:52:55 +00:00
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
if (tlb1_mapin_region(va, pa, size, tlb_calc_wimg(pa, ma)) != size)
return (NULL);
In mmu_booke_mapdev(), handle mappings that cannot be represented by a single TLB entry. The boot ROM on the MPC85555CDS is 8MB, for example, and in order to map that we need 2 4MB TLB entries.
2008-10-25 03:36:21 +00:00
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
return ((void *)retva);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* 'Unmap' a range mapped by mmu_booke_mapdev().
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_unmapdev(vm_offset_t va, vm_size_t size)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
#ifdef SUPPORTS_SHRINKING_TLB1
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_offset_t base, offset;
/*
* Unmap only if this is inside kernel virtual space.
*/
if ((va >= VM_MIN_KERNEL_ADDRESS) && (va <= VM_MAX_KERNEL_ADDRESS)) {
base = trunc_page(va);
offset = va & PAGE_MASK;
size = roundup(offset + size, PAGE_SIZE);
Revert r240317 to prevent leaking pmap entries Subsequent to r240317, kmem_free() was replaced with kva_free() (r254025). kva_free() releases the KVA allocation for the mapped region, but no longer clears the pmap (pagetable) entries. An affected pmap_unmapdev operation would leave the still-pmap'd VA space free for allocation by other KVA consumers. However, this bug easily avoided notice for ~7 years because most devices (1) never call pmap_unmapdev and (2) on amd64, mostly fit within the DMAP and do not need KVA allocations. Other affected arch are less popular: i386, MIPS, and PowerPC. Arm64, arm32, and riscv are not affected. Reported by: Don Morris <dgmorris AT earthlink.net> Submitted by: Don Morris (amd64 part) Reviewed by: kib, markj, Don (!amd64 parts) MFC after: I don't intend to, but you might want to Sponsored by: Dell Isilon Differential Revision: https://reviews.freebsd.org/D25689
2020-07-16 23:29:26 +00:00
mmu_booke_qremove(base, atop(size));
Replace kernel virtual address space allocation with vmem. This provides transparent layering and better fragmentation. - Normalize functions that allocate memory to use kmem_* - Those that allocate address space are named kva_* - Those that operate on maps are named kmap_* - Implement recursive allocation handling for kmem_arena in vmem. Reviewed by: alc Tested by: pho Sponsored by: EMC / Isilon Storage Division
2013-08-07 06:21:20 +00:00
kva_free(base, size);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
* mmu_booke_object_init_pt preloads the ptes for a given object into the
* specified pmap. This eliminates the blast of soft faults on process startup
* and immediately after an mmap.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_object_init_pt(pmap_t pmap, vm_offset_t addr,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_object_t object, vm_pindex_t pindex, vm_size_t size)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
Switch the vm_object mutex to be a rwlock. This will enable in the future further optimizations where the vm_object lock will be held in read mode most of the time the page cache resident pool of pages are accessed for reading purposes. The change is mostly mechanical but few notes are reported: * The KPI changes as follow: - VM_OBJECT_LOCK() -> VM_OBJECT_WLOCK() - VM_OBJECT_TRYLOCK() -> VM_OBJECT_TRYWLOCK() - VM_OBJECT_UNLOCK() -> VM_OBJECT_WUNLOCK() - VM_OBJECT_LOCK_ASSERT(MA_OWNED) -> VM_OBJECT_ASSERT_WLOCKED() (in order to avoid visibility of implementation details) - The read-mode operations are added: VM_OBJECT_RLOCK(), VM_OBJECT_TRYRLOCK(), VM_OBJECT_RUNLOCK(), VM_OBJECT_ASSERT_RLOCKED(), VM_OBJECT_ASSERT_LOCKED() * The vm/vm_pager.h namespace pollution avoidance (forcing requiring sys/mutex.h in consumers directly to cater its inlining functions using VM_OBJECT_LOCK()) imposes that all the vm/vm_pager.h consumers now must include also sys/rwlock.h. * zfs requires a quite convoluted fix to include FreeBSD rwlocks into the compat layer because the name clash between FreeBSD and solaris versions must be avoided. At this purpose zfs redefines the vm_object locking functions directly, isolating the FreeBSD components in specific compat stubs. The KPI results heavilly broken by this commit. Thirdy part ports must be updated accordingly (I can think off-hand of VirtualBox, for example). Sponsored by: EMC / Isilon storage division Reviewed by: jeff Reviewed by: pjd (ZFS specific review) Discussed with: alc Tested by: pho
2013-03-09 02:32:23 +00:00
VM_OBJECT_ASSERT_WLOCKED(object);
Add a new type of VM object: OBJT_SG. An OBJT_SG object is very similar to a device pager (OBJT_DEVICE) object in that it uses fictitious pages to provide aliases to other memory addresses. The primary difference is that it uses an sglist(9) to determine the physical addresses for a given offset into the object instead of invoking the d_mmap() method in a device driver. Reviewed by: alc Approved by: re (kensmith) MFC after: 2 weeks
2009-07-24 13:50:29 +00:00
KASSERT(object->type == OBJT_DEVICE || object->type == OBJT_SG,
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
("mmu_booke_object_init_pt: non-device object"));
}
/*
* Perform the pmap work for mincore.
*/
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_mincore(pmap_t pmap, vm_offset_t addr, vm_paddr_t *pap)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Do not panic if pmap_mincore() is called. This prevents crashing userland binaries from bringing down the kernel. MFC after: 3 days
2013-11-06 14:36:38 +00:00
/* XXX: this should be implemented at some point */
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
static int
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_change_attr(vm_offset_t addr, vm_size_t sz, vm_memattr_t mode)
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
{
vm_offset_t va;
pte_t *pte;
int i, j;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
powerpc/booke: Fix pmap_mapdev_attr() for multi-TLB1 entry mappings Also, fix pmap_change_attr() to ignore non-kernel mappings. * Fix a masking bug in mmu_booke_mapdev_attr() which caused it to align mappings to the smallest mapping alignment, instead of the largest. This caused mappings to be potentially pessimally aligned, using more TLB entries than necessary. * Return existing mappings from mmu_booke_mapdev_attr() that span more than one TLB1 entry. The drm-current-kmod drivers map discontiguous segments of the GPU, resulting in more than one TLB entry being used to satisfy the mapping. * Ignore non-kernel mappings in mmu_booke_change_attr(). There's a bug in the linuxkpi layer that causes it to actually try to change physical address mappings, instead of virtual addresses. amd64 doesn't encounter this because it ignores non-kernel mappings. With this it's possible to use drm-current-kmod on Book-E.
2019-11-06 04:40:12 +00:00
addr = trunc_page(addr);
/* Only allow changes to mapped kernel addresses. This includes:
* - KVA
* - DMAP (powerpc64)
* - Device mappings
*/
if (addr <= VM_MAXUSER_ADDRESS ||
#ifdef __powerpc64__
(addr >= tlb1_map_base && addr < DMAP_BASE_ADDRESS) ||
(addr > DMAP_MAX_ADDRESS && addr < VM_MIN_KERNEL_ADDRESS) ||
#else
(addr >= tlb1_map_base && addr < VM_MIN_KERNEL_ADDRESS) ||
#endif
(addr > VM_MAX_KERNEL_ADDRESS))
return (EINVAL);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/* Check TLB1 mappings */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
continue;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (addr >= e.virt && addr < e.virt + e.size)
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
break;
}
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (i < TLB1_ENTRIES) {
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/* Only allow full mappings to be modified for now. */
/* Validate the range. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (j = i, va = addr; va < addr + sz; va += e.size, j++) {
tlb1_read_entry(&e, j);
if (va != e.virt || (sz - (va - addr) < e.size))
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
return (EINVAL);
}
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (va = addr; va < addr + sz; va += e.size, i++) {
tlb1_read_entry(&e, i);
e.mas2 &= ~MAS2_WIMGE_MASK;
e.mas2 |= tlb_calc_wimg(e.phys, mode);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
/*
* Write it out to the TLB. Should really re-sync with other
* cores.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_write_entry(&e, i);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
}
return (0);
}
/* Not in TLB1, try through pmap */
/* First validate the range. */
for (va = addr; va < addr + sz; va += PAGE_SIZE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
if (pte == NULL || !PTE_ISVALID(pte))
return (EINVAL);
}
mtx_lock_spin(&tlbivax_mutex);
tlb_miss_lock();
for (va = addr; va < addr + sz; va += PAGE_SIZE) {
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
pte = pte_find(kernel_pmap, va);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
*pte &= ~(PTE_MAS2_MASK << PTE_MAS2_SHIFT);
Fix a typo (move parenthesis to correct location in the line). Before this, it would cause the one consumer of this API in powerpc usage (dev/dpaa) to set the PTE WIMG flags to empty instead of --M-, making the cache-enabled buffer portals non-coherent.
2016-12-04 02:15:46 +00:00
*pte |= tlb_calc_wimg(PTE_PA(pte), mode) << PTE_MAS2_SHIFT;
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
tlb0_flush_entry(va);
}
tlb_miss_unlock();
mtx_unlock_spin(&tlbivax_mutex);
Correct the return value for pmap_change_attr() pmap_change_attr() returns an error code, not a paddr. This function is currently unused for powerpc. MFC after: 2 weeks
2017-02-21 05:08:07 +00:00
return (0);
Implement pmap_change_attr() for PowerPC (Book-E only for now) Summary: Some drivers need special memory requirements. X86 solves this with a pmap_change_attr() API, which DRM uses for changing the mapping of the GART and other memory regions. Implement the same function for PowerPC. AIM currently does not need this, but will in the future for DRM, so a default is added for that, for business as usual. Book-E has some drivers coming down that do require non-default memory coherency. In this case, the Datapath Acceleration Architecture (DPAA) based ethernet controller has 2 regions for the buffer portals: cache-inhibited, and cache-enabled. By default, device memory is cache-inhibited. If the cache-enabled memory regions are mapped cache-inhibited, an alignment exception is thrown on access. Test Plan: Tested with a new driver to be added after this (DPAA dTSEC ethernet driver). No alignment exceptions thrown, driver works as expected with this. Reviewed By: nwhitehorn Sponsored by: Alex Perez/Inertial Computing Differential Revision: https://reviews.freebsd.org/D5471
2016-02-27 20:39:36 +00:00
}
powerpc/pmap: NUMA-ize vm_page_array on powerpc Summary: This matches r351198 from amd64. This only applies to AIM64 and Book-E. On AIM64 it short-circuits with one domain, to behave similar to existing. Otherwise it will allocate 16MB huge pages to hold the page array, across all NUMA domains. On the first domain it will shift the page array base up, to "upper-align" the page array in that domain, so as to reduce the number of pages from the next domain appearing in this domain. After the first domain, subsequent domains will be allocated in full 16MB pages, until the final domain, which can be short. This means some inner domains may have pages accounted in earlier domains. On Book-E the page array is setup at MMU bootstrap time so that it's always mapped in TLB1, on both 32-bit and 64-bit. This reduces the TLB0 overhead for touching the vm_page_array, which reduces up to one TLB miss per array access. Since page_range (vm_page_startup()) is no longer used on Book-E but is on 32-bit AIM, mark the variable as potentially unused, rather than using a nasty #if defined() list. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21449
2019-12-07 03:34:03 +00:00
static void
powerpc/mmu: Convert PowerPC pmap drivers to ifunc from kobj With IFUNC support in the kernel, we can finally get rid of our poor-man's ifunc for pmap, utilizing kobj. Since moea64 uses a second tier kobj as well, for its own private methods, this adds a second pmap install function (pmap_mmu_init()) to perform pmap 'post-install pre-bootstrap' initialization, before the IFUNCs get initialized. Reviewed by: bdragon
2020-05-27 01:24:12 +00:00
mmu_booke_page_array_startup(long pages)
powerpc/pmap: NUMA-ize vm_page_array on powerpc Summary: This matches r351198 from amd64. This only applies to AIM64 and Book-E. On AIM64 it short-circuits with one domain, to behave similar to existing. Otherwise it will allocate 16MB huge pages to hold the page array, across all NUMA domains. On the first domain it will shift the page array base up, to "upper-align" the page array in that domain, so as to reduce the number of pages from the next domain appearing in this domain. After the first domain, subsequent domains will be allocated in full 16MB pages, until the final domain, which can be short. This means some inner domains may have pages accounted in earlier domains. On Book-E the page array is setup at MMU bootstrap time so that it's always mapped in TLB1, on both 32-bit and 64-bit. This reduces the TLB0 overhead for touching the vm_page_array, which reduces up to one TLB miss per array access. Since page_range (vm_page_startup()) is no longer used on Book-E but is on 32-bit AIM, mark the variable as potentially unused, rather than using a nasty #if defined() list. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21449
2019-12-07 03:34:03 +00:00
{
vm_page_array_size = pages;
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/**************************************************************************/
/* TID handling */
/**************************************************************************/
/*
* Allocate a TID. If necessary, steal one from someone else.
* The new TID is flushed from the TLB before returning.
*/
static tlbtid_t
tid_alloc(pmap_t pmap)
{
tlbtid_t tid;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
int thiscpu;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((pmap != kernel_pmap), ("tid_alloc: kernel pmap"));
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: s (pmap = %p)", __func__, pmap);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
thiscpu = PCPU_GET(cpuid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
tid = PCPU_GET(booke.tid_next);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
if (tid > TID_MAX)
tid = TID_MIN;
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
PCPU_SET(booke.tid_next, tid + 1);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* If we are stealing TID then clear the relevant pmap's field */
if (tidbusy[thiscpu][tid] != NULL) {
CTR2(KTR_PMAP, "%s: warning: stealing tid %d", __func__, tid);
tidbusy[thiscpu][tid]->pm_tid[thiscpu] = TID_NONE;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Flush all entries from TLB0 matching this TID. */
Rewrite tid_flush() in C. There's no need for it to be in asm. Also, by writing in C, and marking it static in pmap.c, it saves a branch to the function itself, as it's only used in one location. The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
tid_flush(tid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tidbusy[thiscpu][tid] = pmap;
pmap->pm_tid[thiscpu] = tid;
__asm __volatile("msync; isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR3(KTR_PMAP, "%s: e (%02d next = %02d)", __func__, tid,
Merge AIM and Book-E PCPU fields This is part of a long-term goal of merging Book-E and AIM into a single GENERIC kernel. As more work is done, the struct may be optimized further. Reviewed by: nwhitehorn
2018-02-17 20:59:12 +00:00
PCPU_GET(booke.tid_next));
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
return (tid);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/**************************************************************************/
/* TLB0 handling */
/**************************************************************************/
/* Convert TLB0 va and way number to tlb0[] table index. */
static inline unsigned int
tlb0_tableidx(vm_offset_t va, unsigned int way)
{
unsigned int idx;
idx = (way * TLB0_ENTRIES_PER_WAY);
idx += (va & MAS2_TLB0_ENTRY_IDX_MASK) >> MAS2_TLB0_ENTRY_IDX_SHIFT;
return (idx);
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Invalidate TLB0 entry.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
static inline void
tlb0_flush_entry(vm_offset_t va)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR2(KTR_PMAP, "%s: s va=0x%08x", __func__, va);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
mtx_assert(&tlbivax_mutex, MA_OWNED);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
__asm __volatile("tlbivax 0, %0" :: "r"(va & MAS2_EPN_MASK));
__asm __volatile("isync; msync");
__asm __volatile("tlbsync; msync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
CTR1(KTR_PMAP, "%s: e", __func__);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/**************************************************************************/
/* TLB1 handling */
/**************************************************************************/
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/*
* TLB1 mapping notes:
*
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
* TLB1[0] Kernel text and data.
* TLB1[1-15] Additional kernel text and data mappings (if required), PCI
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* windows, other devices mappings.
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
/*
* Read an entry from given TLB1 slot.
*/
void
tlb1_read_entry(tlb_entry_t *entry, unsigned int slot)
{
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
register_t msr;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
uint32_t mas0;
KASSERT((entry != NULL), ("%s(): Entry is NULL!", __func__));
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
msr = mfmsr();
Use the more common Book-E idiom for disabling interrupts. Book-E has the wrteei/wrtee instructions for writing the PSL_EE bit, ignoring all others. Use this instead of the AIM-typical mtmsr. MFC with: r320392
2017-06-30 02:11:32 +00:00
__asm __volatile("wrteei 0");
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(slot);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync; tlbre");
entry->mas1 = mfspr(SPR_MAS1);
entry->mas2 = mfspr(SPR_MAS2);
entry->mas3 = mfspr(SPR_MAS3);
switch ((mfpvr() >> 16) & 0xFFFF) {
case FSL_E500v2:
case FSL_E500mc:
case FSL_E5500:
Take into account mas7/8 when reading/writing TLB entries on e6500 Summary: Current booke/pmap code ignores mas7 and mas8 on e6500 CPU. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7606
2016-08-23 04:26:30 +00:00
case FSL_E6500:
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry->mas7 = mfspr(SPR_MAS7);
break;
default:
entry->mas7 = 0;
break;
}
powerpc/booke: Use wrtee instead of msr to restore EE bit The MSR[EE] bit does not require synchronization when changing. This is a trivial micro-optimization, removing the trailing isync from mtmsr(). MFC after: 1 week
2019-05-22 02:43:17 +00:00
__asm __volatile("wrtee %0" :: "r"(msr));
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry->virt = entry->mas2 & MAS2_EPN_MASK;
entry->phys = ((vm_paddr_t)(entry->mas7 & MAS7_RPN) << 32) |
(entry->mas3 & MAS3_RPN);
entry->size =
tsize2size((entry->mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT);
}
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
struct tlbwrite_args {
tlb_entry_t *e;
unsigned int idx;
};
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
static uint32_t
tlb1_find_free(void)
{
tlb_entry_t e;
int i;
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if ((e.mas1 & MAS1_VALID) == 0)
return (i);
}
return (-1);
}
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
static void
tlb1_purge_va_range(vm_offset_t va, vm_size_t size)
{
tlb_entry_t e;
int i;
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if ((e.mas1 & MAS1_VALID) == 0)
continue;
if ((e.mas2 & MAS2_EPN_MASK) >= va &&
(e.mas2 & MAS2_EPN_MASK) < va + size) {
mtspr(SPR_MAS1, e.mas1 & ~MAS1_VALID);
__asm __volatile("isync; tlbwe; isync; msync");
}
}
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
static void
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
tlb1_write_entry_int(void *arg)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
struct tlbwrite_args *args = arg;
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
uint32_t idx, mas0;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
idx = args->idx;
if (idx == -1) {
powerpc/booke: Use power-of-two mappings in 64-bit pmap_mapdev Summary: This reduces the precious TLB1 entry consumption (64 possible in existing 64-bit cores), by adjusting the size and alignment of a device mapping to a power of 2, to encompass the full mapping and its surroundings. One caveat with this: If a mapping really is smaller than a power of 2, it's possible to get a machine check or hang if the 'missing' physical space is accessed. In practice this should not be an issue for users, as devices overwhelmingly have physical spaces on power-of-two sizes and alignments, and any design that includes devices which don't follow this can be addressed by undefining the POW2_MAPPINGS guard. Reviewed by: bdragon Differential Revision: https://reviews.freebsd.org/D24248
2020-04-11 00:17:55 +00:00
tlb1_purge_va_range(args->e->virt, args->e->size);
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
idx = tlb1_find_free();
if (idx == -1)
panic("No free TLB1 entries!\n");
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Select entry */
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(idx);
Disable interrupts when updating the TLB Without disabling interrupts it's possible for another thread to preempt and update the registers post-read (tlb1_read_entry) or pre-write (tlb1_write_entry), and confuse the kernel with mixed register states. MFC after: 2 weeks
2017-06-27 01:57:22 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtspr(SPR_MAS0, mas0);
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS1, args->e->mas1);
mtspr(SPR_MAS2, args->e->mas2);
mtspr(SPR_MAS3, args->e->mas3);
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
switch ((mfpvr() >> 16) & 0xFFFF) {
case FSL_E500mc:
case FSL_E5500:
Take into account mas7/8 when reading/writing TLB entries on e6500 Summary: Current booke/pmap code ignores mas7 and mas8 on e6500 CPU. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7606
2016-08-23 04:26:30 +00:00
case FSL_E6500:
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
mtspr(SPR_MAS8, 0);
/* FALLTHROUGH */
case FSL_E500v2:
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
mtspr(SPR_MAS7, args->e->mas7);
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
break;
default:
break;
}
powerpc/booke: depessimize MAS register updates We only need to isync before we actually use the MAS registers, so before and after the TLB read/write/sync/search operations. MFC after: 2 weeks
2019-02-16 04:38:34 +00:00
__asm __volatile("isync; tlbwe; isync; msync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
}
static void
tlb1_write_entry_sync(void *arg)
{
/* Empty synchronization point for smp_rendezvous(). */
}
/*
* Write given entry to TLB1 hardware.
*/
static void
tlb1_write_entry(tlb_entry_t *e, unsigned int idx)
{
struct tlbwrite_args args;
args.e = e;
args.idx = idx;
#ifdef SMP
if ((e->mas2 & _TLB_ENTRY_SHARED) && smp_started) {
mb();
smp_rendezvous(tlb1_write_entry_sync,
tlb1_write_entry_int,
tlb1_write_entry_sync, &args);
} else
#endif
{
register_t msr;
msr = mfmsr();
__asm __volatile("wrteei 0");
tlb1_write_entry_int(&args);
powerpc/booke: Use wrtee instead of msr to restore EE bit The MSR[EE] bit does not require synchronization when changing. This is a trivial micro-optimization, removing the trailing isync from mtmsr(). MFC after: 1 week
2019-05-22 02:43:17 +00:00
__asm __volatile("wrtee %0" :: "r"(msr));
Synchronize TLB1 mappings when created This allows modules creating mappings to be loaded post-boot, after SMP has started. Without this, the TLB1 mappings can become unsynchronized and lead to kernel page faults when accessed on the alternate CPUs. MFC after: 3 weeks
2017-11-26 20:30:02 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Convert TLB TSIZE value to mapped region size.
*/
static vm_size_t
tsize2size(unsigned int tsize)
{
/*
* size = 4^tsize KB
* size = 4^tsize * 2^10 = 2^(2 * tsize - 10)
*/
return ((1 << (2 * tsize)) * 1024);
}
/*
* Convert region size (must be power of 4) to TLB TSIZE value.
*/
static unsigned int
size2tsize(vm_size_t size)
{
return (ilog2(size) / 2 - 5);
}
/*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Register permanent kernel mapping in TLB1.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
* Entries are created starting from index 0 (current free entry is
* kept in tlb1_idx) and are not supposed to be invalidated.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
Add support for the Freescale dTSEC DPAA-based ethernet controller. Freescale's QorIQ line includes a new ethernet controller, based on their Datapath Acceleration Architecture (DPAA). This uses a combination of a Frame manager, Buffer manager, and Queue manager to improve performance across all interfaces by being able to pass data directly between hardware acceleration interfaces. As part of this import, Freescale's Netcomm Software (ncsw) driver is imported. This was an attempt by Freescale to create an OS-agnostic sub-driver for managing the hardware, using shims to interface to the OS-specific APIs. This work was abandoned, and Freescale's primary work is in the Linux driver (dual BSD/GPL license). Hence, this was imported directly to sys/contrib, rather than going through the vendor area. Going forward, FreeBSD-specific changes may be made to the ncsw code, diverging from the upstream in potentially incompatible ways. An alternative could be to import the Linux driver itself, using the linuxKPI layer, as that would maintain parity with the vendor-maintained driver. However, the Linux driver has not been evaluated for reliability yet, and may have issues with the import, whereas the ncsw-based driver in this commit was completed by Semihalf 4 years ago, and is very stable. Other SoC modules based on DPAA, which could be added in the future: * Security and Encryption engine (SEC4.x, SEC5.x) * RAID engine Additional work to be done: * Implement polling mode * Test vlan support * Add support for the Pattern Matching Engine, which can do regular expression matching on packets. This driver has been tested on the P5020 QorIQ SoC. Others listed in the dtsec(4) manual page are expected to work as the same DPAA engine is included in all. Obtained from: Semihalf Relnotes: Yes Sponsored by: Alex Perez/Inertial Computing
2016-02-29 03:38:00 +00:00
int
Use the correct type for physical addresses. On Book-E, physical addresses are actually 36-bits, not 32-bits. This is currently worked around by ignoring the top bits. However, in some cases, the boot loader configures CCSR to something above the 32-bit mark. This is stage 1 in updating the pmap to handle 36-bit physaddr.
2015-07-04 19:00:38 +00:00
tlb1_set_entry(vm_offset_t va, vm_paddr_t pa, vm_size_t size,
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t flags)
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
uint32_t ts, tid;
Fix concurrency issues with TLB1 updates and make pmap_kextract() search TLB1 mappings as well, which is required for the console to work after r257111.
2013-10-26 16:49:41 +00:00
int tsize, index;
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
/* First try to update an existing entry. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (index = 0; index < TLB1_ENTRIES; index++) {
tlb1_read_entry(&e, index);
/* Check if we're just updating the flags, and update them. */
if (e.phys == pa && e.virt == va && e.size == size) {
e.mas2 = (va & MAS2_EPN_MASK) | flags;
tlb1_write_entry(&e, index);
return (0);
}
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Convert size to TSIZE */
tsize = size2tsize(size);
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
tid = (TID_KERNEL << MAS1_TID_SHIFT) & MAS1_TID_MASK;
/* XXX TS is hard coded to 0 for now as we only use single address space */
ts = (0 << MAS1_TS_SHIFT) & MAS1_TS_MASK;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
e.phys = pa;
e.virt = va;
e.size = size;
e.mas1 = MAS1_VALID | MAS1_IPROT | ts | tid;
e.mas1 |= ((tsize << MAS1_TSIZE_SHIFT) & MAS1_TSIZE_MASK);
e.mas2 = (va & MAS2_EPN_MASK) | flags;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Rework BookE pmap towards multi-core support. o Eliminate tlb0[] (a s/w copy of TLB0) - The table contents cannot be maintained reliably in multiple MMU environments, where asynchronous events (invalidations from other cores) can change our local TLB0 contents underneath. - Simplify and optimize TLB flushing: system wide invalidations are performed using tlbivax instruction (propagates to other cores), for local MMU invalidations a new optimized routine (assembly) is introduced. o Improve and simplify TID allocation and management. - Let each core keep track of its TID allocations. - Simplify TID recycling, eliminate dead code. - Drop the now unused powerpc/booke/support.S file. o Improve page tables management logic. o Simplify TLB1 manipulation routines. o Other improvements and polishing. Obtained from: Freescale, Semihalf
2009-01-13 15:41:58 +00:00
/* Set supervisor RWX permission bits */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
e.mas3 = (pa & MAS3_RPN) | MAS3_SR | MAS3_SW | MAS3_SX;
e.mas7 = (pa >> 32) & MAS7_RPN;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/booke: Fix TLB1 entry accounting It's possible, with per-CPU mappings, for TLB1 indices to get out of sync. This presents a problem when trying to insert an entry into TLB1 of all CPUs. Currently that's done by assuming (hoping) that the TLBs are perfectly synced, and inserting to the same index for all CPUs. However, with aforementioned private mappings, this can result in overwriting mappings on the other CPUs. An example: CPU0 CPU1 <setup all mappings> <idle> 3 private mappings kick off CPU 1 initialize shared mappings (3 indices low) Load kernel module, triggers 20 new mappings Sync mappings at N-3 initialize 3 private mappings. At this point, CPU 1 has all the correct mappings, while CPU 0 is missing 3 mappings that were shared across to CPU 1. When CPU 0 tries to access memory in one of the overwritten mappings, it hangs while tripping through the TLB miss handler. Device mappings are not stored in any page table. This fixes by introducing a '-1' index for tlb1_write_entry_int(), so each CPU searches for an available index private to itself. MFC after: 3 weeks
2019-11-01 02:55:58 +00:00
tlb1_write_entry(&e, -1);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
/*
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
* Map in contiguous RAM region into the TLB1.
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*/
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
static vm_size_t
tlb1_mapin_region(vm_offset_t va, vm_paddr_t pa, vm_size_t size, int wimge)
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
{
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
vm_offset_t base;
vm_size_t mapped, sz, ssize;
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
mapped = 0;
base = va;
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
ssize = size;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
while (size > 0) {
sz = 1UL << (ilog2(size) & ~1);
/* Align size to PA */
if (pa % sz != 0) {
do {
sz >>= 2;
} while (pa % sz != 0);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
}
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
/* Now align from there to VA */
if (va % sz != 0) {
do {
sz >>= 2;
} while (va % sz != 0);
}
[PowerPC] Fix Book-E direct map for >=16G ram on e5500 It turns out the maximum TLB1 page size on e5500 is 4G, despite the format being defined for up to 1TB. So, we need to clamp the DMAP TLB1 entries to not attempt to create 16G or larger entries. Fixes boot on my X5000 in which I just installed 16G of RAM. Reviewed by: jhibbits Sponsored by: Tag1 Consulting, Inc. Differential Revision: https://reviews.freebsd.org/D23244
2020-01-18 01:22:54 +00:00
#ifdef __powerpc64__
/*
* Clamp TLB1 entries to 4G.
*
* While the e6500 supports up to 1TB mappings, the e5500
* only supports up to 4G mappings. (0b1011)
*
* If any e6500 machines capable of supporting a very
* large amount of memory appear in the future, we can
* revisit this.
*
* For now, though, since we have plenty of space in TLB1,
* always avoid creating entries larger than 4GB.
*/
sz = MIN(sz, 1UL << 32);
#endif
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
if (bootverbose)
printf("Wiring VA=%p to PA=%jx (size=%lx)\n",
(void *)va, (uintmax_t)pa, (long)sz);
if (tlb1_set_entry(va, pa, sz,
_TLB_ENTRY_SHARED | wimge) < 0)
return (mapped);
size -= sz;
pa += sz;
va += sz;
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
mapped = (va - base);
booke pmap: hide debug-ish printf behind bootverbose It's not necessary during normal operation to know the mapped region size and wasted space.
2018-08-19 18:54:43 +00:00
if (bootverbose)
printf("mapped size 0x%"PRIxPTR" (wasted space 0x%"PRIxPTR")\n",
powerpc/pmap: Make use of tlb1_mapin_region in pmap_mapdev_attr() tlb1_mapin_region() and pmap_mapdev_attr() do roughly the same thing -- map a chunk of physical address space(memory or MMIO) into virtual, but do it in differing ways. Unify the code, settling on pmap_mapdev_attr()'s algorithm, to simplify and unify the logic. This fixes a bug with growing the kernel mappings in mmu_booke_bootstrap(), where part of the mapping was not getting done, leading to a hang when the unmapped VAs were accessed.
2019-11-04 00:35:40 +00:00
mapped, mapped - ssize);
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
return (mapped);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* TLB1 initialization routine, to be called after the very first
* assembler level setup done in locore.S.
*/
void
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
tlb1_init()
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
{
powerpc: Link Book-E kernels at the same address as AIM kernels Summary: Reduce the diff between AIM and Book-E even more. This also cleans up vmparam.h significantly. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21301
2019-08-20 01:26:02 +00:00
vm_offset_t mas2;
uint32_t mas0, mas1, mas3, mas7;
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
uint32_t tsz;
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
tlb1_get_tlbconf();
Add support for Juniper's loader. The difference between FreeBSD's and Juniper's loader is that Juniper's loader maps all of the kernel and preloaded modules at the right virtual address before jumping into the kernel. FreeBSD's loader simply maps 16MB using the physical address and expects the kernel to jump through hoops to relocate itself to it's virtual address. The problem with the FreeBSD loader's approach is that it typically maps too much or too little. There's no harm if it's too much (other than wasting space), but if it's too little then the kernel will simply not boot, because the first thing the kernel needs is the bootinfo structure, which is never mapped in that case. The page fault that early is fatal. The changes constitute: 1. Do not remap the kernel in locore.S. We're mapped where we need to be so we can pretty much call into C code after setting up the stack. 2. With kernload and kernload_ap not set in locore.S, we need to set them in pmap.c: kernload gets defined when we preserve the TLB1. Here we also determine the size of the kernel mapped. kernload_ap is set first thing in the pmap_bootstrap() method. 3. Fix tlb1_map_region() and its use to properly externd the mapped kernel size to include low-level data structures. Approved by: re (blanket) Obtained from: Juniper Networks, Inc
2011-08-02 15:35:43 +00:00
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(0);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync; tlbre");
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
kernload = ((vm_paddr_t)(mas7 & MAS7_RPN) << 32) |
Extend pmap to support e500mc and e5500. As part of this, clean up tlb1_init(), since bootinfo is always NULL here just eliminate the loop altogether. Also, fix a bug in mmu_booke_mapdev_attr() where it's possible to map a larger immediately following a smaller page, causing the mappings to overlap. Instead, break up the new mapping into smaller chunks. The downside to this is that it uses more precious TLB1 entries, which, on smaller chips (e500v2) it could cause problems with TLB1 being out of space (e500v2 only has 16 TLB1 entries). Obtained from: Semihalf (partial) Sponsored by: Alex Perez/Inertial Computing
2015-08-28 03:03:09 +00:00
(mas3 & MAS3_RPN);
tsz = (mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
kernsize += (tsz > 0) ? tsize2size(tsz) : 0;
powerpc: Link Book-E kernels at the same address as AIM kernels Summary: Reduce the diff between AIM and Book-E even more. This also cleans up vmparam.h significantly. Reviewed by: luporl Differential Revision: https://reviews.freebsd.org/D21301
2019-08-20 01:26:02 +00:00
kernstart = trunc_page(mas2);
Invalidate any TLB1 entries we don't need. The firmware (e.g. U-Boot) may have added entries that conflict with TLB0 entries.
2012-07-02 21:15:56 +00:00
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Setup TLB miss defaults */
set_mas4_defaults();
}
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
/*
* pmap_early_io_unmap() should be used in short conjunction with
* pmap_early_io_map(), as in the following snippet:
*
* x = pmap_early_io_map(...);
* <do something with x>
* pmap_early_io_unmap(x, size);
*
* And avoiding more allocations between.
*/
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
void
pmap_early_io_unmap(vm_offset_t va, vm_size_t size)
{
int i;
tlb_entry_t e;
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
vm_size_t isize;
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
size = roundup(size, PAGE_SIZE);
isize = size;
for (i = 0; i < TLB1_ENTRIES && size > 0; i++) {
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
continue;
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
if (va <= e.virt && (va + isize) >= (e.virt + e.size)) {
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
size -= e.size;
e.mas1 &= ~MAS1_VALID;
tlb1_write_entry(&e, i);
}
}
Allow pmap_early_io_unmap() to reclaim memory pmap_early_io_map()/pmap_early_io_unmap(), if used in pairs, should be used in the form: pmap_early_io_map() ..do stuff.. pmap_early_io_unmap() Without other allocations in the middle. Without reclaiming memory this can leave large holes in the device space. While here, make a simple change to the unmap loop which now permits it to unmap multiple TLB entries in the range.
2016-09-07 03:26:55 +00:00
if (tlb1_map_base == va + isize)
tlb1_map_base -= isize;
Fix system hang when large FDT is in use Summary: Kernel maps only one page of FDT. When FDT is more than one page in size, data TLB miss occurs on memmove() when FDT is moved to kernel storage (sys/powerpc/booke/booke_machdep.c, booke_init()) This introduces a pmap_early_io_unmap() to complement pmap_early_io_map(), which can be used for any early I/O mapping, but currently is only used when mapping the fdt. Submitted by: Ivan Krivonos <int0dster_gmail.com> Differential Revision: https://reviews.freebsd.org/D7605
2016-08-24 03:51:40 +00:00
}
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
vm_offset_t
pmap_early_io_map(vm_paddr_t pa, vm_size_t size)
{
vm_paddr_t pa_base;
vm_offset_t va, sz;
int i;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
KASSERT(!pmap_bootstrapped, ("Do not use after PMAP is up!"));
powerpc: clean up empty lines in .c and .h files
2020-09-01 21:20:08 +00:00
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
for (i = 0; i < TLB1_ENTRIES; i++) {
tlb1_read_entry(&e, i);
if (!(e.mas1 & MAS1_VALID))
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
continue;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (pa >= e.phys && (pa + size) <=
(e.phys + e.size))
return (e.virt + (pa - e.phys));
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
}
trunc_page() goes through unsigned long, which is too short. sizeof(unsigned long) < sizeof(vm_paddr_t) on Book-E, which uses 36-bit addressing. With this, a CCSR with a physical address above 4GB successfully maps. Sponsored by: Alex Perez/Inertial Computing
2015-11-21 06:03:46 +00:00
pa_base = rounddown(pa, PAGE_SIZE);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size = roundup(size + (pa - pa_base), PAGE_SIZE);
Make sure that TLB1 mappings are aligned correctly.
2013-11-17 01:59:42 +00:00
tlb1_map_base = roundup2(tlb1_map_base, 1 << (ilog2(size) & ~1));
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
va = tlb1_map_base + (pa - pa_base);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
do {
sz = 1 << (ilog2(size) & ~1);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_set_entry(tlb1_map_base, pa_base, sz,
_TLB_ENTRY_SHARED | _TLB_ENTRY_IO);
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
size -= sz;
pa_base += sz;
Make devices with registers into the KVA region work reliably. Without this, previous KVA allocations (which the PMAP lazily invalidates) in TLB0 could shadow device maps in TLB1. Add a big block comment about some of the caveats with this approach.
2013-10-26 20:57:26 +00:00
tlb1_map_base += sz;
Interrelated improvements to early boot mappings: - Remove explicit requirement that the SOC registers be found except as an optimization (although the MPC85XX LAW drivers still require they be found externally, which should change). - Remove magic CCSRBAR_VA value. - Allow bus_machdep.c's early-boot code to handle non 1:1 mappings and systems not in real-mode or global 1:1 maps in early boot. - Allow pmap_mapdev() on Book-E to reissue previous addresses if the area is already mapped. Additionally have it check all mappings, not just the CCSR area. This allows the console on e500 systems to actually work on systems where the boot loader was not kind enough to set up a 1:1 mapping before starting the kernel.
2013-10-26 18:18:14 +00:00
} while (size > 0);
return (va);
}
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
void
pmap_track_page(pmap_t pmap, vm_offset_t va)
{
vm_paddr_t pa;
vm_page_t page;
struct pv_entry *pve;
Use trunc_page() instead of rolling my own in pmap_track_page()
2016-12-05 02:27:50 +00:00
va = trunc_page(va);
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
pa = pmap_kextract(va);
Retrieve the page outside of holding locks pmap_track_page() only works with physical memory pages, which have a constant vm_page_t address. Microoptimize pmap_track_page() to perform one less operation under the lock.
2017-12-10 04:43:27 +00:00
page = PHYS_TO_VM_PAGE(pa);
Simplify the page tracking for VA<->PA translations. Drop the tracking down to the pmap layer, with optimizations to only track necessary pages. This should give a (slight) performance improvement, as well as a stability improvement, as the tracking is already mostly handled by the pmap layer.
2016-11-16 05:24:42 +00:00
rw_wlock(&pvh_global_lock);
PMAP_LOCK(pmap);
TAILQ_FOREACH(pve, &page->md.pv_list, pv_link) {
if ((pmap == pve->pv_pmap) && (va == pve->pv_va)) {
goto out;
}
}
page->md.pv_tracked = true;
pv_insert(pmap, va, page);
out:
PMAP_UNLOCK(pmap);
rw_wunlock(&pvh_global_lock);
}
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/*
* Setup MAS4 defaults.
* These values are loaded to MAS0-2 on a TLB miss.
*/
static void
set_mas4_defaults(void)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t mas4;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Defaults: TLB0, PID0, TSIZED=4K */
mas4 = MAS4_TLBSELD0;
mas4 |= (TLB_SIZE_4K << MAS4_TSIZED_SHIFT) & MAS4_TSIZED_MASK;
Initial support for SMP on PowerPC MPC85xx. Tested with Freescale dual-core MPC8572DS development system. Obtained from: Freescale, Semihalf
2009-05-21 11:43:37 +00:00
#ifdef SMP
mas4 |= MAS4_MD;
#endif
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
mtspr(SPR_MAS4, mas4);
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
__asm __volatile("isync");
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
}
/*
* Return 0 if the physical IO range is encompassed by one of the
* the TLB1 entries, otherwise return related error code.
*/
static int
tlb1_iomapped(int i, vm_paddr_t pa, vm_size_t size, vm_offset_t *va)
{
Clean up BookE pmap. Improve comments, eliminate redundant debug output, fix style(9) and other minor tweaks for code readability. Obtained from: Freescale, Semihalf
2009-01-13 16:15:49 +00:00
uint32_t prot;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
vm_paddr_t pa_start;
vm_paddr_t pa_end;
unsigned int entry_tsize;
vm_size_t entry_size;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb_entry_t e;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
*va = (vm_offset_t)NULL;
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
tlb1_read_entry(&e, i);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
/* Skip invalid entries */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
if (!(e.mas1 & MAS1_VALID))
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (EINVAL);
/*
* The entry must be cache-inhibited, guarded, and r/w
* so it can function as an i/o page
*/
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
prot = e.mas2 & (MAS2_I | MAS2_G);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (prot != (MAS2_I | MAS2_G))
return (EPERM);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
prot = e.mas3 & (MAS3_SR | MAS3_SW);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if (prot != (MAS3_SR | MAS3_SW))
return (EPERM);
/* The address should be within the entry range. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
entry_tsize = (e.mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
KASSERT((entry_tsize), ("tlb1_iomapped: invalid entry tsize"));
entry_size = tsize2size(entry_tsize);
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
pa_start = (((vm_paddr_t)e.mas7 & MAS7_RPN) << 32) |
(e.mas3 & MAS3_RPN);
Enhance book-e pmap for 36-bit physaddr Summary: This is (probably step 1) of enhancing the book-e pmap to support the full 36-bit physical address space on Freescale e500 and e5500 cores. Thus far it has only been regression tested on one platform. Since I only have one other Book-E platform (e5500), that needs work beyond this, I haven't yet tested it on this. Test Plan: Regression tested on my RouterBoard RB800. Reviewed By: marcel Differential Revision: https://reviews.freebsd.org/D3027
2015-08-22 07:20:03 +00:00
pa_end = pa_start + entry_size;
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
if ((pa < pa_start) || ((pa + size) > pa_end))
return (ERANGE);
/* Return virtual address of this mapping. */
Fix SMP booting for PowerPC Book-E Summary: PowerPC Book-E SMP is currently broken for unknown reasons. Pull in Semihalf changes made c2012 for e500mc/e5500, which enables booting SMP. This eliminates the shared software TLB1 table, replacing it with tlb1_read_entry() function. This does not yet support ePAPR SMP booting, and doesn't handle resetting CPUs already released (ePAPR boot releases APs to a spin loop waiting on a specific address). This will be addressed in the near future by using the MPIC to reset the AP into our own alternate boot address. This does include a change to the dpaa/dtsec(4) driver, to mark the portals as CPU-private. Test Plan: Tested on Amiga X5000/20 (P5020). Boots, prints the following messages: Adding CPU 0, pir=0, awake=1 Waking up CPU 1 (dev=1) Adding CPU 1, pir=20, awake=1 SMP: AP CPU #1 launched top(1) shows CPU1 active. Obtained from: Semihalf Relnotes: Yes Differential Revision: https://reviews.freebsd.org/D5945
2016-04-19 01:48:18 +00:00
*va = (e.mas2 & MAS2_EPN_MASK) + (pa - pa_start);
Initial support for Freescale PowerQUICC III MPC85xx system-on-chip family. The PQ3 is a high performance integrated communications processing system based on the e500 core, which is an embedded RISC processor that implements the 32-bit Book E definition of the PowerPC architecture. For details refer to: http://www.freescale.com/webapp/sps/site/prod_summary.jsp?code=MPC8555E This port was tested and successfully run on the following members of the PQ3 family: MPC8533, MPC8541, MPC8548, MPC8555. The following major integrated peripherals are supported: * On-chip peripherals bus * OpenPIC interrupt controller * UART * Ethernet (TSEC) * Host/PCI bridge * QUICC engine (SCC functionality) This commit brings the main functionality and will be followed by individual drivers that are logically separate from this base. Approved by: cognet (mentor) Obtained from: Juniper, Semihalf MFp4: e500
2008-03-03 17:17:00 +00:00
return (0);
}
Rewrite tid_flush() in C. There's no need for it to be in asm. Also, by writing in C, and marking it static in pmap.c, it saves a branch to the function itself, as it's only used in one location. The generated asm is virtually identical to the handwritten code.
2015-12-30 02:23:14 +00:00
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
#ifdef DDB
/* Print out contents of the MAS registers for each TLB0 entry */
static void
#ifdef __powerpc64__
tlb_print_entry(int i, uint32_t mas1, uint64_t mas2, uint32_t mas3,
#else
tlb_print_entry(int i, uint32_t mas1, uint32_t mas2, uint32_t mas3,
#endif
uint32_t mas7)
{
int as;
char desc[3];
tlbtid_t tid;
vm_size_t size;
unsigned int tsize;
desc[2] = '\0';
if (mas1 & MAS1_VALID)
desc[0] = 'V';
else
desc[0] = ' ';
if (mas1 & MAS1_IPROT)
desc[1] = 'P';
else
desc[1] = ' ';
as = (mas1 & MAS1_TS_MASK) ? 1 : 0;
tid = MAS1_GETTID(mas1);
tsize = (mas1 & MAS1_TSIZE_MASK) >> MAS1_TSIZE_SHIFT;
size = 0;
if (tsize)
size = tsize2size(tsize);
printf("%3d: (%s) [AS=%d] "
powerpc/booke pmap: Fix printf format type warnings
2019-10-19 16:09:06 +00:00
"sz = 0x%jx tsz = %d tid = %d mas1 = 0x%08x "
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
"mas2(va) = 0x%"PRI0ptrX" mas3(pa) = 0x%08x mas7 = 0x%08x\n",
powerpc/booke pmap: Fix printf format type warnings
2019-10-19 16:09:06 +00:00
i, desc, as, (uintmax_t)size, tsize, tid, mas1, mas2, mas3, mas7);
powerpc/booke: Turn tlb*_print_tlbentries() into 'show tlb*' DDB commands debugf() is unnecessary for the TLB printing functions, as they're only intended to be used from ddb. Instead, make them full DDB 'show' commands, so now it can be written as 'show tlb1' and 'show tlb0' instead of calling the function, hoping DEBUG has been defined.
2018-10-22 00:21:27 +00:00
}
DB_SHOW_COMMAND(tlb0, tlb0_print_tlbentries)
{
uint32_t mas0, mas1, mas3, mas7;
#ifdef __powerpc64__
uint64_t mas2;
#else
uint32_t mas2;
#endif
int entryidx, way, idx;
printf("TLB0 entries:\n");
for (way = 0; way < TLB0_WAYS; way ++)
for (entryidx = 0; entryidx < TLB0_ENTRIES_PER_WAY; entryidx++) {
mas0 = MAS0_TLBSEL(0) | MAS0_ESEL(way);
mtspr(SPR_MAS0, mas0);
mas2 = entryidx << MAS2_TLB0_ENTRY_IDX_SHIFT;
mtspr(SPR_MAS2, mas2);
__asm __volatile("isync; tlbre");
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
idx = tlb0_tableidx(mas2, way);
tlb_print_entry(idx, mas1, mas2, mas3, mas7);
}
}
/*
* Print out contents of the MAS registers for each TLB1 entry
*/
DB_SHOW_COMMAND(tlb1, tlb1_print_tlbentries)
{
uint32_t mas0, mas1, mas3, mas7;
#ifdef __powerpc64__
uint64_t mas2;
#else
uint32_t mas2;
#endif
int i;
printf("TLB1 entries:\n");
for (i = 0; i < TLB1_ENTRIES; i++) {
mas0 = MAS0_TLBSEL(1) | MAS0_ESEL(i);
mtspr(SPR_MAS0, mas0);
__asm __volatile("isync; tlbre");
mas1 = mfspr(SPR_MAS1);
mas2 = mfspr(SPR_MAS2);
mas3 = mfspr(SPR_MAS3);
mas7 = mfspr(SPR_MAS7);
tlb_print_entry(i, mas1, mas2, mas3, mas7);
}
}
#endif
Reference in New Issue Copy Permalink